IQ4_NL GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 32452 models have a IQ4_NL file. Sizes run from 1M to 1030B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5 9B RolePlay Merged
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Roleplay
QW
MiMo V2.6 Distill Qwen 9B Ablitrated
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool use
NE
Neural 4 ARC 7b
Neural · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
GE
VeriCoder_Gemma12b
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
Coding
DE
deepseek-coder-1.3B-kexer
DeepSeek · bartowski
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
GE
Gemma 4 E4B It SDFT_Heretic_RP
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
Roleplay
AN
AngelSlayer 12B Unslop Mell RPMax DARKNESS v3 i1
Angelslayer · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
QW
Qwythos 9B Claude Mythos Uncensored Abliterated 1M
Qwythos · mradermacher
Params8.95B
Context1024K
Q4_K_M5.6 GB
ReasoningTool use
IN
Januscoder-8B
InternLM · unsloth
Params8.19B
Context32K
Q4_K_M5.0 GB
CodingVision
LL
Selene 1 Mini Llama 3.1 8B
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
DE
FuseO1 DeepSeekR1 QwQ SkyT1 32B Preview
DeepSeek · bartowski
Params32.8B
Context128K
Q4_K_M19.9 GB
Reasoning
LL
Dobby Mini Unhinged Plus Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
UM
CENTAURUS Engine v1 i1
Uma · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Dobby Mini Leashed Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
AP
Apriel-1.5-15B-Thinker
Apriel · unsloth
Params14.4B
Context256K
Q4_K_M8.8 GB
ReasoningVision
LL
Llama 3.1 8B EZO 1.1 it
Llama · mmnga
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 7b Hf
Llama · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QU
QuantumBruins 7B slerp
Quantumbruins · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
MI
Mistral 7B v0.3 Middo Alpaca 4o Mini
Mistral · mradermacher
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
TH
AM Thinking v1
Thinking · bartowski
Params32.8B
Context128K
Q4_K_M19.9 GB
Reasoning
LL
Meta Llama 3.1 8B Instruct
Llama · mmnga
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 mini 4k instruct iMat
Phi · qwp4w3hyb
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
MI
PVD 160k Mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
WI
WizardLM-2-7B-abliterated
Wizardlm · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
SvS LLama 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite Speech 5.0 2b
Granite · mradermacher
Params1.84B
Context16K
Q4_K_M1.1 GB
GGUF
QW
Qwq-32B
Qwen · unsloth
Params32.8B
Context32K
Q4_K_M19.9 GB
Reasoning
OP
knoveleng Open RS3
Open · bartowski
Params1.78B
Context128K
Q4_K_M1.1 GB
GGUF
AS
ASearcher Local 14B
Asearcher · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
Tool use
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
CodingTool use

Other quants

Keep browsing