GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
HO
K2-Horizon-MoVA-36B-A4B
Horizon · bartowski
Params37.4B
Context512K
Q4_K_M22.7 GB
QW
Qwen3 VL 4B Instruct
Qwen · bartowski
Params4B
Context256K
Best fitQ8_0 6.0 GB
JU
Julia-1
Julia · ggml-org
Params144M
Context8K
Best fitBF16 1.1 GB
GE
gemma-4-E4B-it
Gemma · TheStageAI
Params7.46B
Context128K
Best fitQ4_K_S 5.7 GB
AV
ava lfm25 toolcall
Ava · vitorcalvi
Params1.17B
Context125K
Best fitQ4_0 2.3 GB
GL
GLM 5.3 Flash abliterated Maya M
GLM · alesha-pro
Params370B
Context64K
Smallest0.0 GB
QW
Qwen3 4B Abliterated
Qwen · bartowski
Params4B
Context40K
Best fitQ8_0 6.0 GB
SK
Skyfall 31B v4.2
Skyfall · bartowski
Params31.4B
Context128K
Q4_K_M19.3 GB
NS
NSFW flash
Nsfw · UnfilteredAI
Params2.24B
Context4K
Best fitQ4_K_M 4.0 GB
QW
Qwen3.8 35B A3B Distill
Qwen · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
ST
StartLux-Decision-v1.1-4B-AutoThink
Startlux · startlux-models
Params4.2B
Context256K
Best fitQ4_K_M 4.3 GB
MI
MiniCPM5 2B heretic abliterated
MiniCPM · Abiray
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
EM
Embeddinggemma 300m Qat Q4_0 Unquantized
Embeddinggemma · unsloth
Params303M
Context2K
Best fitF32 1.9 GB
QW
Qwen3.8-27B
Qwen · kingjones777
Params26.9B
Context256K
Best fitQ8_0 5.8 GB
QW
Qwen2.5 3B Instruct Uncensored
Qwen · mradermacher
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
MI
Ministral-3-3B-Reasoning-2512
Mistral · unsloth
Params3.43B
Context256K
Best fitQ8_K_XL 5.9 GB
MI
Ministral 8B Instruct 2410 TEST
Mistral · bartowski
Params8B
Context32K
Best fitQ3_K_L 6.0 GB
NU
NuExtract3
Nuextract · numind
Params4.33B
Context256K
Best fitQ6_K 5.2 GB
PA
PaddleOCR VL 1.6
Paddleocr · LunarOilRig
Params467M
Context128K
Best fitQ4_K_M 1.0 GB
HY
Hy-Mt2-7B
unsloth
Params7.5B
Context256K
Best fitQ4_0 6.0 GB
QW
Qwen3.6 35B A3B MTP APEX I MiniPlus V2.1 Abliterated
Qwen · IsValorum
Params34.7B
Context256K
Best fitQ8_0 3.2 GB
GE
Gemmasutra Mini 2B v1
Gemma · TheDrummer
Params2.61B
Context8K
Best fitQ8_0 4.2 GB
QW
Qwen3-Coder-30B-A3B-Instruct-1M
Qwen · unsloth
Params30.5B
Context1024K
Q4_K_M18.6 GB
QW
Huihui Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
QW
Huihui Qwen3.8 27B Coder390 EfficientThink Abliterated
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
PH
phi 4
Phi · microsoft
Params14.7B
Context16K
Best fitTQ1_0 5.8 GB
AP
APUS Reflex 9B
Apus · apus-ailab
Params8.95B
Context256K
Q4_K_M5.6 GB
MI
mistralai_mistral-7b-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
MI
Mistral Nemo Instruct 2407 12B Thinking M Claude Opus High Reasoning
Mistral · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
GR
TwIL LM3 Pro
Granite · webAI-Official
Params3.66B
Context128K
Best fitQ8_0 5.1 GB