GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
IN
Index Homura 9B
Index · IndexTeam
Params8.95B
Context256K
Best fitQ3_K_S 5.9 GB
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Best fitIQ2_M 5.9 GB
LF
LFM2 350M ENJP MT
Lfm · LiquidAI
Params354M
Context125K
Best fitF32 2.5 GB
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Best fitQ3_K_L 6.0 GB
TI
py007_tinyllama-1.1b-python-v0.1
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
QW
Qwen2.5 1.5B Instruct
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GE
NVIDIA GEM-X SOMA regression denoiser
Gem · LocalAI-io
Params44M
Context120
Best fitF32 4.1 GB
MI
Mistral Small 3.1 DRAFT 0.5B
Mistral · bartowski
Params593M
Context32K
Best fitBF16 1.8 GB
QW
Qwen3 4b Deslop v4
Qwen · erikkannike
Params4B
Context256K
Best fitQ8_0 6.0 GB
LE
LeeMerge 7B slerp
Leemerge · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
QW
Qwen2-0.5B-Instruct
Qwen · legraphista
Params494M
Context32K
Best fitBF16 1.6 GB
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Best fitQ3_K_L 6.0 GB
QW
Qwen2.5 Coder 3B
Qwen · ggml-org
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
PH
calme 2.2 phi3 4b
Phi · MaziyarPanahi
Params3.82B
Context4K
Best fitQ4_K_S 5.9 GB
GE
Forge 1 Gemma 4 E2B
Gemma · YoSaNoo
Params4.63B
Context128K
Best fitQ8_0 6.0 GB
GR
Granite-4.0-H-1B
Granite · unsloth
Params1.46B
Context1024K
Best fitBF16 5.4 GB
ST
stable-code-3b
Stable · mradermacher
Params2.8B
Context16K
Best fitQ6_K 5.5 GB
MI
Ministral 3 14B Reasoning 2512
Mistral · bartowski
Params13.5B
Context256K
Q4_K_M8.2 GB
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Best fitF16 2.1 GB
PY
Pythia 6.9B Instruct v1 Merged
Pythia · bartowski
Params6.86B
Context2K
Q4_K_M4.7 GB
GE
google gemma 2b it
Gemma · RichardErkhov
Params2.51B
Context8K
Best fitQ6_K 2.7 GB
PL
Plamo 3 Nict 2b Base
Plamo · mmnga-o
Params2.6B
Context4K
Best fitQ8_0 3.7 GB
GP
GPT2-774M-CINDER-SHOW-MULTI-CHAT
Gpt · RichardErkhov
Params838M
Context1K
Best fitQ8_0 2.9 GB
QW
Qwen3-Vl-2B-Instruct-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitBF16 4.9 GB
DE
DeepCoder 1.5B Preview
Deepcoder · bartowski
Params1.78B
Context128K
Best fitBF16 4.3 GB
MA
MN 12B Mag Mell R1 i1
Mag · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Best fitIQ1_M 5.7 GB
MA
Maincoder 1B
Maincoder · mradermacher
Params1.03B
Context2K
Best fitF16 3.0 GB
QW
hermes qwen3.5 35b a3b
Qwen · DJLougen
Params34.7B
Context256K
Best fitIQ3_XXS 4.7 GB