GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
Index Homura 9B
Index · IndexTeam
Params8.95B
Context256K
Best fitQ3_K_S 5.9 GB
GGUF
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Best fitIQ2_M 5.9 GB
Roleplay
LF
LFM2 350M ENJP MT
Lfm · LiquidAI
Params354M
Context125K
Best fitF32 2.5 GB
GGUF
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Best fitQ3_K_L 6.0 GB
Coding
TI
py007_tinyllama-1.1b-python-v0.1
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
QW
Qwen2.5 1.5B Instruct
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
GGUF
GE
NVIDIA GEM-X SOMA regression denoiser
Gem · LocalAI-io
Params44M
Context120
Best fitF32 4.1 GB
GGUF
MI
Mistral Small 3.1 DRAFT 0.5B
Mistral · bartowski
Params593M
Context32K
Best fitBF16 1.8 GB
GGUF
QW
Qwen3 4b Deslop v4
Qwen · erikkannike
Params4B
Context256K
Best fitQ8_0 6.0 GB
GGUF
LE
LeeMerge 7B slerp
Leemerge · mradermacher
Params7.24B
Context32K
Best fitQ4_K_S 5.9 GB
GGUF
QW
Qwen2-0.5B-Instruct
Qwen · legraphista
Params494M
Context32K
Best fitBF16 1.6 GB
GGUF
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Best fitQ3_K_L 6.0 GB
GGUF
QW
Qwen2.5 Coder 3B
Qwen · ggml-org
Params3.09B
Context32K
Best fitQ8_0 4.1 GB
Coding
PH
calme 2.2 phi3 4b
Phi · MaziyarPanahi
Params3.82B
Context4K
Best fitQ4_K_S 5.9 GB
GGUF
GE
Forge 1 Gemma 4 E2B
Gemma · YoSaNoo
Params4.63B
Context128K
Best fitQ8_0 6.0 GB
CodingReasoningTool use
GR
Granite-4.0-H-1B
Granite · unsloth
Params1.46B
Context1024K
Best fitBF16 5.4 GB
GGUF
ST
stable-code-3b
Stable · mradermacher
Params2.8B
Context16K
Best fitQ6_K 5.5 GB
Coding
MI
Ministral 3 14B Reasoning 2512
Mistral · bartowski
Params13.5B
Context256K
Q4_K_M8.2 GB
ReasoningVision
GP
Gpt2 Medium
Gpt · mradermacher
Params406M
Context1K
Best fitF16 2.1 GB
GGUF
PY
Pythia 6.9B Instruct v1 Merged
Pythia · bartowski
Params6.86B
Context2K
Q4_K_M4.7 GB
GGUF
GE
google gemma 2b it
Gemma · RichardErkhov
Params2.51B
Context8K
Best fitQ6_K 2.7 GB
GGUF
PL
Plamo 3 Nict 2b Base
Plamo · mmnga-o
Params2.6B
Context4K
Best fitQ8_0 3.7 GB
GGUF
GP
GPT2-774M-CINDER-SHOW-MULTI-CHAT
Gpt · RichardErkhov
Params838M
Context1K
Best fitQ8_0 2.9 GB
GGUF
QW
Qwen3-Vl-2B-Instruct-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitBF16 4.9 GB
Vision
DE
DeepCoder 1.5B Preview
Deepcoder · bartowski
Params1.78B
Context128K
Best fitBF16 4.3 GB
Coding
MA
MN 12B Mag Mell R1 i1
Mag · mradermacher
Params12.2B
Context1000K
Best fitIQ2_S 6.0 GB
Reasoning
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Best fitIQ1_M 5.7 GB
ReasoningRoleplayTool use
MA
Maincoder 1B
Maincoder · mradermacher
Params1.03B
Context2K
Best fitF16 3.0 GB
Coding
QW
hermes qwen3.5 35b a3b
Qwen · DJLougen
Params34.7B
Context256K
Best fitIQ3_XXS 4.7 GB
Tool use

Other memory sizes

Keep browsing