GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
KI
kitten tts 2
Kitten · KittenML
Params1.73B
Context40K
Smallest1.0 GB
BG
bge m3
BGE · gpustack
Params567M
Context8K
Q4_K_M0.4 GB
ST
L3-8B-Stheno-v3.2
Stheno · bartowski
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.8-Flash-Next
Qwen · ggml-org
Params177B
Context256K
Best fitQ8_0 4.8 GB
LL
Meta Llama 3 8B Instruct
Llama · MaziyarPanahi
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
GE
Gemma 3 4b It
Gemma · lmstudio-community
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
ST
stories15M MOE
Stories · ggml-org
Params8M
Context256
Smallest0.0 GB
QW
Qwen3 Embedding 4B
Qwen · Qwen
Params4B
Context40K
Best fitQ8_0 6.0 GB
NE
NVIDIA Nemotron 3.5 Bolt 74B A9B
Nemotron · unsloth
Params74B
Context256K
Smallest1.0 GB
FA
Jiqing tiny random falcon
Falcon · RichardErkhov
Params—
Context—
Best fitIQ3_M 0.5 GB
QW
Qwen3.5 0.8B
Qwen · ggml-org
Params773M
Context256K
Best fitBF16 2.5 GB
HU
Cyber Tiel Coder 35B A3B
Huihui · peculiar-ragdoll
Params34.7B
Context256K
Q4_K_M22.1 GB
FR
FrontTerrain-HARP-1B
Frontterrain · frontterrain
Params123M
Context512
Best fitF16 0.8 GB
GE
Winnow 12B
Gemma · EldanRing
Params423M
Context256K
Best fitBF16 2.4 GB
IN
Index Translate 9B
Index · IndexTeam
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
SM
SmolLM2 135M Instruct
SmolLM · bartowski
Params135M
Context8K
Best fitF16 1.0 GB
WA
Wald-4B v1.2
Wald · org2ai
Params4.2B
Context256K
Best fitQ6_K 5.0 GB
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Best fitQ8_0 5.8 GB
QW
Qwen3.8 27B pi
Qwen · bytkim
Params27B
Context256K
Best fitBF16 4.5 GB
GE
Gemma 2 2b It
Gemma · bartowski
Params2.61B
Context8K
Best fitQ8_0 4.2 GB
GR
ibm granite granite 20b code instruct
Granite · RichardErkhov
Params1.9B
Context—
Best fitIQ3_S 1.5 GB
LL
Llama 3.2 3B Instruct
Llama · lmstudio-community
Params3.21B
Context128K
Best fitQ8_0 4.9 GB
LL
Llama3.3 8B Instruct Thinking Heretic Uncensored Claude 4.5 Opus High Reasoning
Llama · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.5-2B-Uncensored-HauhauCS-Aggressive
Qwen · HauhauCS
Params1.88B
Context256K
Best fitBF16 4.7 GB
GE
Models Google Gemma 3 4b It
Gemma · MaziyarPanahi
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
KE
Kev 4B
Kev · ggml-org
Params4.2B
Context256K
Best fitQ4_K_M 4.6 GB
QW
Qwen3.5 27B
Qwen · bartowski
Params27.3B
Context256K
Q4_K_M18.0 GB
NE
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B
Nemotron · ggml-org
Params1.66B
Context1024K
Best fitQ8_0 4.9 GB
QW
Qwen3.8 9B Heretic Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB