GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
HU
HunyuanOCR
Hunyuanocr · ggml-org
Params539M
Context32K
Best fitBF16 2.4 GB
MO
moondream2
Moondream · ggml-org
Params1.42B
Context2K
Best fitF16 5.0 GB
QW
Qwen3 4B Instruct 2507 Polaris Alpha Distill Heretic Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
IM
Z-Image-Engineer-V6-SMART-DoRA-StyleBlendV9-25V17-75
Image · BennyDaBall
Params4B
Context40K
Best fitQ8_0 6.0 GB
RW
Rwkv 6 World 7b
Rwkv · latestissue
Params7.64B
Context1024K
Best fitQ6_K 7.8 GB
KE
Kev 9B
Kev · ggml-org
Params9B
Context256K
Best fitQ4_K_M 7.9 GB
HO
K2-Horizon-32B
Horizon · bartowski
Params34.8B
Context512K
Q4_K_M22.0 GB
QW
Huihui Qwen3.5 0.8B abliterated claude mythos distilled 25k
Qwen · azminawwar
Params773M
Context256K
Best fitBF16 2.5 GB
QW
Qwen3.5 27B Ultra Uncensored Heretic v2
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen3.8 Flash Next Uncensored UD Q4 K XL
Qwen · NeosQ
Params177B
Context256K
Best fitQ4_K_XL 5.5 GB
LL
tap-m_luna-ai-llama2-uncensored
Llama · TheBloke
Params6.74B
Context2K
Best fitQ3_K_S 7.7 GB
GR
Granite-4.0-H-Tiny
Granite · unsloth
Params6.94B
Context1024K
Best fitQ5_K_M 7.5 GB
QW
Qwen3 VL 8B Instruct Uncensored V2
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ5_K_M 7.6 GB
MI
dolphin 2.8 mistral 7b v02
Mistral · lmstudio-community
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
PY
Pyrut 8B Instruct Uncensored i1
Pyrex · mradermacher
Params7.62B
Context32K
Best fitQ6_K 7.2 GB
RO
RoLlama2-7b-Instruct
Rollama · legraphista
Params6.74B
Context4K
Best fitIQ3_M 7.9 GB
AP
Apex 35B V1.0
Apex · bartowski
Params35.5B
Context256K
Q4_K_M22.8 GB
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
PH
phonon 40m es
Phonon · gradium
Params51M
Context—
Smallest0.1 GB
GE
gemma 3 4b it qat q4 0
Gemma · google
Params3.88B
Context—
Best fitQ4_0 4.0 GB
PH
phonon 40m multilingual
Phonon · gradium
Params51M
Context—
Smallest0.1 GB
GE
Gemma 4 E4B Abliterated Uncensored
Gemma · mradermacher
Params7.46B
Context128K
Best fitQ5_K_S 7.6 GB
QW
Raw Uncensored Qwen3 14b Heretic Recovered
Qwen · mradermacher
Params14.8B
Context32K
Best fitIQ3_XXS 7.8 GB
MA
magnum v4 12b
Magnum · mradermacher
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
LL
Llama 3.2 1B Instruct Uncensored
Llama · mradermacher
Params1.24B
Context128K
Best fitQ6_K 1.8 GB
HY
Hy MT2 7B
mradermacher
Params7.5B
Context256K
Best fitQ6_K 7.7 GB
QW
Qwen3.8 27B TURBO Fable Cold Fusion 735 882 Heretic Uncensored GGUF UltraOptimised DSpark MTP
Qwen · Solstice-AI
Params26.9B
Context256K
Best fitQ8_0 2.7 GB
QW
Qwen3VL 8B Uncensored HauhauCS Balanced
Qwen · HauhauCS
Params8.2B
Context256K
Best fitQ4_K_M 6.7 GB
NE
Nex N2 Mini
Nex · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
UM
BojroPromptMaster uncensored v2 8B
Umbral · bojrodev
Params8B
Context8K
Best fitQ5_K_M 7.3 GB