GGUF models that fit in 8 GB
151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
FR
FrogNano 4B 2609 Heretic
Frognano · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
EX
Experiment26 7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
NO
Noema 1.5 4B Deep
Noema · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen3.5 4B Anon Reasoning v2
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
OR
Ornith 1.0 35B Uncensored Heretic
Ornith · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
PY
Pythia 160m Deduped
Pythia · mradermacher
Params162M
Context2K
Best fitIQ1_S 0.9 GB
SU
SummLlama3.1 8B
Summllama · mradermacher
Params8.03B
Context128K
Best fitQ5_K_M 7.3 GB
LL
Llama 3.2 1B Instruct Abliterated
Llama · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
GE
Gemma 2 Ataraxy 9B
Gemma · bartowski
Params9.24B
Context8K
Best fitIQ3_M 7.8 GB
FI
Fin R1
Fin · mradermacher
Params7.6B
Context32K
Best fitQ6_K 7.2 GB
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
LL
Llm Jp 3 3.7b
Llm · mmnga
Params3.78B
Context4K
Best fitQ8_0 7.3 GB
AP
Apertus 70B Instruct 2509 Abliterated
Apertus · mradermacher
Params70.6B
Context64K
Q4_K_M43.7 GB
QW
Huihui Qwen3.5 4B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
PH
Phi 4 Reasoning plus_Nemotron Reasoning 0803_SFT GRPO
Phi · mradermacher
Params14.7B
Context32K
Best fitIQ3_XXS 8.0 GB
IN
InternLM2
InternLM · legraphista
Params1.89B
Context8K
Best fitBF16 5.1 GB
QW
imajev 4b
Qwen · mindchain
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
HU
Huihui OmniCoder 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
QW
Qwen Image Edit Rapid AIO NSFW v23 NVFP4
Qwen · FreedomAISVR
Params20.4B
Context32K
Smallest4.1 GB
QW
SummOrchestra Qwen3 8B GRPO BRLP SAMSUM
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
MI
MiniCPM5 2B DSpark
MiniCPM · magnitudedev
Params324M
Context128K
Smallest0.7 GB
LF
LFM3-VL-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
LL
Huihui Mistral Small 3.2 24B Instruct 2506 Abliterated Llamacppfixed
Llama · mradermacher
Params23.6B
Context128K
Best fitIQ1_M 7.6 GB
VI
VibeThinker 3B
Vibethinker · mradermacher
Params3.09B
Context128K
Best fitQ6_K 3.3 GB
QW
Qwen3.5 9B Claude Fable 5xMerge
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
GL
GLM-ASR-Nano-2512
GLM · Mungert
Params1.59B
Context8K
Best fitBF16 4.2 GB
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Best fitQ5_K_M 7.7 GB
KU
Kunokukulemonchini 7b
Kunokukulemonchini · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.7 GB