GGUF models that fit in 8 GB

151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
CodingTool use
FR
FrogNano 4B 2609 Heretic
Frognano · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
GGUF
EX
Experiment26 7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
NO
Noema 1.5 4B Deep
Noema · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
CodingReasoning
QW
Qwen3.5 4B Anon Reasoning v2
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
ReasoningTool use
OR
Ornith 1.0 35B Uncensored Heretic
Ornith · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
PY
Pythia 160m Deduped
Pythia · mradermacher
Params162M
Context2K
Best fitIQ1_S 0.9 GB
GGUF
SU
SummLlama3.1 8B
Summllama · mradermacher
Params8.03B
Context128K
Best fitQ5_K_M 7.3 GB
Tool use
LL
Llama 3.2 1B Instruct Abliterated
Llama · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
GGUF
GE
Gemma 2 Ataraxy 9B
Gemma · bartowski
Params9.24B
Context8K
Best fitIQ3_M 7.8 GB
GGUF
FI
Fin R1
Fin · mradermacher
Params7.6B
Context32K
Best fitQ6_K 7.2 GB
Reasoning
CH
Chocolatine 3B Instruct DPO v1.2
Chocolatine · bartowski
Params3.82B
Context128K
Best fitQ8_0 7.8 GB
GGUF
LL
Llm Jp 3 3.7b
Llm · mmnga
Params3.78B
Context4K
Best fitQ8_0 7.3 GB
GGUF
AP
Apertus 70B Instruct 2509 Abliterated
Apertus · mradermacher
Params70.6B
Context64K
Q4_K_M43.7 GB
GGUF
QW
Huihui Qwen3.5 4B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ8_0 6.1 GB
ReasoningVision
PH
Phi 4 Reasoning plus_Nemotron Reasoning 0803_SFT GRPO
Phi · mradermacher
Params14.7B
Context32K
Best fitIQ3_XXS 8.0 GB
Reasoning
IN
InternLM2
InternLM · legraphista
Params1.89B
Context8K
Best fitBF16 5.1 GB
GGUF
QW
imajev 4b
Qwen · mindchain
Params4.2B
Context256K
Best fitQ8_0 6.1 GB
Vision
HU
Huihui OmniCoder 9B Abliterated
Huihui · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
Coding
QW
Qwen Image Edit Rapid AIO NSFW v23 NVFP4
Qwen · FreedomAISVR
Params20.4B
Context32K
Smallest4.1 GB
GGUF
QW
SummOrchestra Qwen3 8B GRPO BRLP SAMSUM
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ5_K_M 7.6 GB
ReasoningTool use
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ6_K 7.2 GB
Roleplay
MI
MiniCPM5 2B DSpark
MiniCPM · magnitudedev
Params324M
Context128K
Smallest0.7 GB
GGUF
LF
LFM3-VL-2B
Lfm · AtomicChat
Params1.73B
Context125K
Best fitQ8_0 4.0 GB
ReasoningTool useVision
LL
Huihui Mistral Small 3.2 24B Instruct 2506 Abliterated Llamacppfixed
Llama · mradermacher
Params23.6B
Context128K
Best fitIQ1_M 7.6 GB
GGUF
VI
VibeThinker 3B
Vibethinker · mradermacher
Params3.09B
Context128K
Best fitQ6_K 3.3 GB
CodingReasoning
QW
Qwen3.5 9B Claude Fable 5xMerge
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
CodingReasoning
GL
GLM-ASR-Nano-2512
GLM · Mungert
Params1.59B
Context8K
Best fitBF16 4.2 GB
GGUF
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Best fitQ5_K_M 7.7 GB
GGUF
KU
Kunokukulemonchini 7b
Kunokukulemonchini · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.7 GB
GGUF

Other memory sizes

Keep browsing