GGUF models that fit in 8 GB

151917 GGUF language models have at least one quant whose estimated memory use is 8 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
TO
TooManyMixRolePlay-7B-Story_V3.5
Toomanymixroleplay · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
Roleplay
LF
LFM2 1.2B Extract
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
GGUF
QW
Qwen3.5 9B Uncensored
Qwen · ccharnkij
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
Vision
QW
Ministral 3 3B Qwen3.8 Max Reasoning Distilled
Qwen · mradermacher
Params3.43B
Context256K
Best fitQ6_K 4.2 GB
ReasoningTool use
MI
nousresearch_yarn-mistral-7b-128k
Mistral · TheBloke
Params7.24B
Context128K
Best fitQ6_K 7.5 GB
GGUF
GE
Gemma 3 1b It Heretic Extreme Uncensored Abliterated
Gemma · mradermacher
Params1B
Context32K
Best fitQ6_K 1.7 GB
GGUF
WE
WeVisOne VL 4B
Wevisone · mradermacher
Params4.42B
Context256K
Best fitQ8_0 6.4 GB
Tool useVision
NO
Noema 1.5 4B Fast
Noema · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
CodingReasoning
HE
Hermes 4 14B
Hermes · gabriellarson
Params14.8B
Context40K
Best fitIQ3_XXS 7.8 GB
ReasoningRoleplayTool use
MI
TowerInstruct Mistral 7B v0.2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.5 GB
GGUF
TO
ToriiGate 0.5
Toriigate · DraconicDragon
Params4.84B
Context256K
Best fitQ8_0 6.7 GB
Vision
LF
LFM2 VL 450M
Lfm · bartowski
Params354M
Context125K
Best fitBF16 1.7 GB
Vision
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Best fitQ5_K 8.0 GB
GGUF
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · chfm
Params8.2B
Context256K
Best fitQ4_K_M 6.7 GB
CodingVision
DA
danube3-4B
Danube · h2oai
Params3.96B
Context8K
Best fitQ8_0 5.5 GB
GGUF
PY
Pythia 410m
Pythia · mradermacher
Params405M
Context2K
Best fitF16 2.1 GB
GGUF
NO
Nock Coder 1.5b
Nock · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
Coding
JE
Jebadiah 27b v2 1
Jebadiah · bartowski
Params27.3B
Context256K
Q4_K_M17.4 GB
Vision
MI
Mistral Nemo 12B ArliAI RPMax v1.2
Mistral · ArliAI
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
GGUF
LL
Humanish Roleplay Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Best fitQ5_K_M 7.3 GB
Roleplay
NE
Neuralgamma 2.5 7B slerp
Neuralgamma · mradermacher
Params7.24B
Context32K
Best fitQ6_K 7.7 GB
Tool use
QW
Qwen3.8 4B Agentic Distill
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
GGUF
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Best fitQ6_K 5.2 GB
GGUF
OR
Ornith 1.5 9B Heretic
Ornith · mradermacher
Params8.95B
Context256K
Best fitQ5_K_M 8.0 GB
Vision
MO
models moved
Models · hfmaster
Params8.03B
Context256K
Best fitQ4_K_M 6.7 GB
Vision
ED
EduMixtral 4x7B i1
Edumixtral · mradermacher
Params24.2B
Context32K
Best fitIQ2_XXS 8.0 GB
GGUF
QW
Qwen3 VL Reranker 2B
Qwen · mradermacher
Params1.72B
Context256K
Best fitF16 4.9 GB
EmbeddingTool useVision
RW
rwkv 6 finch 14b
Rwkv · latestissue
Params14.1B
Context1024K
Best fitIQ3_XXS 7.3 GB
GGUF
TI
tinyllama_tinyllama-1.1b-intermediate-step-1431k-3t
Tinyllama · TheBloke
Params1.1B
Context2K
Best fitQ8_0 1.9 GB
Coding
VI
Vikhr Nemo 12B Instruct R 21 09 24
Vikhr · bartowski
Params12.2B
Context1000K
Best fitQ3_K_M 7.9 GB
GGUF

Other memory sizes

Keep browsing