GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
MiniCPM4.1 8B
MiniCPM · openbmb
Params8.2B
Context64K
Best fitQ4_K_M 5.7 GB
GGUF
NE
Nehuronion Dvt 9b
Nehuronion · mradermacher
Params9.24B
Context8K
Best fitIQ1_M 5.9 GB
GGUF
LL
models--Groq--Llama-3-Groq-8B-Tool-Use
Llama · MaziyarPanahi
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
Tool use
QW
Qwen3.8-27B-DFlash2
Qwen · magnitudedev
Params1.92B
Context256K
Best fitQ8_0 2.7 GB
GGUF
OR
Ornith 1.5 122B
Ornith · mradermacher
Params125B
Context256K
Q4_K_M75.8 GB
ReasoningTool use
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Best fitQ6_K 6.0 GB
Coding
QW
Qwen3.6 35B A3B Uncensored Genesis Final Safetensors
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
ReasoningTool useVision
QW
qwen3loop 0.6b sft deep supervision v1
Qwen · Lordnyx
Params596M
Context40K
Best fitF16 4.5 GB
ReasoningVision
DE
deepseek r1
DeepSeek · cortexso
Params70.6B
Context128K
Best fitQ4_K_M 5.7 GB
Reasoning
LL
Llama 3.1 WhiteRabbitNeo 2 8B
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
ZE
Zeta 2.1
Zeta · mradermacher
Params8.25B
Context32K
Best fitQ3_K_L 6.0 GB
GGUF
BR
Breeze-13B-32k-Instruct-v1_0
Breeze · mradermacher
Params12.7B
Context32K
Best fitIQ2_XXS 5.9 GB
GGUF
QW
Qwen3 14B Abliterated
Qwen · mradermacher
Params14.8B
Context40K
Best fitIQ1_M 5.7 GB
GGUF
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated Heretic
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
Reasoning
OP
openPangu Embedded 7B V1.1
Openpangu · mradermacher
Params8.03B
Context32K
Best fitQ3_K_L 6.0 GB
Embedding
TH
TherapyBeagle 11B v1 i1
Therapybeagle · mradermacher
Params10.7B
Context32K
Best fitIQ2_M 5.8 GB
GGUF
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Best fitQ4_1 5.7 GB
GGUF
EC
EchoVLM_v2
Echovlm · mradermacher
Params8.95B
Context256K
Best fitIQ3_S 6.0 GB
ReasoningTool useVision
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
GPT neo Custom 7B
Qwen · Szymon-Wachelko
Params7.6B
Context32K
Best fitQ4_K_M 5.7 GB
Coding
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Best fitQ3_K_L 6.0 GB
GGUF
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Best fitQ4_0 6.0 GB
GGUF
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · FrosT-LyRo
Params39.1B
Context256K
Q4_K_M23.9 GB
CodingReasoningRoleplay
QW
Qwen3-Vl-2B-Thinking-1M
Qwen · unsloth
Params1.72B
Context977K
Best fitBF16 4.9 GB
ReasoningVision
CA
CarbonVillain En 10.7B v4
Carbonvillain · mradermacher
Params10.7B
Context4K
Best fitIQ2_M 5.8 GB
GGUF
LF
LMF 2.5 2B Code
Lfm · bunnycore
Params2.7B
Context128K
Best fitQ5_K_M 4.5 GB
Coding
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
QW
Qwen3.8 9B Function Calling xLAM Unsloth
Qwen · ermiaazarkhalili
Params9.2B
Context256K
Best fitQ2_K 5.5 GB
ReasoningTool use
TH
Thaki ASR KO Dialects v1 i1
Thaki · mradermacher
Params1.72B
Context64K
Best fitQ6_K 2.9 GB
GGUF

Other memory sizes

Keep browsing