GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
NO
nomic-embed-text-v1
Nomic · nomic-ai
Params137M
Context2K
Best fitF32 1.3 GB
Embedding
QU
R3GE Quen2.5 7B
Quen · JackFRost0703
Params7.62B
Context128K
Best fitQ4_K_M 5.7 GB
GGUF
QW
Qwen3 4B Instruct Awq
Qwen · electricpipelines
Params4B
Context40K
Best fitQ4_K_M 4.2 GB
GGUF
LL
Llama-3.1-8B-Instruct
Llama · unsloth
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
LF
LFM2 2.6B
Lfm · LiquidAI
Params2.57B
Context125K
Best fitQ8_0 5.2 GB
GGUF
QW
Qwen2.5 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
Vision
QW
qwen2-1_5b-instruct
Qwen · Qwen
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
SM
SmolLM2 1.7B Instruct
SmolLM · bartowski
Params1.71B
Context8K
Best fitF16 5.5 GB
GGUF
QW
Qwen2.5 7b Ins v3
Qwen · bartowski
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
DE
deepseek-ai_deepseek-coder-1.3b-instruct
DeepSeek · TheBloke
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
LL
Llama 3.1 8B Stheno v3.4
Llama · bartowski
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
OR
Ornith 1.5 9B
Ornith · abenzerps
Params8.95B
Context256K
Best fitIQ3_M 6.0 GB
ReasoningTool use
MI
MinerU2.5 Pro 2605 1.2B
Mineru · jinzhenj
Params494M
Context8K
Best fitQ8_0 1.1 GB
Vision
LL
Anubis Mini 8B v1
Llama · TheDrummer
Params8B
Context128K
Best fitQ3_K_M 5.6 GB
GGUF
QW
speakoflow mini
Qwen · SpeakoFlow
Params773M
Context256K
Best fitBF16 2.5 GB
GGUF
QW
Qwen2.5-Omni-7B
Qwen · unsloth
Params7.62B
Context32K
Best fitQ4_K_XL 5.8 GB
Vision
MI
vllm-d3-mini
Mini · ggml-org
Params4.84B
Context256K
Best fitQ4_K_M 5.3 GB
Vision
QW
Qwen2.5 3B Instruct Abliterated
Qwen · mradermacher
Params3.4B
Context32K
Best fitQ8_0 4.4 GB
GGUF
QW
Qwen3 VL 8B Instruct Abliterated v2.0
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
Tool useVision
QW
Qwen3.5 9B Claude 4.6 HighIQ INSTRUCT HERETIC UNCENSORED
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
ReasoningRoleplayTool use
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
QW
Mica v0.1 4B
Qwen · sky7350
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
GGUF
UN
Undi95 Emerhyst 20B
Undi · RichardErkhov
Params920M
Context—
Best fitIQ3_XS 4.1 GB
GGUF
MI
Dolphin Mistral 24B Venice Edition
Mistral · douganger
Params23.6B
Context128K
Q4_K_M14.3 GB
Vision
MI
Ministral-3-14B-Instruct-2512
Mistral · unsloth
Params13.5B
Context256K
Best fitIQ2_XXS 5.9 GB
Vision
BG
bge-small-zh-v1.5
BGE · CompendiumLabs
Params24M
Context512
Best fitF32 0.7 GB
Embedding
OP
opendecider small
Opendecider · manjunathshiva
Params4.02B
Context256K
Best fitQ8_0 6.0 GB
Tool use
QW
LocalAI functioncall qwen2.5 7b v0.5 Q4 K M
Qwen · LocalAI-io
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
GGUF
JA
Jan-v2-VL-high
Jan · janhq
Params8.2B
Context256K
Best fitQ3_K_M 5.8 GB
Vision
ME
Medgemma-1.5-4B-It
Medgemma · unsloth
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
ReasoningVision

Other memory sizes

Keep browsing