GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Strand-Rust-Coder-14B-v1
Qwen · Mungert
Params14.8B
Context32K
Q4_K_M9.3 GB
Coding
QW
Fathom-Search-4B
Qwen · Mungert
Params4B
Context40K
Best fitQ8_0 6.0 GB
GGUF
SK
SkyJM Edit 9B
Skyjm · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
Vision
HU
Huihui3.5 67B A3B Abliterated
Huihui · mradermacher
Params67B
Context256K
Q4_K_M40.8 GB
GGUF
HO
Sakura K2 Horizon 3.7B
Horizon · webmp3
Params5.06B
Context512K
Smallest2.2 GB
GGUF
MI
Nous Hermes 2 Mistral 7B DPO
Mistral · NousResearch
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Tool use
MI
Ministral 3 14B Reasoning 2512 Absolute Heresy
Mistral · mradermacher
Params13.5B
Context256K
Best fitIQ2_XXS 5.8 GB
Reasoning
QW
Qwen2.5 Coder 0.5B
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB
Coding
QW
Qwen3 8B Abliterated
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
GGUF
LL
Llama3 8B Chinese Chat
Llama · Monor
Params8B
Context8K
Best fitQ3_K_L 5.9 GB
GGUF
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Best fitQ6_K 5.1 GB
ReasoningTool use
TR
bluenipples_trion-m-7b
Trion · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
GR
Granite 3b Code Base
Granite · RichardErkhov
Params3.48B
Context2K
Best fitQ6_K 6.0 GB
Coding
NE
neo_7b_instruct_v0.1
Neo · legraphista
Params7.79B
Context8K
Best fitIQ2_M 6.0 GB
GGUF
QW
Qwen3 0.6B Heretic Abliterated Uncensored
Qwen · Andycurrent
Params596M
Context40K
Q4_K_M0.4 GB
GGUF
QW
rombos Replete Coder Qwen2 1.5b i1
Qwen · mradermacher
Params1.54B
Context128K
Best fitQ6_K 2.0 GB
CodingTool use
YI
Mahou-1.3-yi-9B
Yi · mradermacher
Params8.83B
Context4K
Best fitQ3_K_L 6.0 GB
GGUF
AD
AdaGuard 8B
Adaguard · mradermacher
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
GGUF
FA
Falcon3 1B Instruct
Falcon · bartowski
Params1.67B
Context8K
Best fitF16 4.4 GB
GGUF
MI
Ministral-3-8B-Nymphaea-RP
Mistral · mradermacher
Params8.49B
Context256K
Best fitQ3_K_M 5.9 GB
RoleplayTool useVision
MI
Mistral 7B Instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
AR
Arcee-Agent
Arcee · crusoeai
Params7.62B
Context128K
Best fitQ4_1 5.8 GB
GGUF
QW
Qwen3.6 35B A3B REAM 160 ru agent
Qwen · WaveCut
Params23.1B
Context256K
Best fitQ1_0 5.4 GB
GGUF
AY
aya-23-8B
Aya · legraphista
Params8B
Context8K
Best fitQ3_K 5.8 GB
GGUF
AR
ArrowPro-7B-KillerWhale
Arrowpro · mmnga
Params7.24B
Context4K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Qwen3 32B VL GLM 4.7 Flash HI16 Heretic Uncensored Thinking
Qwen · mradermacher
Params32.8B
Context256K
Q4_K_M19.8 GB
ReasoningRoleplayVision
OL
OLMoE 1B 7B 0924 Instruct
OLMo · mradermacher
Params6.9B
Context4K
Best fitQ4_K_M 5.8 GB
GGUF
QW
Qwen3.5 27B Abliterated Ko
Qwen · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
ReasoningTool use
GR
Granite 3b Code Instruct 2k
Granite · mradermacher
Params3.48B
Context2K
Best fitQ6_K 6.0 GB
CodingTool use
QW
Qwen3-8B
Qwen · ggml-org
Params8.19B
Context40K
Best fitBF16 2.9 GB
GGUF

Other memory sizes

Keep browsing