GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
SM
TwIL LM3
SmolLM · webAI-Official
Params3.08B
Context64K
Best fitQ8_0 4.4 GB
QW
Qwen2.5-Vl-7B-Instruct
Qwen · unsloth
Params7.62B
Context125K
Best fitQ4_K_XL 5.8 GB
GP
gpt-oss-120b
gpt-oss · ggml-org
Params800M
Context128K
Best fitBF16 2.1 GB
LI
Ling 3.0 Tiny
Ling · bloomer010
Params7.89B
Context128K
Q4_K_M4.8 GB
QW
qwen2.5-3b-instruct
Qwen · Qwen
Params3.4B
Context32K
Best fitQ8_0 4.4 GB
SP
spark s1 4b v6
Spark · abhishek085
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen2.5 Coder 1.5B Instruct GGUF
Qwen · Qwen
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
QW
Qwen2.5 7B Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ4_K_M 5.7 GB
MI
s1 mini
Mini · superwhisper
Params752M
Context40K
Best fitF16 2.9 GB
SP
Spark X2.5 1.7B
Spark · XHToken
Params1.71B
Context1024K
Q4_K_M1.1 GB
QW
Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · HauhauCS
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
QW
Qwen3-1.7B
Qwen · unsloth
Params1.72B
Context40K
Best fitBF16 4.9 GB
XX
T5 V1_1 Xxl
Xxl · city96
Params4.76B
Context512
Best fitQ3_K_M 6.0 GB
QW
Qwen3 0.6B Instruct
Qwen · Qwen
Params596M
Context40K
Best fitQ8_0 2.1 GB
GE
gemma 4 12B it qat
Gemma · unsloth
Params440M
Context256K
Best fitBF16 2.4 GB
TE
Tess-4.5-27B
Tess · migtissera
Params27B
Context256K
Best fitQ8_0 5.8 GB
QW
Parable Qwen3 4B Claude Fable 5
Qwen · AnkitAI
Params4B
Context40K
Best fitQ8_0 6.0 GB
MI
MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
QW
Huihui Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
BG
bge small en v1.5
BGE · unsloth
Params33M
Context512
Best fitF16 0.7 GB
MU
Muse Glimmer 30B
Muse · meta-models
Params27.9B
Context128K
Best fitQ4_K_M 2.3 GB
QW
Qwen3-Vl-8B-Instruct
Qwen · unsloth
Params8.19B
Context256K
Best fitQ3_K_XL 6.0 GB
MX
mxbai-embed-large-v1
Mxbai · ChristianAzinn
Params334M
Context512
Q4_K_M0.2 GB
CA
Carbon 3B
Carbon · HuggingFaceBio
Params3.45B
Context32K
Best fitQ8_0 4.6 GB
MI
MiniCPM5 1B Claude Opus Fable5 Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Best fitF16 2.9 GB
EX
Experiment26 7B passthrough 10slice
Experiment · babybirdprd
Params7B
Context—
Best fitQ4_K_M 0.5 GB
QW
Qwen3.5 9B
Qwen · bartowski
Params9.2B
Context256K
Best fitQ2_K 5.9 GB
LF
LFM2.5 24B A2B Uncensored
Lfm · AtomicChat
Params24B
Context128K
Q4_K_M15.4 GB
QW
Qwen3Vl 2b Instruct
Qwen · Qwen
Params1.72B
Context256K
Best fitF16 4.9 GB
SP
Spark X2.6 4B
Spark · XHToken
Params4.45B
Context1024K
Best fitQ4_K_M 5.3 GB