GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 0.6B Heretic Heresy Edition
Qwen · mradermacher
Params596M
Context40K
Best fitF16 2.6 GB
QW
Qwen3.5 4B Uncensored LocalAI
Qwen · weareapexcreators
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
XY
XYZ Aquila Mini
Xyz · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
HU
Hunyuan MT 7B
Hunyuan · mradermacher
Params7.5B
Context256K
Best fitQ4_K_S 6.0 GB
LF
LFM3 1.7B
Lfm · LiquidAI
Params1.73B
Context256K
Best fitBF16 5.6 GB
AR
BAAI AREX 2
Arex · bartowski
Params27B
Context256K
Q4_K_M17.2 GB
BA
BarryFutureman NeuralTurdusVariant1 7B
Barryfutureman · RichardErkhov
Params7B
Context—
Best fitIQ3_S 4.1 GB
QW
Qwen3.6 35B A3B Uncensored Heretic
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
MiniCPM V 4_6
MiniCPM · openbmb
Params752M
Context256K
Best fitF16 2.4 GB
GE
Gemma-3-4B-It-Qat
Gemma · unsloth
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
QW
Qwen3.5-0.8B
Qwen · weareapexcreators
Params773M
Context256K
Best fitQ4_K_M 1.5 GB
QW
Qwen2 VL 2B Instruct
Qwen · bartowski
Params1.54B
Context32K
Best fitF16 3.8 GB
RO
L3.2 Rogue Creative Instruct Uncensored Abliterated 7B
Rogue · DavidAU
Params7.53B
Context128K
Best fitQ2_K 5.7 GB
FL
Flint-40B-A2.5B-abliterated
Flint · dima6312
Params39.8B
Context256K
Q4_K_M24.2 GB
HO
K2 Horizon 3.7B
Horizon · IFM
Params5.06B
Context512K
Best fitQ6_K 5.9 GB
QW
Huihui Qwen3.5 35B A3B Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Therapy 3.8
Qwen · Verdugie
Params26.9B
Context256K
Q4_K_M16.5 GB
LF
LFM2.5-350M
Lfm · weareapexcreators
Params354M
Context125K
Best fitQ8_0 1.4 GB
QW
Qwen3 VL 8B Instruct Abliterated v2
Qwen · prithivMLmods
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
ST
StartLux-Decision-v1.1-4B-Instant
Startlux · startlux-models
Params4.2B
Context256K
Best fitQ4_K_M 4.3 GB
QW
Qwen3.8-27B-DFlash2
Qwen · perplexity-ai
Params27.3B
Context256K
Best fitQ8_0 3.8 GB
FL
RecursiveSI flint
Flint · bartowski
Params39.8B
Context256K
Q4_K_M25.3 GB
LF
LFM2 VL 1.6B
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
ST
L3-8B-Stheno-v3.2
Stheno · Lewdiculous
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
LL
VAGOsolutions Llama 3 SauerkrautLM 70b Instruct
Llama · RichardErkhov
Params6B
Context—
Best fitQ3_K_L 4.0 GB
PA
PaddleOCR VL 1.5
Paddleocr · PaddlePaddle
Params467M
Context128K
Smallest0.9 GB
GE
gemma-2b-it
Gemma · lmstudio-ai
Params2.51B
Context8K
Best fitQ8_0 3.3 GB
LF
LFM2.5 230M LocalAI
Lfm · weareapexcreators
Params230M
Context125K
Best fitQ8_0 1.2 GB
JI
jina reranker v1 tiny en
Jina · mradermacher
Params33M
Context8K
Best fitQ2_K 0.6 GB
MI
Mistral Small 4 119B 2603
Mistral · bartowski
Params119B
Context1024K
Q4_K_M72.6 GB