GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
VI
Vietnamese 4B Chat Abliterated
Vietnamese · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
CA
calme-2.3-legalkit-8b
Calme · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Liberation Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
HY
HY MT1.5 1.8B
mradermacher
Params1.79B
Context256K
Best fitF16 4.6 GB
PR
Prashna 7b
Prashna · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
MA
Mario Html v2 9B
Mario · bartowski
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
TE
q-bert_terminis-7b
Terminis · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
KA
KAT Ornith HERO 35B A3B Fusion
Kat · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
DA
DataPilot ArrowPro 7B RobinHood
Datapilot · mmnga
Params7.24B
Context4K
Best fitQ4_K_M 5.9 GB
HO
Honey Yuzu 13B
Honey · mradermacher
Params12.5B
Context8K
Best fitIQ2_XXS 5.8 GB
MO
Mox Tiny 1
Mox · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
QW
Qwen3.5 0.8B Finetuned Lora Uncensored
Qwen · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
QW
Qwen2.5 0.5B Instruct heretic
Qwen · saidutta69
Params494M
Context32K
Q4_K_M0.4 GB
MI
monology_openinstruct-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
QW
Qwen3 VL 4B Instruct Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
QW
flux2 klein 4B uncensored text encoder
Qwen · Cordux
Params4B
Context—
Best fitQ4_0 3.2 GB
QW
Qwen3.5 4B Instruct
Qwen · openresearchtools
Params4.21B
Context256K
Best fitQ4_K_M 4.3 GB
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Best fitQ8_0 6.0 GB
GE
Gemma SEA LION v4 4B VL
Gemma · aisingapore
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Best fitQ6_K 5.4 GB
MI
MinerU2.5 Pro 2605 1.2B
Mineru · mradermacher
Params494M
Context8K
Best fitF16 1.6 GB
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Best fitQ4_K 5.7 GB
CL
Clef Flash
Clef · taurusduan
Params9.1B
Context256K
Best fitQ2_K 5.8 GB
LF
LFM2.5 1.2B Instruct DSpark Draft v1
Lfm · LiquidAI
Params296M
Context125K
Best fitF16 1.2 GB
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
MI
Mixtral AI CyberTron Ultra
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ4_K_M 5.7 GB
OR
Ornith 1.5 397B
Ornith · bartowski
Params397B
Context256K
Q4_K_M241.8 GB