GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
VI
Vietnamese 4B Chat Abliterated
Vietnamese · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.1 GB
GGUF
CA
calme-2.3-legalkit-8b
Calme · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
GGUF
QW
Liberation Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
ReasoningTool use
HY
HY MT1.5 1.8B
mradermacher
Params1.79B
Context256K
Best fitF16 4.6 GB
GGUF
PR
Prashna 7b
Prashna · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
ReasoningTool use
MA
Mario Html v2 9B
Mario · bartowski
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
ReasoningTool useVision
TE
q-bert_terminis-7b
Terminis · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
KA
KAT Ornith HERO 35B A3B Fusion
Kat · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
ReasoningTool use
DA
DataPilot ArrowPro 7B RobinHood
Datapilot · mmnga
Params7.24B
Context4K
Best fitQ4_K_M 5.9 GB
GGUF
HO
Honey Yuzu 13B
Honey · mradermacher
Params12.5B
Context8K
Best fitIQ2_XXS 5.8 GB
Roleplay
MO
Mox Tiny 1
Mox · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
ReasoningRoleplay
QW
Qwen3.5 0.8B Finetuned Lora Uncensored
Qwen · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
Vision
QW
Qwen2.5 0.5B Instruct heretic
Qwen · saidutta69
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
MI
monology_openinstruct-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
QW
Huihui Qwen3 VL 8B Instruct Abliterated
Qwen · mradermacher
Params8.19B
Context256K
Best fitQ3_K_M 5.8 GB
Vision
QW
Qwen3 VL 4B Instruct Abliterated
Qwen · mradermacher
Params4B
Context256K
Best fitQ8_0 6.0 GB
Vision
QW
flux2 klein 4B uncensored text encoder
Qwen · Cordux
Params4B
Context—
Best fitQ4_0 3.2 GB
Coding
QW
Qwen3.5 4B Instruct
Qwen · openresearchtools
Params4.21B
Context256K
Best fitQ4_K_M 4.3 GB
Vision
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Best fitQ8_0 6.0 GB
GGUF
GE
Gemma SEA LION v4 4B VL
Gemma · aisingapore
Params3.88B
Context128K
Best fitQ8_0 5.8 GB
Vision
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Best fitQ6_K 5.4 GB
Coding
MI
MinerU2.5 Pro 2605 1.2B
Mineru · mradermacher
Params494M
Context8K
Best fitF16 1.6 GB
Vision
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Best fitQ4_K 5.7 GB
GGUF
CL
Clef Flash
Clef · taurusduan
Params9.1B
Context256K
Best fitQ2_K 5.8 GB
CodingVision
LF
LFM2.5 1.2B Instruct DSpark Draft v1
Lfm · LiquidAI
Params296M
Context125K
Best fitF16 1.2 GB
GGUF
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Best fitQ3_K_S 6.0 GB
ReasoningTool use
MI
Mixtral AI CyberTron Ultra
Mistral · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
CodingReasoningRoleplay
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
GGUF
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ4_K_M 5.7 GB
Reasoning
OR
Ornith 1.5 397B
Ornith · bartowski
Params397B
Context256K
Q4_K_M241.8 GB
Vision

Other memory sizes

Keep browsing