GGUF models that fit in 6 GB

123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
TR
TranslatePsy AfriSLM 4B
Translatepsy · mradermacher
Params4.84B
Context256K
Best fitQ6_K 5.6 GB
ReasoningTool use
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
ON
OneJev 0.8B
Onejev · bartowski
Params1.01B
Context256K
Best fitBF16 2.9 GB
Vision
HO
K2 Horizon MoVA 36B A4B
Horizon · ngquocvinh
Params37.4B
Context512K
Q4_K_M22.4 GB
GGUF
LL
Llama Leonida
Llama · mradermacher
Params8.2B
Context128K
Best fitQ3_K_L 6.0 GB
Tool use
ME
models--automerger--T3qm7Mergerix-7B
Mergerix · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
ST
StartLux-Decision-v1.1-0.8B-Instant
Startlux · startlux-models
Params752M
Context256K
Best fitQ8_0 1.7 GB
Vision
AN
Andromeda 8B Coder i1
Andromeda · mradermacher
Params8B
Context128K
Best fitIQ4_XS 6.0 GB
CodingTool use
YA
models--automerger--Experiment28Yam-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
ST
stablelm zephyr 3b
Stablelm · TheBloke
Params2.8B
Context4K
Best fitQ6_K 5.5 GB
GGUF
EV
EAN 7B i1
Evan · mradermacher
Params7.62B
Context125K
Best fitQ4_1 5.8 GB
Reasoning
SU
SummLlama3 8B i1
Summllama · mradermacher
Params8.03B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
QW
Trohrbaugh Qwen3.5 122B A10B Heretic
Qwen · mradermacher
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool use
QW
Qwen3-4B-128K
Qwen · unsloth
Params4B
Context128K
Best fitQ8_0 6.0 GB
GGUF
TI
Tiamat-7b-1.1-DPO
Tiamat · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
GGUF
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Best fitIQ4_XS 6.0 GB
GGUF
RO
ROMA 7B
Roma · mradermacher
Params7.62B
Context32K
Best fitQ4_1 5.8 GB
GGUF
GR
GRM 2.6 Plus 0628
Grm · bartowski
Params27.3B
Context256K
Q4_K_M17.8 GB
Vision
SM
SmolVLM Instruct
Smolvlm · ggml-org
Params1.81B
Context16K
Best fitF16 5.7 GB
Vision
KA
KAI 7B v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
Coding
LF
LFM2 1.2B Tool
Lfm · LiquidAI
Params1.17B
Context125K
Best fitF16 3.9 GB
Tool use
MO
Mox Small 1
Mox · mradermacher
Params32.2B
Context64K
Q4_K_M19.5 GB
GGUF
MI
MiniMax M2.1
Minimax · mradermacher
Params230B
Context192K
Q4_K_M138.3 GB
GGUF
HU
Huihui Agents A1 Abliterated
Huihui · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
CodingVision
OP
OpenCoder 1.5B Instruct
Opencoder · lmstudio-community
Params1.91B
Context4K
Best fitQ8_0 4.3 GB
Coding
HO
Holo 3.1 9B
Holo · prithivMLmods
Params8.95B
Context256K
Best fitQ3_K_S 5.8 GB
Vision
MI
Devstral 2 123B Instruct 2512
Mistral · bartowski
Params125B
Context256K
Q4_K_M74.9 GB
Coding
QW
Remnant Qwen3 8b
Qwen · bartowski
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
Roleplay
HE
L3.1 RP Hero Dirty Harry 8B
Hero · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
Roleplay

Other memory sizes

Keep browsing