GGUF models that fit in 6 GB
123636 GGUF language models have at least one quant whose estimated memory use is 6 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 4194304K tokens.
OP
openchat_openchat-3.5-0106
Openchat · TheBloke
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
MI
chronos mistral 7b i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitQ4_K_M 5.9 GB
DE
DeepSeek R1 Distill Qwen 14B 3MPER0RR Abliterated
DeepSeek · mradermacher
Params14.8B
Context128K
Best fitIQ1_M 6.0 GB
QW
Qwen3 8B ERP v0.1
Qwen · Aratako
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
JA
AI21 Jamba Reasoning 3B
Jamba · bartowski
Params3.2B
Context256K
Best fitQ6_K_L 5.6 GB
SA
Salience 27B R5
Salience · bartowski
Params27.3B
Context1024K
Q4_K_M17.8 GB
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
VE
Veritas 8B Fact Checker Non Thinking 1.0
Veritas · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · CISCai
Params7.62B
Context32K
Best fitIQ4_XS 5.2 GB
HU
HumanlikeRP
Humanlikerp · mradermacher
Params8.83B
Context12K
Best fitQ3_K_L 6.0 GB
BE
Behemoth X 123B v2.2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M76.3 GB
QW
Qwen3.5 4B EmperoAI Qwen3.8 Distill Heretic Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ6_K 5.0 GB
MI
paixai_astrid-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Best fitQ4_K_M 5.9 GB
LL
LlamaScientific 8B
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ4_XS 6.0 GB
RO
rocket 3B
Rocket · TheBloke
Params2.8B
Context4K
Best fitQ6_K 5.5 GB
JI
Jina Reranker v3.5
Jina · jinaai
Params596M
Context128K
Best fitBF16 2.6 GB
QW
Qwen3.5 4B DFlash
Qwen · magnitudedev
Params634M
Context40K
Best fitQ8_0 1.4 GB
MI
Ministral-3-3B-Reasoning-2512
Mistral · Mungert
Params3.43B
Context256K
Best fitQ8_0 5.0 GB
QW
Agrillm Qwen3 30B A3B
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ4_K_M 5.7 GB
SM
KrynexAI 1.4
SmolLM · KrynexLabs
Params135M
Context8K
Best fitF16 1.0 GB
QW
Co Rewarding I Qwen3 8B Base MATH
Qwen · mradermacher
Params8.19B
Context32K
Best fitQ3_K_M 5.8 GB
SO
Socio Foundation 8B
Socio · mradermacher
Params8.19B
Context40K
Best fitQ3_K_M 5.8 GB
RO
L3.2 Rogue Creative Instruct Uncensored 7B
Rogue · DavidAU
Params7.53B
Context128K
Best fitQ2_K 5.7 GB
DA
L3.1 Dark Planet SpinFire Uncensored 8B
Dark · DavidAU
Params8B
Context128K
Best fitQ3_K_L 5.9 GB
ME
Hy Memory 30B A3B
Memory · vinnipukh
Params30B
Context256K
Best fitQ1_0 5.8 GB
LF
LFM2 VL 450M
Lfm · LiquidAI
Params354M
Context125K
Best fitF16 1.7 GB
VA
Valkyrie 49B v2.1
Valkyrie · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
GP
Gpt Oss 120b
gpt-oss · bartowski
Params117B
Context128K
Q4_K_M62.8 GB
QW
Qwen3 Coder 30B A3B Instruct Westernized
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB