GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen3.5 4B EmperoAI Qwen3.8 Distill Heretic Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
LL
LlamaScientific 8B
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
JI
Jina Reranker v3.5
Jina · jinaai
Params596M
Context128K
Best fitBF16 2.6 GB
QW
Qwen3.5 4B DFlash
Qwen · magnitudedev
Params634M
Context40K
Best fitQ8_0 1.4 GB
MI
Ministral-3-3B-Reasoning-2512
Mistral · Mungert
Params3.43B
Context256K
Best fitQ5_K_M 4.0 GB
QW
Agrillm Qwen3 30B A3B
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
QW
Qwen2.5 VL 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
SM
KrynexAI 1.4
SmolLM · KrynexLabs
Params135M
Context8K
Best fitF16 1.0 GB
QW
Co Rewarding I Qwen3 8B Base MATH
Qwen · mradermacher
Params8.19B
Context32K
Best fitIQ1_M 4.0 GB
SO
Socio Foundation 8B
Socio · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
ME
Hy Memory 30B A3B
Memory · vinnipukh
Params30B
Context256K
Q4_K_M18.2 GB
LF
LFM2 VL 450M
Lfm · LiquidAI
Params354M
Context125K
Best fitF16 1.7 GB
VA
Valkyrie 49B v2.1
Valkyrie · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
GP
Gpt Oss 120b
gpt-oss · bartowski
Params117B
Context128K
Q4_K_M62.8 GB
QW
Qwen3 Coder 30B A3B Instruct Westernized
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
CL
CLM v0.1 8B
Clm · czl
Params8.19B
Context40K
Q4_K_M5.0 GB
SH
Shieldgemma 2b
Shieldgemma · legraphista
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
QW
Qwen3.6 27B ABLITERATED UNCENSORED PHILADELPHIA CLASS
Qwen · mradermacher
Params26.9B
Context256K
Q4_K_M16.5 GB
QW
dolphin-2.9.3-qwen2-1.5b
Qwen · crusoeai
Params1.54B
Context128K
Q4_K_M1.0 GB
AM
Amaretto 8B
Amaretto · mradermacher
Params8.49B
Context256K
Best fitIQ1_M 3.9 GB
HA
Haru Kanana 8b
Haru · mradermacher
Params8B
Context32K
Best fitIQ2_XXS 4.0 GB
YI
Yi-1.5-9B-Chat-16K
Yi · mradermacher
Params8.83B
Context16K
Best fitIQ2_XS 4.0 GB
SM
nisten/Biggie-SmoLlm-0.15B-Base
SmolLM · nisten
Params152M
Context2K
Best fitBF16 1.1 GB
QW
QiMing Holos Plus Qwen3 14B
Qwen · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ST
L3-8B-Stheno-v3.2
Stheno · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
QW
Jan-v2-VL-high
Qwen · Mungert
Params8.2B
Context256K
Q4_K_M5.4 GB
QW
Qwen3.6 35B A3B Uncensored Aggressive
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3 VL 32B Instruct Ultra Uncensored Heretic
Qwen · mradermacher
Params32.8B
Context256K
Q4_K_M19.8 GB
HO
Holo4 27B Abliterix
Holo · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Qwen2.5 Coder 0.5B Instruct Abliterated
Qwen · bartowski
Params494M
Context32K
Best fitF16 1.6 GB