GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QW
Qwen1.5-1.8B-Chat-AWQ-fp16
Qwen · Qwen
Params1.84B
Context32K
Best fitQ6_K 3.7 GB
LL
Llama 3.2 1B Instruct
Llama · QuantFactory
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
GE
Gemma 4 26B A4B QAT Voice GGUFs
Gemma · alexisStacksCode
Params25.2B
Context128K
Best fitQ4_0 1.8 GB
MI
Mistral Military 24B
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
QW
Qwen3.5 9B Artemis
Qwen · turintech
Params8.95B
Context256K
Q4_K_M5.7 GB
JA
Javi0.3 4B
Javi · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
QW
Qwen3 Psychological Reasoning 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
BiggerCoQ Qwen3 10b
Qwen · mradermacher
Params10.9B
Context40K
Q4_K_M6.6 GB
QW
Qwen3.5-0.8B
Qwen · NobodyWho
Params752M
Context256K
Best fitBF16 2.4 GB
GE
gemma 4 26B A4B it MTP
Gemma · ironbcc
Params420M
Context256K
Best fitF16 2.4 GB
NE
neutts air
Neutts · neuphonic
Params750M
Context—
Best fitBF16 2.2 GB
QW
Qwen3.5-35B-A3B
Qwen · Mungert
Params34.7B
Context256K
Q4_K_M21.6 GB
GR
Parable Granite 4.1 3B Claude Fable 5 heretic
Granite · saidutta69
Params3.4B
Context128K
Best fitQ6_K 4.0 GB
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
ST
StartLux Decision v1 2B Q8 0
Startlux · startlux-models
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
MI
MiniAI Quata2 4b
Miniai · mradermacher
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
ST
starcoder2 7b
StarCoder · QuantFactory
Params7.2B
Context16K
Best fitQ2_K 3.9 GB
TE
Teutonic I Reliquary RL
Teutonic · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
KI
moonshotai Kimi K2 Thinking
Kimi · bartowski
Params1030B
Context256K
Q4_K_M624.0 GB
GL
Q3.6 27B GLM 5.1 DA
GLM · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Meissa Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
QW
JoyFox Qwen3.6 35B A3B RP Aggressive
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
AR
ArmenianGPT 1.0 3B
Armeniangpt · mradermacher
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
LF
LFM2.5-24B-A2B-DSpark
Lfm · LiquidAI
Params328M
Context128K
Best fitF16 1.2 GB
GR
Granite 8b Code Instruct 4k
Granite · mradermacher
Params8.05B
Context4K
Best fitIQ2_XXS 3.9 GB
QW
Qwen2.5 Wolverine CODER 11B V2
Qwen · mradermacher
Params10.9B
Context32K
Best fitIQ1_M 4.0 GB
CO
C4AI orpo model
Command R · mradermacher
Params135M
Context2K
Best fitF16 1.0 GB