GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen1.5-1.8B-Chat-AWQ-fp16
Qwen · Qwen
Params1.84B
Context32K
Best fitQ6_K 3.7 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · QuantFactory
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
GGUF
GE
Gemma 4 26B A4B QAT Voice GGUFs
Gemma · alexisStacksCode
Params25.2B
Context128K
Best fitQ4_0 1.8 GB
ReasoningTool useVision
MI
Mistral Military 24B
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
QW
Qwen3.5 9B Artemis
Qwen · turintech
Params8.95B
Context256K
Q4_K_M5.7 GB
ReasoningTool useVision
JA
Javi0.3 4B
Javi · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
ReasoningTool use
QW
Qwen3 Psychological Reasoning 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
Reasoning
QW
BiggerCoQ Qwen3 10b
Qwen · mradermacher
Params10.9B
Context40K
Q4_K_M6.6 GB
GGUF
QW
Qwen3.5-0.8B
Qwen · NobodyWho
Params752M
Context256K
Best fitBF16 2.4 GB
Tool useVision
GE
gemma 4 26B A4B it MTP
Gemma · ironbcc
Params420M
Context256K
Best fitF16 2.4 GB
GGUF
NE
neutts air
Neutts · neuphonic
Params750M
Context—
Best fitBF16 2.2 GB
GGUF
QW
Qwen3.5-35B-A3B
Qwen · Mungert
Params34.7B
Context256K
Q4_K_M21.6 GB
Vision
GR
Parable Granite 4.1 3B Claude Fable 5 heretic
Granite · saidutta69
Params3.4B
Context128K
Best fitQ6_K 4.0 GB
ReasoningTool use
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
Tool use
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
GGUF
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
ST
StartLux Decision v1 2B Q8 0
Startlux · startlux-models
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
GGUF
MI
MiniAI Quata2 4b
Miniai · mradermacher
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Reasoning
ST
starcoder2 7b
StarCoder · QuantFactory
Params7.2B
Context16K
Best fitQ2_K 3.9 GB
Coding
TE
Teutonic I Reliquary RL
Teutonic · bartowski
Params8.95B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
KI
moonshotai Kimi K2 Thinking
Kimi · bartowski
Params1030B
Context256K
Q4_K_M624.0 GB
Reasoning
GL
Q3.6 27B GLM 5.1 DA
GLM · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Reasoning
QW
Meissa Qwen2.5 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
Roleplay
QW
JoyFox Qwen3.6 35B A3B RP Aggressive
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
Roleplay
AR
ArmenianGPT 1.0 3B
Armeniangpt · mradermacher
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
Reasoning
LF
LFM2.5-24B-A2B-DSpark
Lfm · LiquidAI
Params328M
Context128K
Best fitF16 1.2 GB
ReasoningTool use
GR
Granite 8b Code Instruct 4k
Granite · mradermacher
Params8.05B
Context4K
Best fitIQ2_XXS 3.9 GB
CodingTool use
QW
Qwen2.5 Wolverine CODER 11B V2
Qwen · mradermacher
Params10.9B
Context32K
Best fitIQ1_M 4.0 GB
Coding
CO
C4AI orpo model
Command R · mradermacher
Params135M
Context2K
Best fitF16 1.0 GB
GGUF

Other memory sizes

Keep browsing