GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral7B-PairRM-SPPO
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
FR
FrogNano 4B 2609 GGUF jang imatrix
Frognano · roman220220
Params4.33B
Context256K
Smallest0.6 GB
Vision
QW
Qwen3.5 4B Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
KN
KnowLine 4B Gen5.5
Knowline · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
ReasoningTool useVision
MA
Mage VL
Mage · JohnTdi
Params21M
Context256K
Best fitQ8_0 2.2 GB
Vision
LL
Llama3.2 30B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params30B
Context128K
Q4_K_M18.4 GB
Roleplay
QW
Qwen2.5.1 Coder 1.5B Instruct
Qwen · bartowski
Params1.54B
Context32K
Best fitF16 3.8 GB
Coding
GL
GLM-4.7-Flash-Coder
GLM · ngquocvinh
Params30B
Context198K
Q4_K_M18.1 GB
Coding
GL
GLM 4.5V
GLM · mradermacher
Params107B
Context64K
Q4_K_M70.4 GB
GGUF
QW
Qwen3.6 35B A3B DFlash
Qwen · magnitudedev
Params386M
Context40K
Best fitQ8_0 1.1 GB
GGUF
MI
MiniCPM5 2B Claude Fable5 heretic
MiniCPM · saidutta69
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
ReasoningTool use
QW
Qwen3 TTS 12Hz 0.6B Base
Qwen · mradermacher
Params600M
Context32K
Best fitF16 2.7 GB
Vision
QW
Qwen3.8 2B
Qwen · piskle
Params1.94B
Context256K
Best fitQ8_0 3.0 GB
GGUF
QW
Qwythos 9B v2
Qwythos · bartowski
Params9.2B
Context1024K
Q4_K_M6.0 GB
Reasoning
MI
Hermes 2 Pro Mistral 7B
Mistral · backyardai
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
Tool use
LI
H2O Lightning 4b
Lightning · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
GGUF
QW
Qwen3.5 0.8B Japanese SFT v2
Qwen · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
GGUF
QW
Qwen2.5 Coder 3B Uncensored
Qwen · bappi
Params3.1B
Context32K
Best fitQ4_K_M 2.7 GB
Coding
SM
SmolLM2 135M Instruct
SmolLM · lmstudio-community
Params135M
Context8K
Best fitQ3_K_L 0.8 GB
GGUF
RY
RynnUnify Agentic Reasoner
Rynnunify · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
Tool use
MI
chronos mistral 7b i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
GGUF
DE
DeepSeek R1 Distill Qwen 14B 3MPER0RR Abliterated
DeepSeek · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
Reasoning
QW
Qwen3 8B ERP v0.1
Qwen · Aratako
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
JA
AI21 Jamba Reasoning 3B
Jamba · bartowski
Params3.2B
Context256K
Best fitIQ2_M 4.0 GB
Reasoning
SA
Salience 27B R5
Salience · bartowski
Params27.3B
Context1024K
Q4_K_M17.8 GB
CodingReasoningTool use
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
Coding
VE
Veritas 8B Fact Checker Non Thinking 1.0
Veritas · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
Reasoning
QW
Qwen2.5 Coder 7B Instruct
Qwen · CISCai
Params7.62B
Context32K
Best fitIQ2_M 3.8 GB
Coding
HU
HumanlikeRP
Humanlikerp · mradermacher
Params8.83B
Context12K
Best fitIQ2_XS 4.0 GB
GGUF
BE
Behemoth X 123B v2.2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M76.3 GB
GGUF

Other memory sizes

Keep browsing