GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
llama-3-sqlcoder-8b
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
Coding
QW
Qwen2 VL 2B Instruct
Qwen · lmstudio-community
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
Vision
LL
Llama 3.2 3B Instruct
Llama · QuantFactory
Params3.2B
Context128K
Best fitQ5_1 3.9 GB
GGUF
ET
Eternal.Bondange 3.2 1B
Eternal · mradermacher
Params1.5B
Context128K
Best fitQ6_K 2.0 GB
Tool use
HU
Huihui HY MT1.5 1.8B Abliterated
Huihui · mradermacher
Params1.79B
Context256K
Best fitQ8_0 2.9 GB
GGUF
LL
Llama 3.1 8B xLAM Agent
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
QW
Qwen3 1.7B Distilled 30B A3B SFT
Qwen · reaperdoesntknow
Params2.03B
Context40K
Best fitQ8_0 3.6 GB
ReasoningTool use
YI
Yi 34B 200K DARE Merge v7
Yi · RichardErkhov
Params34.4B
Context195K
Best fitIQ3_XS 3.2 GB
GGUF
TH
TheSpice-7b-FT-v0.3.1
Thespice · duyntnet
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 72B Instruct Malaysian
Qwen · mradermacher
Params72.7B
Context32K
Q4_K_M47.4 GB
Tool use
ST
MN Starlight Sylph 12B
Starlight · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
Roleplay
TA
UI TARS 2B SFT
Tars · mradermacher
Params1.78B
Context32K
Best fitQ8_0 2.6 GB
Vision
GE
Gemma4 E4B Mahou Nsfw
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Roleplay
PA
Palette RP 9B 2609 v0.05
Palette · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
Roleplay
QW
Qwen3.5 9B GLM5.1 Distill v1
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
SC
ScienceRSI 8b
Sciencersi · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
ReasoningTool use
JU
juanako 7b UNA i1
Juanako · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
GR
Grug 35b v2
Grug · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool use
FL
Q3.6 27B DS v4 Flash DA
Flash · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Reasoning
NE
NeoHorse 1 4B Max Heretic Uncensored TURBO
Neohorse · prithivMLmods
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
Tool use
OL
OLMo 2 0425 1B Instruct
OLMo · allenai
Params1.48B
Context4K
Best fitQ8_0 3.2 GB
GGUF
TR
Transliteration 2.0 0.8B Safetensors
Transliteration · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
ReasoningTool use
CA
MN CaptainErisNebula Chimera v1.1 THINKING ClaudeOpus4.5 12B Heretic Uncensored
Captainerisnebula · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
ReasoningRoleplay
JI
jina embeddings v4 text code
Jina · jinaai
Params3.1B
Context125K
Best fitQ6_K 3.3 GB
CodingEmbedding
LL
Llama 3.1 8B Instruct Jbliterated
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
MI
MiniCPM V 4.6 Thinking Heretic MTP
MiniCPM · MuXodious
Params773M
Context256K
Best fitBF16 2.5 GB
ReasoningVision
NO
NoisyGRPO_7B
Noisygrpo · mradermacher
Params7.6B
Context125K
Best fitQ2_K 4.0 GB
GGUF
QW
qwen3 tts 0 6b base mix precision
Qwen · Arm
Params83M
Context64K
Best fitQ8_0 1.2 GB
GGUF
LL
Llama 3.1 Nemotron 8B UltraLong 1M Instruct absolute heresy i1
Llama · mradermacher
Params8B
Context1048K
Best fitIQ2_XXS 4.0 GB
GGUF
QW
ExpertLens Qwen3 VL 30B A3B
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
Vision

Other memory sizes

Keep browsing