GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
KU
Kunoichi DPO v2 7B
Kunoichi · brittlewis12
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
GGUF
DE
Josiefied DeepSeek R1 0528 Qwen3 8B Abliterated v1
DeepSeek · mradermacher
Params8.19B
Context128K
Best fitIQ1_M 4.0 GB
Reasoning
QW
Qwen3 Reranker 4B
Qwen · mradermacher
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
Embedding
QW
Models Qwen Qwen2.5 3B Instruct
Qwen · MaziyarPanahi
Params3.09B
Context32K
Best fitQ6_K 3.3 GB
GGUF
RU
MN 12B Runeweaver RP RU
Runeweaver · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
RoleplayTool use
TR
Trinity Mini
Trinity · bartowski
Params26B
Context128K
Q4_K_M15.9 GB
GGUF
MI
Mistralai Shieldstral 1.0 3B
Mistral · Abiray
Params3.43B
Context256K
Best fitQ5_K_S 3.8 GB
Vision
QW
Qwen3.8 27B MindMeld OneJev1.6
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
PU
L3.1 Purosani i1
Purosani · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LL
ToolACE 2.5 Llama 3.1 8B Heretic
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
GE
Gemma 2 2b
Gemma · legraphista
Params2.61B
Context8K
Best fitQ6_K 3.5 GB
GGUF
PI
Piaget 8B
Piaget · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
Reasoning
QW
GrayLine Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
Reasoning
QW
Qwen3.5 9B Kimi K3 Distilled
Qwen · somenetworktrouble
Params9.2B
Context256K
Best fitQ3_K_S 1.6 GB
ReasoningTool useVision
QW
Qwen3.5 4b Agentic Coder v4
Qwen · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
CodingVision
QW
Qwen3 Reranker 4B
Qwen · giladgd
Params4B
Context40K
Best fitQ3_K_L 3.9 GB
Embedding
DR
DrakIdol Roleplayer 1.0
Drakidol · mradermacher
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
Roleplay
GE
gemma 3 4b instruct
Gemma · lm-kit
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB
GGUF
LL
Llama Calix CLM 1.0
Llama · calix-ai-labs
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
MI
Magistral Small 2507
Mistral · bartowski
Params23.6B
Context40K
Q4_K_M14.3 GB
Reasoning
LL
Llama 3.2 3B Agent007 Coder
Llama · QuantFactory
Params3.21B
Context128K
Best fitQ5_1 3.9 GB
Coding
QW
jinaai/test-qwen25-coder-jina-code-embeddings-1.5b
Qwen · jinaai
Params1.54B
Context32K
Best fitBF16 3.8 GB
CodingEmbedding
PO
Polaris VGA 27B Post1.0e
Polaris · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
LL
Kiel 2.1 Sol
Llama · kiel2
Params3.2B
Context128K
Best fitQ4_K_M 3.5 GB
GGUF
QW
Dolphin3.0 Qwen2.5 3B
Qwen · QuantFactory
Params3.4B
Context32K
Best fitQ6_K 3.6 GB
CodingTool use
QW
Qwen3-Vl-4B-Thinking-1M
Qwen · unsloth
Params4B
Context977K
Best fitIQ4_XS 4.0 GB
ReasoningVision
YI
Yi 1.5 6B Chat
Yi · QuantFactory
Params6.06B
Context4K
Best fitQ3_K_M 4.0 GB
GGUF
LL
Hermes 2 Theta Llama 3 70B
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
BG
bge-small-en-v1.5
BGE · ChristianAzinn
Params33M
Context512
Q4_K_M0.0 GB
Embedding
QW
Qwen3.5 40B RoughHouse Claude 4.6 Opus Polar Deckard Uncensored Heretic Thinking
Qwen · mradermacher
Params39.1B
Context256K
Q4_K_M23.9 GB
CodingReasoningRoleplay

Other memory sizes

Keep browsing