GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
NE
Next 8b
Next · mradermacher
Params8.19B
Context128K
Best fitIQ1_M 4.0 GB
Reasoning
RI
Rivermind 24B v1
Rivermind · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
ZE
Zentiq Trained God Core
Zentiq · Prince909097628
Params2.02B
Context128K
Smallest1.0 GB
Tool use
LL
Cogito v1 Preview Llama 3B
Llama · bartowski
Params3.61B
Context128K
Best fitQ4_K_L 4.0 GB
GGUF
GR
Grok Oss Thanatos
Grok · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
IN
Index Translate 35B A3B Preview
Index · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
IN
Index Echo S2ST 2B
Index · IndexTeam
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
Vision
QW
Qwen3.5 9B Claude 4.6 Opus Deckard V4.2 Uncensored Heretic Thinking
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplay
QW
Qwen3.8 4B Distill Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningTool useVision
QW
Qwen2.5 0.5B Instruct 3MPER0RR Abliterated
Qwen · mradermacher
Params494M
Context32K
Best fitQ6_K 1.1 GB
GGUF
QW
CoPaw Flash 9B
Qwen · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
QW
Qwen3.5 35B A3B Heretic Opus 4.6 Distilled
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool use
QW
Qwen3 42B A3B Spock Brutal Recall Instruct
Qwen · mradermacher
Params42.4B
Context256K
Q4_K_M25.7 GB
CodingReasoning
VI
Mythos Nano Final
Vibethinker · squ11z1
Params3.1B
Context128K
Best fitQ6_K 3.3 GB
CodingReasoning
QW
Qwen3 42B A3B 2507 Thinking Abliterated Uncensored TOTAL RECALL v2 Medium MASTER CODER
Qwen · mradermacher
Params42.4B
Context256K
Q4_K_M25.7 GB
CodingReasoning
YI
Yi-1.5-9B-32K
Yi · mradermacher
Params8.83B
Context32K
Best fitIQ2_XS 4.0 GB
GGUF
MI
Dolphin Mistral 24B Venice Edition Heretic
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
YU
YuLan Mini
Yulan · bartowski
Params2.42B
Context28K
Best fitQ6_K 3.8 GB
Coding
JI
jina embeddings v5 omni nano classification
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
EmbeddingVision
QW
Qwen_Qwen3 4B Thinking 2507
Qwen · Alcoft
Params4B
Context256K
Best fitQ4_K_S 4.0 GB
Reasoning
MI
Huihui MiniCPM V 4.6 Abliterated
MiniCPM · mradermacher
Params752M
Context256K
Best fitF16 2.4 GB
Vision
QW
Qwen2.5 7B Instruct 1M Heretic
Qwen · mradermacher
Params7.62B
Context986K
Best fitQ2_K 4.0 GB
Roleplay
QW
ari paraphraser 1.5b
Qwen · sriyuth
Params1.54B
Context32K
Smallest1.0 GB
GGUF
HO
K2-Horizon-0.9B
Horizon · bartowski
Params1.08B
Context128K
Best fitBF16 3.1 GB
Reasoning
QW
Qwen2.5 Coder 0.5B Instruct heretic
Qwen · saidutta69
Params494M
Context32K
Q4_K_M0.4 GB
CodingTool use
QW
Qwen3 Coder 42B A3B Instruct TOTAL RECALL MASTER CODER M 1million Ctx
Qwen · mradermacher
Params42.4B
Context1024K
Q4_K_M25.7 GB
CodingReasoning
CO
C4AI inverse sft model
Command R · mradermacher
Params135M
Context2K
Best fitF16 1.0 GB
GGUF
LL
Fireball Alpaca Llama3.1 8B Philos i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
TI
TigerLLM 12B It
Tigerllm · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
GGUF
NA
Nanonets OCR S
Nanonets · prithivMLmods
Params3.1B
Context125K
Best fitQ6_K 3.3 GB
Vision

Other memory sizes

Keep browsing