GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
ON
OneJev 0.8B
Onejev · mradermacher
Params1.01B
Context256K
Best fitF16 2.9 GB
Vision
MI
OpenChat 3.5 7B Mixtral v2.0 i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
GGUF
MI
MiniCPM5-2B
MiniCPM · Mungert
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
CodingTool use
QW
Qwen3.5-2B
Qwen · trubisky
Params1.88B
Context256K
Best fitIQ3_M 1.9 GB
GGUF
AR
arco reasoner i1
Arco · mradermacher
Params514M
Context8K
Best fitQ5_K_S 1.3 GB
GGUF
DE
Decka 4B
Decka · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
ReasoningTool use
QW
Qwen3 14B RefusalDirection ThinkingAware
Qwen · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
Reasoning
GE
Gemma 3 270m It
Gemma · gguf-org
Params268M
Context32K
Best fitF32 1.7 GB
GGUF
QW
GPT 5 Distill Qwen3 4B Instruct
Qwen · Jackrong
Params4B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
LL
Impish_LLAMA_3B
Llama · l3utterfly
Params3.2B
Context128K
Best fitQ4_K_M 3.5 GB
GGUF
QW
Qwen3.5 4B Heretic
Qwen · mradermacher
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
ReasoningTool useVision
NA
NaNovel 9B
Nanovel · mradermacher
Params8.95B
Context256K
Best fitIQ1_S 4.0 GB
GGUF
US
UserRepGen 4B
Userrepgen · bartowski
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
GGUF
OM
OmniCoder 9B
Omnicoder · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Coding
HU
HuatuoGPT 3 9B
Huatuogpt · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
QW
Qwopus3 5 9B Coder Flash
Qwopus · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
QW
Qwen3 VL 8B Thinking
Qwen · bartowski
Params8.19B
Context256K
Q4_K_M5.0 GB
ReasoningVision
QW
Qwen2.5 0.5B Instruct
Qwen · mradermacher
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
RU
RusMind.Моментум 3.5.1
Rusmind · DICPlay
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningTool use
LR
OrionLLM LRM 3.2
Lrm · bartowski
Params27B
Context256K
Q4_K_M17.5 GB
ReasoningVision
LL
Herplete LLM Llama 3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
SM
SmolLlamix 8x101M take2
Smolllamix · mradermacher
Params399M
Context32K
Best fitQ8_0 1.0 GB
GGUF
GI
GigaChat3 10B A1.8B
Gigachat · bartowski
Params10.7B
Context256K
Q4_K_M6.7 GB
GGUF
QW
Qwen2.5 VL 3B uncensored bughunter v9
Qwen · mmp2055
Params3.1B
Context125K
Best fitQ4_K_M 2.7 GB
Tool useVision
AI
Aitana 7B S Instruct
Aitana · mradermacher
Params7.77B
Context8K
Best fitIQ1_M 4.0 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B Ablit
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool use
QW
open text detector qwen36 35b a3b oct10
Qwen · woog
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
GP
Gpt2 Large
Gpt · mradermacher
Params838M
Context1K
Best fitF16 3.7 GB
GGUF
FA
Fara 7B
Fara · gguf-org
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
Vision
AP
Apocrypha-7b
Apocrypha · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF

Other memory sizes

Keep browsing