GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
WhiteRabbitNeo V3 7B i1
Deephat · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Coding
NA
Nanbeige4.3 33B Base
Nanbeige · mradermacher
Params33.1B
Context128K
Q4_K_M20.1 GB
GGUF
LL
Turkish Llama 8b Instruct v0.1
Llama · ytu-ce-cosmos
Params8B
Context8K
Best fitIQ2_M 2.2 GB
GGUF
DA
danube2-1.8b-CodeFeedback
Danube · mradermacher
Params1.83B
Context8K
Best fitQ8_0 3.0 GB
Coding
QW
Git Commit Message Splitter Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
GGUF
QW
Qwen2.5 Coder 1.5B
Qwen · QuantFactory
Params1.54B
Context32K
Best fitQ8_0 2.4 GB
Coding
LL
llama nemotron embed 1b v2
Llama · sabafallah
Params1.24B
Context128K
Best fitQ8_0 2.1 GB
Embedding
QW
Qwen3 DND Jan v1 256k Ctx Brainstorm40x 8B
Qwen · mradermacher
Params7.66B
Context256K
Q4_K_M4.7 GB
CodingReasoning
IC
IceEspressoRPv1 7b i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Replete LLM V2.5 Qwen 7b
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
GE
Fallen Gemma3 4B v1
Gemma · bartowski
Params3.88B
Context128K
Best fitIQ4_NL 4.0 GB
GGUF
SO
SOCAM V1
Socam · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GGUF
AG
Agents A1 4B
Agents · bartowski
Params4.21B
Context256K
Best fitQ3_K_M 4.0 GB
Vision
LF
LFM2.5 2.6B Tema_Q Abliterated
Lfm · mradermacher
Params2.7B
Context128K
Best fitQ3_K_L 4.0 GB
GGUF
AA
aac pictogram retriever ptbr
Aac · Papuguinho
Params155M
Context2K
Best fitF16 1.0 GB
Embedding
VI
Video ORA 9B
Video · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
ReasoningVision
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
GGUF
LL
Llama 3.3 70B Instruct Heretic
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
GR
granite 3.1 2b instruct
Granite · QuantFactory
Params2.63B
Context128K
Best fitQ8_0 4.0 GB
GGUF
DE
deepseek-coder-1.3b-instruct
DeepSeek · duyntnet
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
Coding
TI
TinySolar-248m-4k-py
Tinysolar · RichardErkhov
Params248M
Context32K
Best fitQ8_0 0.9 GB
GGUF
YI
Yi-1.5-9B
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
GGUF
MA
Mamba2 130m Hf
Mamba · rpatel622
Params168M
Context1024K
Best fitQ8_0 0.7 GB
GGUF
QW
Qwen3.8 27B Uncensored FP8
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
QW
Qwen3.8 27B Continuum
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningRoleplay
QW
Qwen3.6 Fallen Fabulist 35B A3B Heretic
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
Roleplay
QW
Qwen2.5 Coder 7B Instruct
Qwen · QuantFactory
Params7.6B
Context32K
Best fitQ2_K 4.0 GB
Coding
QW
Qwen3 Esper3 Reasoning Instruct 6B Brainstorm20x Enhanced E32
Qwen · mradermacher
Params5.94B
Context40K
Best fitIQ1_M 3.9 GB
CodingReasoning
LL
Llama 3.1 Nemotron Nano 4B v1.1
Llama · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
GGUF
GE
Gemma 2 2b It
Gemma · BafS
Params2.61B
Context8K
Best fitQ4_K_M 3.1 GB
GGUF

Other memory sizes

Keep browsing