GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
MiniCPM V 4.7 35B A3B
MiniCPM · bartowski
Params34.7B
Context256K
Q4_K_M22.3 GB
TR
Trida2.0 4B
Trida · mradermacher
Params4.84B
Context256K
Best fitIQ3_S 4.0 GB
NE
NeoHorse 1 4B
Neohorse · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
PI
piano-medley-7b
Piano · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
FR
FrogNano 4B 2609
Frognano · NANI-Nithin
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
JA
Jais-3-MoVA-36B-A4B
Jais · mradermacher
Params37.4B
Context512K
Q4_K_M22.4 GB
AN
Anubis Mini 8B v1
Anubis · bartowski
Params8B
Context128K
Q4_K_M5.1 GB
NO
nomic-embed-text-v1.5
Nomic · second-state
Params137M
Context2K
Best fitF16 1.1 GB
GE
Gemma-3-270M-It-Qat
Gemma · unsloth
Params268M
Context32K
Best fitF16 1.2 GB
JI
Jina Embeddings v5 Text Nano Classification
Jina · jinaai
Params212M
Context8K
Best fitF16 1.2 GB
NO
Novel Humanize 4b
Novel · bartowski
Params4.02B
Context40K
Best fitIQ4_XS 4.0 GB
NI
Ninja-v1-NSFW-RP
Ninja · mradermacher
Params7.24B
Context4K
Best fitIQ2_S 3.9 GB
LL
dolphin-2.9.1-llama-3-8b
Llama · legraphista
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
HU
HuatuoGPT O1 7B
Huatuogpt · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
HU
Huihui LFM2.5 8B A1B Abliterated
Huihui · mradermacher
Params8.47B
Context125K
Best fitIQ1_S 3.9 GB
LF
LFM2.5-24B-A2B
Lfm · ngquocvinh
Params24B
Context128K
Q4_K_M14.5 GB
TO
tolkio models
Tolkio · datacrunch
Params1.94B
Context256K
Smallest0.6 GB
NO
Nora0.1 1.7B
Nora · mradermacher
Params1.72B
Context40K
Best fitQ6_K 2.9 GB
JI
Jina Embeddings v5 Text Small Clustering
Jina · jinaai
Params596M
Context32K
Best fitF16 2.6 GB
LF
LFM3 1.7B Heretic Abliterated
Lfm · Vortecks
Params1.73B
Context256K
Best fitQ8_0 4.0 GB
WH
whisper large v3 turbo
Whisper · oxide-lab
Params809M
Context—
Best fitQ8_0 1.5 GB
GL
Glm-4.1V-9B-Thinking
GLM · unsloth
Params9.4B
Context64K
Best fitIQ1_S 4.0 GB
DE
DeepSeek R1 Distill Llama 8B Abliterated
DeepSeek · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3 Omni 30B A3B Captioner
Qwen · mradermacher
Params30.5B
Context64K
Q4_K_M18.6 GB
GU
GUI Owl 1.5 2B Instruct
Gui · mradermacher
Params1.72B
Context32K
Best fitQ8_0 3.3 GB
CO
codegeex4-all-9b
Codegeex · legraphista
Params9.4B
Context128K
Best fitIQ1_S 3.9 GB
QW
Qwen3 1.7B Abliterated
Qwen · bartowski
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
PA
PaddleOCR-VL-1.6
Paddleocr · Mungert
Params467M
Context128K
Best fitBF16 1.6 GB
NE
neutts air q4
Neutts · neuphonic
Params748M
Context—
Best fitQ4_0 1.1 GB
PA
Pantheon Reasoning 27B
Pantheon · bartowski
Params27.3B
Context256K
Q4_K_M18.0 GB