GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
NA
Nanbeige4.2 3B
Nanbeige · bartowski
Params4.17B
Context256K
Best fitQ4_1 4.0 GB
QW
Qwen3 ASR 0.6B
Qwen · ggml-org
Params752M
Context64K
Best fitBF16 2.9 GB
LF
LFM2.5 1.2B Instruct LocalAI
Lfm · weareapexcreators
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
QW
Atomic Chat / Qwen3.5 4B
Qwen · AtomicChat
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
QW
Qwen3 1.7B
Qwen · lmstudio-community
Params2.03B
Context32K
Best fitQ8_0 3.6 GB
GE
Gemma 3 1b It
Gemma · bartowski
Params1B
Context32K
Best fitBF16 2.7 GB
PI
PII Tracer
Pii · perplexity-ai
Params596M
Context32K
Best fitF16 2.6 GB
QW
Qwen3.8 3.6 27B blend
Qwen · JetBrains
Params27.3B
Context256K
Best fitQ4_K_M 1.8 GB
CH
Chandra Ocr 2
Chandra · prithivMLmods
Params4.84B
Context256K
Best fitQ3_K_S 3.9 GB
MI
MiniCPM OCR 1.0
MiniCPM · openbmb
Params752M
Context256K
Best fitF16 2.4 GB
QW
Qwen3 Zero Coder Reasoning V2 0.8B NEO EX
Qwen · DavidAU
Params816M
Context40K
Best fitF16 3.5 GB
GR
Granite Embedding 107m Multilingual
Granite · bartowski
Params107M
Context512
Best fitF16 0.8 GB
QW
Qwen 3.5 27B Derestricted
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
NasledieLab Qwen3.5 0.8B GPT 6 Astra Luna Fable 5.1 Opus 5.5 distill Karpy
Qwen · aemmeath
Params752M
Context256K
Best fitQ4_K_M 1.4 GB
GR
Granite-4.1-3B
Granite · weareapexcreators
Params3.4B
Context128K
Best fitQ5_K_M 3.6 GB
QW
Qwen3.5 27B WebNovel Writer Zh
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
MU
Muse-Glimmer-30B
Muse · AtomicChat
Params27.9B
Context128K
Q4_K_M16.9 GB
QW
Qwen3.5 27B Claude 4.6 Opus Reasoning Distilled Heretic
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
NO
nomic-embed-text-v1
Nomic · nomic-ai
Params137M
Context2K
Best fitF32 1.3 GB
LL
Llama-3.1-8B-Instruct
Llama · unsloth
Params8B
Context128K
Best fitIQ1_M 3.9 GB
LF
LFM2 2.6B
Lfm · LiquidAI
Params2.57B
Context125K
Best fitQ4_0 4.0 GB
QW
Qwen2.5 VL 7B Instruct
Qwen · bartowski
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
qwen2-1_5b-instruct
Qwen · Qwen
Params1.54B
Context32K
Q4_K_M1.0 GB
SM
SmolLM2 1.7B Instruct
SmolLM · bartowski
Params1.71B
Context8K
Best fitQ8_0 3.9 GB
QW
Qwen2.5 7b Ins v3
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
DE
deepseek-ai_deepseek-coder-1.3b-instruct
DeepSeek · TheBloke
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
MI
MinerU2.5 Pro 2605 1.2B
Mineru · jinzhenj
Params494M
Context8K
Best fitQ8_0 1.1 GB
QW
speakoflow mini
Qwen · SpeakoFlow
Params773M
Context256K
Best fitBF16 2.5 GB
QW
Qwen2.5-Omni-7B
Qwen · unsloth
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen2.5 3B Instruct Abliterated
Qwen · mradermacher
Params3.4B
Context32K
Best fitQ6_K 3.6 GB