GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
GGUF
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Best fitQ3_K_M 4.0 GB
Coding
MI
MinerU2.5 Pro 2605 1.2B
Mineru · mradermacher
Params494M
Context8K
Best fitF16 1.6 GB
Vision
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
LF
LFM2.5 1.2B Instruct DSpark Draft v1
Lfm · LiquidAI
Params296M
Context125K
Best fitF16 1.2 GB
GGUF
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
GGUF
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
Reasoning
OR
Ornith 1.5 397B
Ornith · bartowski
Params397B
Context256K
Q4_K_M241.8 GB
Vision
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Best fitQ3_K_L 4.0 GB
GGUF
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Best fitQ4_0 4.0 GB
Reasoning
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
ReasoningTool use
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
ReasoningTool use
LL
Llama Song Stream 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitQ5_K_L 3.9 GB
GGUF
DE
Janus Pro 1B
DeepSeek · mradermacher
Params1.65B
Context16K
Best fitQ8_0 3.9 GB
Vision
TR
TranslatePsy AfriSLM 2B
Translatepsy · mradermacher
Params2.39B
Context256K
Best fitQ6_K 2.9 GB
ReasoningTool use
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Best fitQ8_0 2.8 GB
GGUF
QW
Qwen3 Jan v1 256k Ctx 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context256K
Best fitIQ1_M 3.9 GB
Coding
MI
Mistral 7B v0.1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
JI
JiuZhou
Jiuzhou · mradermacher
Params7.5B
Context32K
Best fitIQ2_S 4.0 GB
GGUF
AL
AlphaMonarch 7B i1
Alphamonarch · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GGUF
GP
gpt2
Gpt · mradermacher
Params163M
Context1K
Best fitF16 1.1 GB
GGUF
LL
Reasoning Llama 1b v0.1
Llama · bartowski
Params1.24B
Context128K
Best fitF16 3.2 GB
Reasoning
CO
CodeQwen1.5-7B-Chat
Codeqwen · bartowski
Params7.25B
Context64K
Best fitIQ2_M 4.0 GB
GGUF
QW
Qwen3 1.7B Coder Distilled SFT
Qwen · reaperdoesntknow
Params2.03B
Context40K
Best fitQ8_0 3.6 GB
CodingReasoningTool use
SM
Biggie SmoLlm 0.15B Base
SmolLM · mradermacher
Params181M
Context2K
Best fitF16 1.1 GB
GGUF
MA
MN 12B Mag Mell R1 Uncensored Scale1.2 i1
Mag · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
Reasoning
LF
LFM2.5 1.2B Nova Function Calling
Lfm · NovachronoAI
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
Tool use
DA
Darwin 36B Opus ABLITERATED HERETIC
Darwin · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF

Other memory sizes

Keep browsing