GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Best fitIQ4_XS 4.0 GB
ED
EdgeRock 4B
Edgerock · hkust-aaron
Params4.58B
Context256K
Best fitQ3_K_M 4.0 GB
MI
MinerU2.5 Pro 2605 1.2B
Mineru · mradermacher
Params494M
Context8K
Best fitF16 1.6 GB
QW
Qwen2-7B-Instruct
Qwen · legraphista
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
LF
LFM2.5 1.2B Instruct DSpark Draft v1
Lfm · LiquidAI
Params296M
Context125K
Best fitF16 1.2 GB
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Best fitQ8_0 3.5 GB
DE
DeepSeek R1 Distill Qwen 8B Abliterated
DeepSeek · mradermacher
Params7.62B
Context128K
Best fitQ2_K 4.0 GB
OR
Ornith 1.5 397B
Ornith · bartowski
Params397B
Context256K
Q4_K_M241.8 GB
TR
Trinity Nano Preview
Trinity · arcee-ai
Params6.12B
Context128K
Best fitQ3_K_L 4.0 GB
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Best fitQ4_0 4.0 GB
MI
MiMo Ornith 9B AGSI
Mimo · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
NE
NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash
Nemotron · ggml-org
Params7.52B
Context128K
Smallest0.6 GB
LL
Llama Song Stream 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Best fitQ5_K_L 3.9 GB
DE
Janus Pro 1B
DeepSeek · mradermacher
Params1.65B
Context16K
Best fitQ8_0 3.9 GB
TR
TranslatePsy AfriSLM 2B
Translatepsy · mradermacher
Params2.39B
Context256K
Best fitQ6_K 2.9 GB
OL
AMD OLMo 1B SFT DPO
OLMo · lmstudio-community
Params1.18B
Context2K
Best fitQ8_0 2.8 GB
QW
Qwen3 Jan v1 256k Ctx 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context256K
Best fitIQ1_M 3.9 GB
MI
Mistral 7B v0.1
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
JI
JiuZhou
Jiuzhou · mradermacher
Params7.5B
Context32K
Best fitIQ2_S 4.0 GB
AL
AlphaMonarch 7B i1
Alphamonarch · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
GP
gpt2
Gpt · mradermacher
Params163M
Context1K
Best fitF16 1.1 GB
LL
Reasoning Llama 1b v0.1
Llama · bartowski
Params1.24B
Context128K
Best fitF16 3.2 GB
CO
CodeQwen1.5-7B-Chat
Codeqwen · bartowski
Params7.25B
Context64K
Best fitIQ2_M 4.0 GB
QW
Qwen3 1.7B Coder Distilled SFT
Qwen · reaperdoesntknow
Params2.03B
Context40K
Best fitQ8_0 3.6 GB
SM
Biggie SmoLlm 0.15B Base
SmolLM · mradermacher
Params181M
Context2K
Best fitF16 1.1 GB
MA
MN 12B Mag Mell R1 Uncensored Scale1.2 i1
Mag · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
LF
LFM2.5 1.2B Nova Function Calling
Lfm · NovachronoAI
Params1.17B
Context125K
Best fitQ8_0 2.8 GB
DA
Darwin 36B Opus ABLITERATED HERETIC
Darwin · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB