GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
ST
Step 3.5 Flash REAP 149B A11B i1
Step · mradermacher
Params149B
Context256K
Q4_K_M89.9 GB
AU
Aura 4B
Aura · bartowski
Params4.51B
Context128K
Best fitQ3_K_L 4.0 GB
QW
dgpl linux assistant 0.8b
Qwen · dgpl
Params752M
Context256K
Best fitF16 2.4 GB
ER
Erk 14B
Erk · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
LL
Llama-3.1-Argunaut-1-8B-HIRPO
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
ST
L3.1 Storniitova 8B i1
Storniitova · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
LL
dolphin 2.9.4 llama3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
QW
Qwen3 VL 8B NSFW Caption V4.5
Qwen · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
GP
Gpt Oss 20b RichardErkhov Heresy
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
PY
Pythia 160m
Pythia · mradermacher
Params162M
Context2K
Best fitF16 1.1 GB
LO
LongWriter V 7B
Longwriter · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
Qwen2-7B-Instruct
Qwen · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
PE
Pelican1.0 VL 235B A22B FC
Pelican · mradermacher
Params235B
Context256K
Q4_K_M142.2 GB
MI
MissLizzy_7b_HF
Misslizzy · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
QW
Qwen3 1.7B Jailbreak Reasoning
Qwen · mradermacher
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
QW
I2S Qwen3 8B Confidence
Qwen · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen3.5 9B heretic imatrix
Qwen · FadedRedStar
Params456M
Context256K
Best fitQ8_0 3.7 GB
CU
CursorCore QW2.5 7B i1
Cursorcore · mradermacher
Params7.61B
Context32K
Best fitQ2_K 4.0 GB
QW
Qwen3.5 9B Polaris HighIQ THINKING
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen2.5 1.5B Instruct Abliterated
Qwen · mradermacher
Params1.54B
Context32K
Best fitQ6_K 2.0 GB
DA
Darwin 35B A3B Opus
Darwin · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
QW
Bharat Tiny LLM v3
Qwen · eulogik
Params1.72B
Context32K
Best fitQ4_K_M 2.5 GB
SP
Spark-X2.6-4B
Spark · Mungert
Params4.11B
Context1024K
Best fitQ3_K_M 4.0 GB
QW
Qwen3.5 9B heretic
Qwen · FadedRedStar
Params8.95B
Context256K
Best fitQ8_0 3.7 GB
QW
Qwen3 32B LoRA ECHO KK GRPO
Qwen · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
YI
Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Best fitIQ2_XS 4.0 GB
TI
Tinyllama 15M
Tinyllama · QuantFactory
Params15M
Context256
Best fitQ2_K 0.6 GB
OM
OmniCoder Queen 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
UwU Qwen3.5 27B v0.1
Qwen · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
ZE
Zen4 Coder
Zen · mradermacher
Params79.7B
Context256K
Q4_K_M48.5 GB