GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
MI
Mistral7B-PairRM-SPPO
Mistral · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
FR
FrogNano 4B 2609 GGUF jang imatrix
Frognano · roman220220
Params4.33B
Context256K
Smallest0.6 GB
QW
Qwen3.5 4B Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Best fitIQ4_XS 4.0 GB
KN
KnowLine 4B Gen5.5
Knowline · mradermacher
Params4.33B
Context256K
Best fitQ3_K_M 3.9 GB
MA
Mage VL
Mage · JohnTdi
Params21M
Context256K
Best fitQ8_0 2.2 GB
LL
Llama3.2 30B A3B II Dark Champion INSTRUCT Heretic Abliterated Uncensored
Llama · mradermacher
Params30B
Context128K
Q4_K_M18.4 GB
QW
Qwen2.5.1 Coder 1.5B Instruct
Qwen · bartowski
Params1.54B
Context32K
Best fitF16 3.8 GB
GL
GLM-4.7-Flash-Coder
GLM · ngquocvinh
Params30B
Context198K
Q4_K_M18.1 GB
GL
GLM 4.5V
GLM · mradermacher
Params107B
Context64K
Q4_K_M70.4 GB
QW
Qwen3.6 35B A3B DFlash
Qwen · magnitudedev
Params386M
Context40K
Best fitQ8_0 1.1 GB
MI
MiniCPM5 2B Claude Fable5 heretic
MiniCPM · saidutta69
Params2.52B
Context128K
Best fitQ8_0 3.5 GB
QW
Qwen3 TTS 12Hz 0.6B Base
Qwen · mradermacher
Params600M
Context32K
Best fitF16 2.7 GB
QW
Qwen3.8 2B
Qwen · piskle
Params1.94B
Context256K
Best fitQ8_0 3.0 GB
QW
Qwythos 9B v2
Qwythos · bartowski
Params9.2B
Context1024K
Q4_K_M6.0 GB
MI
Hermes 2 Pro Mistral 7B
Mistral · backyardai
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
LI
H2O Lightning 4b
Lightning · mradermacher
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
QW
Qwen3.5 0.8B Japanese SFT v2
Qwen · mradermacher
Params752M
Context256K
Best fitQ5_K_S 1.5 GB
QW
Qwen2.5 Coder 3B Uncensored
Qwen · bappi
Params3.1B
Context32K
Best fitQ4_K_M 2.7 GB
SM
SmolLM2 135M Instruct
SmolLM · lmstudio-community
Params135M
Context8K
Best fitQ3_K_L 0.8 GB
RY
RynnUnify Agentic Reasoner
Rynnunify · mradermacher
Params8.19B
Context256K
Best fitIQ1_M 4.0 GB
MI
chronos mistral 7b i1
Mistral · mradermacher
Params7.24B
Context8K
Best fitIQ2_S 3.9 GB
DE
DeepSeek R1 Distill Qwen 14B 3MPER0RR Abliterated
DeepSeek · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
QW
Qwen3 8B ERP v0.1
Qwen · Aratako
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
JA
AI21 Jamba Reasoning 3B
Jamba · bartowski
Params3.2B
Context256K
Best fitIQ2_M 4.0 GB
SA
Salience 27B R5
Salience · bartowski
Params27.3B
Context1024K
Q4_K_M17.8 GB
KA
KAI 7B Instruct v0.1
Kai · mradermacher
Params7.24B
Context32K
Best fitIQ2_S 3.9 GB
VE
Veritas 8B Fact Checker Non Thinking 1.0
Veritas · mradermacher
Params8.19B
Context40K
Best fitIQ1_M 4.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · CISCai
Params7.62B
Context32K
Best fitIQ2_M 3.8 GB
HU
HumanlikeRP
Humanlikerp · mradermacher
Params8.83B
Context12K
Best fitIQ2_XS 4.0 GB
BE
Behemoth X 123B v2.2
Behemoth · bartowski
Params123B
Context128K
Q4_K_M76.3 GB