GGUF models that fit in 4 GB
79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.
TE
Tess 4 9B
Tess · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
BU
BuddyGlassNeverSleeps methheadmethod v0.2 i1
Buddyglassneversleeps · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
LF
LFM3 1.7B Frontier SmartAgent
Lfm · mohommed
Params1.73B
Context512K
Best fitQ8_0 4.0 GB
AL
ALIA 40b Fc 2606
Alia · mradermacher
Params40.4B
Context160K
Q4_K_M24.6 GB
PY
Pythia 410m
Pythia · mradermacher
Params405M
Context2K
Best fitF16 2.1 GB
OP
open thoughts OpenThinker2 7B
Openthinker · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
JI
jina embeddings v2 base code Q8 0
Jina · ggml-org
Params160M
Context8K
Best fitQ8_0 1.0 GB
GR
GRM 3.2 Sky Abliterated
Grm · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
QW
Qwen3.8-27B
Qwen · Snowwhite00
Params27.3B
Context256K
Q4_K_M16.5 GB
QW
Mixture Summarizer Qwen3.5 2B
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
BO
Borealis 1b
Borealis · mradermacher
Params1B
Context32K
Best fitQ6_K 1.7 GB
GE
Gemma 3 270m
Gemma · gguf-org
Params268M
Context32K
Best fitF32 1.7 GB
NE
news title generator
News · mradermacher
Params248M
Context512
Best fitF16 1.3 GB
QW
ThinkLite Critic 7b
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
QW
Qwen3 1.7B heretic
Qwen · saidutta69
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
AU
Aurora Code 1
Aurora · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
SL
Slimaki 24B v1
Slimaki · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
SE
Seed OSS 36B Base
Seed · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
MI
Devstral Small 2 24B Instruct Abliterated
Mistral · mradermacher
Params23.6B
Context384K
Q4_K_M14.3 GB
LL
llama-3-tulu-2-dpo-8b
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
PO
Polar 32B
Polar · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
LL
liberalis-cogitator-llama-3.1-8b-dpo
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
KR
Kreatix K1
Kreatix · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
SM
SmolLM 135M GrammarSFT
SmolLM · mradermacher
Params135M
Context2K
Best fitF16 1.0 GB
RE
lb reranker 0.5B v1.0
Reranker · bartowski
Params494M
Context32K
Best fitF32 2.6 GB
QW
Qwen 3.5 0.8B Uncensored Conversational
Qwen · minas2025
Params752M
Context256K
Smallest0.8 GB
LL
Llama 3.1 8B Instruct Jbliterated v2
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
TE
Terminal RSI 27B
Terminal · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ON
OneDecision VisionGuard 4B SFT
Onedecision · prithivMLmods
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
GE
Gemma 3 4B VL It GLM 4.7 Flash Heretic Uncensored Thinking
Gemma · mradermacher
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB