GGUF models that fit in 4 GB

79024 GGUF language models have at least one quant whose estimated memory use is 4 GB or less at 8K context. Each card shows the largest quant that fits. Sizes run from 1M to 20260731B parameters. The longest trained context is 1280000K tokens.

Most downloadedNewestLargestLongest contextSmallest
TE
Tess 4 9B
Tess · bartowski
Params9.2B
Context256K
Q4_K_M6.0 GB
ReasoningTool useVision
BU
BuddyGlassNeverSleeps methheadmethod v0.2 i1
Buddyglassneversleeps · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
GGUF
LF
LFM3 1.7B Frontier SmartAgent
Lfm · mohommed
Params1.73B
Context512K
Best fitQ8_0 4.0 GB
CodingReasoningTool use
AL
ALIA 40b Fc 2606
Alia · mradermacher
Params40.4B
Context160K
Q4_K_M24.6 GB
ReasoningTool use
PY
Pythia 410m
Pythia · mradermacher
Params405M
Context2K
Best fitF16 2.1 GB
GGUF
OP
open thoughts OpenThinker2 7B
Openthinker · bartowski
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
GGUF
JI
jina embeddings v2 base code Q8 0
Jina · ggml-org
Params160M
Context8K
Best fitQ8_0 1.0 GB
CodingEmbedding
GR
GRM 3.2 Sky Abliterated
Grm · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool use
QW
Qwen3.8-27B
Qwen · Snowwhite00
Params27.3B
Context256K
Q4_K_M16.5 GB
ReasoningTool useVision
QW
Mixture Summarizer Qwen3.5 2B
Qwen · mradermacher
Params1.88B
Context256K
Best fitQ8_0 2.9 GB
ReasoningTool use
BO
Borealis 1b
Borealis · mradermacher
Params1B
Context32K
Best fitQ6_K 1.7 GB
GGUF
GE
Gemma 3 270m
Gemma · gguf-org
Params268M
Context32K
Best fitF32 1.7 GB
GGUF
NE
news title generator
News · mradermacher
Params248M
Context512
Best fitF16 1.3 GB
GGUF
QW
ThinkLite Critic 7b
Qwen · mradermacher
Params7.62B
Context125K
Best fitQ2_K 4.0 GB
ReasoningVision
QW
Qwen3 1.7B heretic
Qwen · saidutta69
Params1.72B
Context40K
Best fitQ8_0 3.3 GB
ReasoningTool use
AU
Aurora Code 1
Aurora · bartowski
Params34.7B
Context256K
Q4_K_M21.4 GB
Coding
SL
Slimaki 24B v1
Slimaki · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
SE
Seed OSS 36B Base
Seed · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
GGUF
MI
Devstral Small 2 24B Instruct Abliterated
Mistral · mradermacher
Params23.6B
Context384K
Q4_K_M14.3 GB
Coding
LL
llama-3-tulu-2-dpo-8b
Llama · mradermacher
Params8B
Context8K
Best fitIQ2_XXS 4.0 GB
GGUF
PO
Polar 32B
Polar · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
LL
liberalis-cogitator-llama-3.1-8b-dpo
Llama · mradermacher
Params8B
Context128K
Best fitIQ2_XXS 4.0 GB
Roleplay
KR
Kreatix K1
Kreatix · mradermacher
Params7.62B
Context32K
Best fitQ2_K 4.0 GB
Tool use
SM
SmolLM 135M GrammarSFT
SmolLM · mradermacher
Params135M
Context2K
Best fitF16 1.0 GB
GGUF
RE
lb reranker 0.5B v1.0
Reranker · bartowski
Params494M
Context32K
Best fitF32 2.6 GB
Embedding
QW
Qwen 3.5 0.8B Uncensored Conversational
Qwen · minas2025
Params752M
Context256K
Smallest0.8 GB
Vision
LL
Llama 3.1 8B Instruct Jbliterated v2
Llama · mradermacher
Params8.03B
Context128K
Best fitIQ2_XXS 4.0 GB
Tool use
TE
Terminal RSI 27B
Terminal · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
CodingReasoningTool use
ON
OneDecision VisionGuard 4B SFT
Onedecision · prithivMLmods
Params4.21B
Context256K
Best fitQ3_K_L 4.0 GB
Vision
GE
Gemma 3 4B VL It GLM 4.7 Flash Heretic Uncensored Thinking
Gemma · mradermacher
Params3.88B
Context128K
Best fitQ4_K_S 4.0 GB
ReasoningVision

Other memory sizes

Keep browsing