Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM2 8B A1B
Lfm · bartowski
Params8.34B
Context125K
Q4_K_M5.1 GB
GGUF
QW
Qwen3 4B Thinking 2507 Qwen3.8 Max Distillation Detrax
Qwen · detrax
Params4B
Context256K
Q4_K_M2.5 GB
CodingReasoning
RU
RusMind.Вспышка 3.2
Rusmind · DICPlay
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool use
AF
AFM 4.5B
Afm · bartowski
Params4.62B
Context64K
Q4_K_M2.9 GB
GGUF
GE
models--google--gemma-7b
Gemma · MaziyarPanahi
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
PE
Peach 2.0 9B 8k Roleplay Heretic
Peach · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
Roleplay
MA
MAI Gym 9B
Mai · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · failspy
Params8B
Context8K
Smallest4.0 GB
GGUF
MI
dolphin 2.8 mistral 7b v02
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 7B vietnamese 20k
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
DE
OREAL DeepSeek R1 Distill Qwen 7B
DeepSeek · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HA
Hathor_Respawn-L3-8B-v0.8
Hathor · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
OP
OpenChat 3.5 7B SOLAR v2.0 i1
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
NE
Nemotron Cascade 8B
Nemotron · bartowski
Params8.19B
Context32K
Q4_K_M5.0 GB
Reasoning
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
WE
WestLake 7B v2
Westlake · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
RI
RigoChat 7b v2
Rigochat · IIC
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
FL
Flux2 Klein 9B True V3
Flux · sanrenti
Params9.08B
Context—
Smallest5.7 GB
GGUF
PR
prodigy sm base v0.1
Prodigy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EU
EuroLLM 9B Instruct Heretic
Eurollm · mradermacher
Params9.15B
Context4K
Q4_K_M5.6 GB
Roleplay
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
Coding
QW
Huihui Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
NE
NeuralExperiment 7b MagicCoder v7
Neuralexperiment · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
OX
OxytocinErosEngineeringF1 7B slerp
Oxytocinerosengineeringf · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
AS
Astrollama 3 8b Chat_Summary
Astrollama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
DA
L3 Dark Planet Horror City 8B NEO Imatrix
Dark · DavidAU
Params8B
Context8K
Q4_K_M4.9 GB
Roleplay
TU
Turbo Ai 7b
Turbo · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
GE
Gemma 4 E2B It GMCE
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningTool use
NA
Nanbeige4.1 3B
Nanbeige · Edge-Quant
Params3.93B
Context192K
Q4_K_M2.4 GB
GGUF

Other sizes

Keep browsing