Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
LF
LFM2 8B A1B
Lfm · bartowski
Params8.34B
Context125K
Q4_K_M5.1 GB
QW
Qwen3 4B Thinking 2507 Qwen3.8 Max Distillation Detrax
Qwen · detrax
Params4B
Context256K
Q4_K_M2.5 GB
RU
RusMind.Вспышка 3.2
Rusmind · DICPlay
Params7.52B
Context128K
Q4_K_M5.3 GB
AF
AFM 4.5B
Afm · bartowski
Params4.62B
Context64K
Q4_K_M2.9 GB
GE
models--google--gemma-7b
Gemma · MaziyarPanahi
Params8.54B
Context8K
Q4_K_M5.3 GB
PE
Peach 2.0 9B 8k Roleplay Heretic
Peach · mradermacher
Params8.83B
Context16K
Q4_K_M5.3 GB
MA
MAI Gym 9B
Mai · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Meta-Llama-3-8B-Instruct-abliterated-v3
Llama · failspy
Params8B
Context8K
Smallest4.0 GB
MI
dolphin 2.8 mistral 7b v02
Mistral · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 7B vietnamese 20k
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
DE
OREAL DeepSeek R1 Distill Qwen 7B
DeepSeek · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
MI
flammenai flammen20 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HA
Hathor_Respawn-L3-8B-v0.8
Hathor · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
OP
OpenChat 3.5 7B SOLAR v2.0 i1
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
NE
Nemotron Cascade 8B
Nemotron · bartowski
Params8.19B
Context32K
Q4_K_M5.0 GB
SP
Spark X2.6 4B Uncensored
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
WE
WestLake 7B v2
Westlake · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
RI
RigoChat 7b v2
Rigochat · IIC
Params7.62B
Context32K
Q4_K_M4.7 GB
FL
Flux2 Klein 9B True V3
Flux · sanrenti
Params9.08B
Context—
Smallest5.7 GB
PR
prodigy sm base v0.1
Prodigy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
EU
EuroLLM 9B Instruct Heretic
Eurollm · mradermacher
Params9.15B
Context4K
Q4_K_M5.6 GB
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
QW
Huihui Qwen3 4B Instruct 2507 Abliterated
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
NE
NeuralExperiment 7b MagicCoder v7
Neuralexperiment · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
OX
OxytocinErosEngineeringF1 7B slerp
Oxytocinerosengineeringf · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
AS
Astrollama 3 8b Chat_Summary
Astrollama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
DA
L3 Dark Planet Horror City 8B NEO Imatrix
Dark · DavidAU
Params8B
Context8K
Q4_K_M4.9 GB
TU
Turbo Ai 7b
Turbo · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma 4 E2B It GMCE
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
NA
Nanbeige4.1 3B
Nanbeige · Edge-Quant
Params3.93B
Context192K
Q4_K_M2.4 GB