Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SP
Spark X2.6 4B Base
Spark · mradermacher
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
AT
Athena Codegemma 2 9b
Athena · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SE
SearchQwen2.5 7B
Searchqwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
PL
Pluto i1
Pluto · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
GI
Gille StrangeMerges 50 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
Meta Llma 3.2 3B Instruct Paul Graham GGUF
Meta · dhruvsangani
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
General Stories Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TR
Triumvirate Qwopus MiMo Ornith 9B Coder
Triumvirate · pragmaticcs
Params8.95B
Context256K
Smallest5.4 GB
CodingReasoningTool use
EG
Ege 8b 1.1
Ege · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
GR
GRM 3.2 Cliff
Grm · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
NA
Nanbeige4.1-3B
Nanbeige · Mungert
Params3.93B
Context256K
Q4_K_M2.6 GB
GGUF
KE
Keural Cortex 8B v1.0 1
Keural · mradermacher
Params8.19B
Context64K
Q4_K_M5.0 GB
Tool use
HA
Harmonic Hermes 9B
Harmonic · DJLougen
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
CodeQwen1.5 7B Chat
Qwen · Qwen
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
DE
deepseek coder 7b instruct v1.5
DeepSeek · LoneStriker
Params6.91B
Context4K
Q4_K_M4.2 GB
Coding
ED
EdgeRunner Command Nested
Edgerunner · mradermacher
Params7.6B
Context32K
Q4_K_M4.7 GB
GGUF
CL
clef flash
Clef · Livesport
Params8.95B
Context—
Smallest0.0 GB
GGUF
ZE
Zephyr 7b Beta Abliterated
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral NeuralDPO
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
RO
RolmOCR
Rolmocr · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Tool useVision
LL
alfredplpl Llama 3 8B Instruct Ja
Llama · mmnga
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
OL
Olmo 3 7B Think
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Reasoning
QW
MiMo V2.6 Distill Qwen 9B
Qwen · webmp3
Params8.95B
Context256K
Smallest4.1 GB
Vision
QW
Qwen3.5-9b-Sushi-Coder-RL
Qwen · bigatuna
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningVision
QW
Qwen3.5 9B Ultra Uncensored Heretic v2
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
GE
TheDrummer Gemmasutra Small 4B v1
Gemma · bartowski
Params3.86B
Context8K
Q4_K_M2.5 GB
GGUF
CO
Yi Coder 9B
Coder · mradermacher
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
NE
Orchestrator 8B
Nemotron · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
SP
Abliterated
Spark · alexwkleung
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
ZE
zephyr 7b alpha
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other sizes

Keep browsing