Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM2.5 8B A1B Hermes Agentic Coder Abliterated v2
Lfm · mradermacher
Params8.47B
Context125K
Q4_K_M5.2 GB
CodingReasoningTool use
QW
Crow 9B Opus 4.6 Distill Heretic_Qwen3.5
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
ON
OneJev1.6 4B
Onejev · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningRoleplay
GE
Gemma-4-E2B-It
Gemma · mnsims1024
Params4.65B
Context128K
Q4_K_M3.1 GB
Vision
NE
Nemotron Terminal 8B
Nemotron · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
Coding
OP
openchat-3.5-0106
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Abliterated v2 MAX
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
IN
Index Homura 9B
Index · IndexTeam
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
Roleplay
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
Coding
QW
Qwen3 4b Deslop v4
Qwen · erikkannike
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
LE
LeeMerge 7B slerp
Leemerge · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
PH
calme 2.2 phi3 4b
Phi · MaziyarPanahi
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
GE
Forge 1 Gemma 4 E2B
Gemma · YoSaNoo
Params4.63B
Context128K
Q4_K_M3.4 GB
CodingReasoningTool use
PY
Pythia 6.9B Instruct v1 Merged
Pythia · bartowski
Params6.86B
Context2K
Q4_K_M4.7 GB
GGUF
GP
gpt oss nano
gpt-oss · squ11z1
Params9B
Context128K
Q4_K_M6.8 GB
Reasoning
KA
keynote-technology_kai-7b-beta
Kai · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
DE
deepseek coder 6.7b instruct
DeepSeek · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
Mistral NeMo Minitron 8B Instruct
Mistral · bartowski
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
CO
CorticalStack neurotic crown clown 7b ties
Corticalstack · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama Phi 3 DoRA i1
Llama · mradermacher
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
NE
NVIDIA Nemotron 3.5 Bolt E4B
Nemotron · prithivMLmods
Params7.52B
Context64K
Q4_K_M5.3 GB
ReasoningTool useVision
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningVision
MI
models--automerger--CalmexperimentT3q-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
sciphi_sciphi-self-rag-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Sakura Qwen Image 2.1 PE I2I Uncensored
Qwen · webmp3
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding

Other sizes

Keep browsing