Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
LF
LFM2.5 8B A1B Hermes Agentic Coder Abliterated v2
Lfm · mradermacher
Params8.47B
Context125K
Q4_K_M5.2 GB
QW
Crow 9B Opus 4.6 Distill Heretic_Qwen3.5
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ON
OneJev1.6 4B
Onejev · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
GE
Gemma-4-E2B-It
Gemma · mnsims1024
Params4.65B
Context128K
Q4_K_M3.1 GB
NE
Nemotron Terminal 8B
Nemotron · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
OP
openchat-3.5-0106
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
QW
Qwen3.5 9B Abliterated v2 MAX
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
IN
Index Homura 9B
Index · IndexTeam
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Meta llama_Meta Llama 3.1 8B Instruct
Llama · ThomasBaruzier
Params8B
Context128K
Q4_K_M4.9 GB
GE
Gemma 4 E4B Luchador
Gemma · bartowski
Params7.5B
Context128K
Q4_K_M5.4 GB
SE
Seed Coder 8B Instruct
Seed · mradermacher
Params8.25B
Context32K
Q4_K_M5.1 GB
QW
Qwen3 4b Deslop v4
Qwen · erikkannike
Params4B
Context256K
Q4_K_M2.5 GB
LE
LeeMerge 7B slerp
Leemerge · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
YI
EVA Yi 1.5 9B 32K V1 i1
Yi · mradermacher
Params8.83B
Context32K
Q4_K_M5.3 GB
PH
calme 2.2 phi3 4b
Phi · MaziyarPanahi
Params3.82B
Context4K
Q4_K_M2.4 GB
GE
Forge 1 Gemma 4 E2B
Gemma · YoSaNoo
Params4.63B
Context128K
Q4_K_M3.4 GB
PY
Pythia 6.9B Instruct v1 Merged
Pythia · bartowski
Params6.86B
Context2K
Q4_K_M4.7 GB
GP
gpt oss nano
gpt-oss · squ11z1
Params9B
Context128K
Q4_K_M6.8 GB
KA
keynote-technology_kai-7b-beta
Kai · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
deepseek coder 6.7b instruct
DeepSeek · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
Mistral NeMo Minitron 8B Instruct
Mistral · bartowski
Params8.41B
Context8K
Q4_K_M5.1 GB
CO
CorticalStack neurotic crown clown 7b ties
Corticalstack · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama Phi 3 DoRA i1
Llama · mradermacher
Params3.82B
Context4K
Q4_K_M2.3 GB
NE
NVIDIA Nemotron 3.5 Bolt E4B
Nemotron · prithivMLmods
Params7.52B
Context64K
Q4_K_M5.3 GB
QW
Qwen3.5 4B MultiDistill 3xMerge UNCENSORED Heretic
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
MI
models--automerger--CalmexperimentT3q-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
sciphi_sciphi-self-rag-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Sakura Qwen Image 2.1 PE I2I Uncensored
Qwen · webmp3
Params8.95B
Context256K
Q4_K_M5.6 GB
OX
OxCoder 9B
Oxcoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB