Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
NE
NeoHorse 1 4B Max Heretic Uncensored TURBO
Neohorse · prithivMLmods
Params4.21B
Context256K
Q4_K_M2.7 GB
Tool use
MI
nondzu_mistral-7b-codealpaca-lora
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
LL
Llama 3.1 8B Instruct Jbliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Meta Llama 3.1 8B Instruct
Llama · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Huihui Qwen3.5 4B Abliterated
Qwen · interimlabs
Params4.21B
Context256K
Smallest2.7 GB
GGUF
WA
Wan2.2 TI2V 5B
Wan · squirrelae
Params5B
Context—
Q4_K_M3.4 GB
GGUF
GE
DiarizationLM Gemma 4 E4B v1
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
NO
NoisyGRPO_7B
Noisygrpo · mradermacher
Params7.6B
Context125K
Q4_K_M4.7 GB
GGUF
PI
ddh0_pirouette-7b
Pirouette · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
CodeLlama 7b Instruct Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3.1 Nemotron 8B UltraLong 1M Instruct absolute heresy i1
Llama · mradermacher
Params8B
Context1048K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 4B Thinking Heretic
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
ReasoningRoleplayTool use
GL
GLM 4 9B 0414
GLM · lmstudio-community
Params9.4B
Context32K
Q4_K_M6.2 GB
GGUF
YI
Yi 6B i1
Yi · mradermacher
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
LL
Llama 3.1 8B ArliAI RPMax v1.3
Llama · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
ZI
Zion_Alpha_Instruction_Tuned
Zion · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
Nemotron 3 Nano 4B
Nemotron · bartowski
Params3.97B
Context1024K
Q4_K_M3.0 GB
Tool use
QW
Qwen3.8 9B Distill
Qwen · superisaac1262
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
OU
Oumuamua-7b-instruct-v2
Oumuamua · mradermacher
Params7.33B
Context32K
Q4_K_M4.4 GB
GGUF
MO
models--automerger--Inex12M7-7B
MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FL
Flux
Flux · StableDiffusionVN
Params4.76B
Context512
Smallest2.7 GB
GGUF
GE
muraqib judge
Gemma · mohamedbalkhi169
Params7.46B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Alif 1.0 8B Instruct
Llama · large-traversaal
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Einstein-v7-Qwen2-7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
PH
Phi 3.5 Mini Instruct
Phi · duyntnet
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
YI
Yi 9B i1
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
FL
PornMaster flux2 klein 9b
Flux · cl0537furry
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
OS
Osmosis Mcp 4b
Osmosis · osmosis-ai
Params4.41B
Context40K
Q4_K_M2.7 GB
GGUF

Other sizes

Keep browsing