Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LO
Louis GRC 7B slerp M2
Louis · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
MI
MiMo Ornith 9B AGSI Abliterated HQ
Mimo · OliviaRossi
Params8.95B
Context256K
Smallest6.5 GB
Reasoning
AN
Anima 2.9B
Anima · vanes430
Params5.84B
Context—
Smallest3.3 GB
GGUF
QW
gohumanize open humanizer
Qwen · gohumanize
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · itlwas
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
BI
Bielik Minitron 7B v3.0 Instruct
Bielik · speakleash
Params7.48B
Context32K
Q4_K_M4.5 GB
GGUF
TU
allenai_tulu-2-dpo-7b
Tulu · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
FA
Falcon H1 7B Instruct
Falcon · mradermacher
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
QW
Qwen3.5 9B Coder
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningTool use
QW
MathSmith HC Problem Synthesizer Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter0
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NeoHorse 1 9B Heretic
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
WA
WazoCore 8B v6 Long
Wazocore · mradermacher
Params8.15B
Context128K
Q4_K_M5.0 GB
Tool use
MI
openpipe_mistral-ft-optimized-1227
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MO
Mochi 4B
Mochi · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
QW
PAW 27B X3
Qwen · lackonendes
Params4.85B
Context256K
Smallest0.7 GB
GGUF
IN
models--automerger--Inex16Calme-7B
Inex · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AC
AceGPT v1.5 7B
Acegpt · mradermacher
Params6.84B
Context4K
Q4_K_M4.2 GB
GGUF
LL
Mahou-1.3b-llama3-8B
Llama · mradermacher
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
Indian Legal Assitant i1
Indian · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
qwen3 4b pokerbench grpo
Qwen · YiPz
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
QW
MathSmith HC Qwen3 8B Qwen2_5 32B ShortCoT
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen Image 2.1 Uncensored
Qwen · rayss868123
Params7.12B
Context—
Q4_K_M4.6 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningRoleplay
DR
Dream v0 Instruct 7B
Dream · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
MI
mrm8488 mistral 7b ft AgentInstruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NU
Nutopia 7B
Nutopia · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
MI
uukuguy_speechless-mistral-moloras-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SP
uukuguy_speechless-tora-code-7b-v1.0
Speechless · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
PH
Phi 3 mini 4k instruct
Phi · QuantFactory
Params3.82B
Context4K
Q4_K_M2.3 GB
Coding

Other sizes

Keep browsing