Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
BR
Breeze-7B-Instruct-v0_1
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
QW
Qwen3.8 9B
Qwen · Atomic-Germ
Params9.2B
Context256K
Q4_K_M5.8 GB
NO
argilla_notus-7b-v1
Notus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
Ornith 1.0 9B MTP
Ornith · protoLabsAI
Params9.2B
Context256K
Q4_K_M5.8 GB
MI
dolphin 2.2.1 mistral 7b i1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
Meltemi 7B v1.5
Meltemi · mradermacher
Params7.48B
Context32K
Q4_K_M4.5 GB
SA
Sailor2 8B Chat
Sailor · bartowski
Params8.55B
Context32K
Q4_K_M5.2 GB
KU
sanjiwatsuki_kunoichi-7b
Kunoichi · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
MA
magnum v3 9b chatml
Magnum · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
voxtral mini 4b realtime
Mistral · cstr
Params4.43B
Context—
Smallest2.5 GB
GE
Gemma 4 E2B It Ultra Uncensored Heretic
Gemma · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
RO
RoLlama3-8b-Instruct
Rollama · legraphista
Params8B
Context8K
Smallest2.0 GB
CO
ConfigurableHermes 7B i1
Configurablehermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
Ornith 1.5 9B uncensored
Ornith · ressl
Params9.2B
Context256K
Q4_K_M5.8 GB
MI
Mistral7B-PairRM-SPPO
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
FR
FrogNano 4B 2609 GGUF jang imatrix
Frognano · roman220220
Params4.33B
Context256K
Smallest0.6 GB
MI
norquinal_mistral-7b-claude-chat
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 4B Abliterated
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
QW
Qwen Image 2.1 Turbo
Qwen · molbal
Params7.12B
Context—
Q4_K_M4.5 GB
KN
KnowLine 4B Gen5.5
Knowline · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
AS
astrollama 2 7b chat aic i1
Astrollama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
OP
openchat_openchat_3.5
Openchat · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Llama 3.1 1 million cxt Dark Planet 8B
Llama · DavidAU
Params8B
Context1048K
Q4_K_M4.9 GB
MI
Mistral dolphin mix cine open Ne NSFW
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
SW
Swallow-7b-hf
Swallow · mradermacher
Params6.83B
Context4K
Q4_K_M4.1 GB
QW
Qwythos 9B v2
Qwythos · bartowski
Params9.2B
Context1024K
Q4_K_M6.0 GB
MI
Hermes 2 Pro Mistral 7B
Mistral · backyardai
Params7.24B
Context32K
Q4_K_M4.4 GB
LI
H2O Lightning 4b
Lightning · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CA
CAJAL 4B Merged 16bit
Cajal · Agnuxo
Params4.21B
Context256K
Q4_K_M2.7 GB
LL
llava 1.6
LLaVA · cmp-nct
Params7.24B
Context32K
Smallest4.4 GB