Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
MI
Ministral 3 8B Base 2512
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
IC
IceTea21EnergyDrinkRPV13 dpo240 i1
Icetea · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 Uncensored TextEncoders FLUX Klein Z Image Turbo
Qwen · wazimondo
Params4.02B
Context40K
Smallest4.3 GB
BU
Buddy E4B v2
Buddy · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
JO
JOSIExMistral 7B Instruct v0.2
Josiexmistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
MI
Mistral v0.2 Nexus Internal Knowledge Map 7B i1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llava v1.5 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
AU
Aura 4B
Aura · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
LL
Bio Medical Llama 3 8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
LL
Llama-3.1-Argunaut-1-8B-HIRPO
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
IN
Emollm InternLM2.5 7B chat GGUF fp16
InternLM · Slipstream-Max
Params7.74B
Context32K
Smallest15.5 GB
ST
L3.1 Storniitova 8B i1
Storniitova · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
LL
dolphin 2.9.4 llama3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3 VL 8B NSFW Caption V4.5
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
LO
LongWriter V 7B
Longwriter · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen2-7B-Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
MissLizzy_7b_HF
Misslizzy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 Alpha Ko 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
QW
I2S Qwen3 8B Confidence
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
LL
Llama-3-8B-Magpie-Pro-MT-UltraDPO2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CU
CursorCore QW2.5 7B i1
Cursorcore · mradermacher
Params7.61B
Context32K
Q4_K_M4.7 GB
SQ
OG SQL 7B
Sql · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
QW
Qwen3.5 9B Polaris HighIQ THINKING
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
SP
Spark-X2.6-4B
Spark · Mungert
Params4.11B
Context1024K
Q4_K_M2.6 GB
QW
Qwen3.5 9B heretic
Qwen · FadedRedStar
Params8.95B
Context256K
Q4_K_M5.6 GB
YI
Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
ME
Meliodas 7b dare
Meliodas · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
OM
OmniCoder Queen 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
HA
Hathor_Tahsin-L3-8B-v0.85
Hathor · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
FL
Flux3 PE GGUF Comfy
Flux · t8star
Params8.95B
Context256K
Q4_K_M5.6 GB