Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Ministral 3 8B Base 2512
Mistral · mradermacher
Params8.49B
Context256K
Q4_K_M5.2 GB
GGUF
IC
IceTea21EnergyDrinkRPV13 dpo240 i1
Icetea · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Uncensored TextEncoders FLUX Klein Z Image Turbo
Qwen · wazimondo
Params4.02B
Context40K
Smallest4.3 GB
Coding
BU
Buddy E4B v2
Buddy · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
JO
JOSIExMistral 7B Instruct v0.2
Josiexmistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
MI
Mistral v0.2 Nexus Internal Knowledge Map 7B i1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llava v1.5 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
AU
Aura 4B
Aura · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
RoleplayTool use
LL
Bio Medical Llama 3 8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama-3.1-Argunaut-1-8B-HIRPO
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
IN
Emollm InternLM2.5 7B chat GGUF fp16
InternLM · Slipstream-Max
Params7.74B
Context32K
Smallest15.5 GB
GGUF
ST
L3.1 Storniitova 8B i1
Storniitova · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
dolphin 2.9.4 llama3.1 8b i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Coding
QW
Qwen3 VL 8B NSFW Caption V4.5
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
LO
LongWriter V 7B
Longwriter · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
GGUF
QW
Qwen2-7B-Instruct
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
MissLizzy_7b_HF
Misslizzy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Alpha Ko 8B Instruct
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
I2S Qwen3 8B Confidence
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
LL
Llama-3-8B-Magpie-Pro-MT-UltraDPO2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
CU
CursorCore QW2.5 7B i1
Cursorcore · mradermacher
Params7.61B
Context32K
Q4_K_M4.7 GB
Coding
SQ
OG SQL 7B
Sql · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.5 9B Polaris HighIQ THINKING
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplay
SP
Spark-X2.6-4B
Spark · Mungert
Params4.11B
Context1024K
Q4_K_M2.6 GB
ReasoningTool use
QW
Qwen3.5 9B heretic
Qwen · FadedRedStar
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingVision
YI
Yi-1.5-9B-Chat
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
ME
Meliodas 7b dare
Meliodas · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
OM
OmniCoder Queen 9B
Omnicoder · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingRoleplayVision
HA
Hathor_Tahsin-L3-8B-v0.85
Hathor · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
FL
Flux3 PE GGUF Comfy
Flux · t8star
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision

Other sizes

Keep browsing