Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SK
Skywork OR1 Math 7B
Skywork · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
IN
Internlm2 Math Base 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
PR
PreciseCoder 4B
Precisecoder · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Coding
FL
PornMaster flux2 klein 9b
Flux · VenerableRolliPolli
Params8.19B
Context256K
Q4_K_M5.0 GB
GGUF
OL
Olmo-3-1025-7B
OLMo · Mungert
Params7.3B
Context64K
Q4_K_M4.7 GB
GGUF
QW
Qwen3 Esper3 Reasoning CODER Instruct 6B Brainstorm20x
Qwen · mradermacher
Params5.94B
Context40K
Q4_K_M3.6 GB
CodingReasoning
QW
Huihui Qwen3.5 9B Abliterated
Qwen · interimlabs
Params8.95B
Context256K
Smallest5.6 GB
GGUF
MO
moshika candle q8
Moshika · kyutai
Params7.69B
Context—
Smallest8.2 GB
GGUF
NE
Nemotron-3.5-Bolt-E4B-Heretic
Nemotron · grumd0
Params7.46B
Context64K
Smallest3.7 GB
ReasoningTool useVision
MI
Mistral 7B Cymraeg Welsh v2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
HO
K2 Horizon 3.7B Claude Opus Reasoning Distilled
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
ReasoningTool use
DE
Deepseek Coder 6.7b Instruct
DeepSeek · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
FL
Flux2 Klein 9b Uncensored Text Encoder
Flux · wy8edhuh
Params8.19B
Context40K
Smallest6.7 GB
Coding
QW
qwen3 4b heretic zimage
Qwen · Lockout
Params4B
Context40K
Smallest4.3 GB
GGUF
QW
ColdBrew Qwen3.5 RP_Base CharThink
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningRoleplayTool use
MI
YuisekinAIEvol Mistral 7B ja math v0.1.1
Mistral · mmnga
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternVL3_5 4B
Internvl · bartowski
Params4.41B
Context40K
Q4_K_M2.7 GB
Vision
DE
Deepseek R1 Distill NSFW RPv1
DeepSeek · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Reasoning
EX
EXAONE 3.5 7.8B Instruct
Exaone · ThomasBaruzier
Params7.8B
Context32K
Q4_K_M4.8 GB
GGUF
QW
ThinCtx Qwen3.5 4B
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
OP
weyaxi_openchat-3.5-1210-seraph-slerp
Openchat · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Meerkat 8b v1.0
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LI
Lily-Cybersecurity-7B-v0.2
Lily · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AM
llm360_amberchat
Amberchat · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Roblox Coder v2
Qwen · preferedev
Params4.33B
Context256K
Q4_K_M2.8 GB
CodingVision
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
CodingTool use
GE
gemma 4 E4B it The DECKARD Expresso Universe HERETIC UNCENSORED Thinking
Gemma · DavidAU
Params7.5B
Context128K
Smallest11.3 GB
CodingReasoningRoleplay
FR
FrogNano 4B 2609 Heretic
Frognano · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
EX
Experiment26 7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NO
Noema 1.5 4B Deep
Noema · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoning

Other sizes

Keep browsing