Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HE
HelpingAI-9B
Helpingai · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
NS
NSK 128k 7B slerp
Nsk · mradermacher
Params7.24B
Context128K
Q4_K_M4.4 GB
GGUF
QW
qwen3 4b Z Image Engineer
Qwen · BennyDaBall
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
QW
Qwen3.8 Flash Next MTP
Qwen · dzannotti
Params3.88B
Context256K
Q4_K_M2.6 GB
GGUF
JA
japanese-novel-gpt-j-6b
Japanese · mmnga
Params6.05B
Context2K
Q4_K_M3.7 GB
GGUF
MA
magnum v2 4b
Magnum · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
MI
dolphin-2.9.3-mistral-7B-32k
Mistral · mradermacher
Params7.25B
Context32K
Q4_K_M4.4 GB
CodingTool use
LL
CodeLlama 7b CypherGen
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.2 GB
Coding
GE
rijgersberg_geitje-7b-chat
Geitje · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
StableBeluga 7B
Stablebeluga · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OP
OpenChat 3.5 7B Starling v2.0
Openchat · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
EU
Eubiota Planner 8b
Eubiota · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
HY
Hypnos I1 8B
Hypnos · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
QW
Qwopus3.5 4B Coder
Qwen · Jackrong
Params4.33B
Context256K
Q4_K_M2.8 GB
CodingReasoningTool use
QW
MiMo-V2.6-Distill-Qwen-9B
Qwen · ngquocvinh
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingTool useVision
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
GR
Granite 4.1 8b Abliterated
Granite · mradermacher
Params8.38B
Context128K
Q4_K_M5.1 GB
GGUF
DE
Decision 4b Hf
Decision · evalengine
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
CO
codegemma-7b-it
Codegemma · bartowski
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
MI
MiMo VL 7B RL 2508
Mimo · bartowski
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
ST
Stheno Hercules 3.1 8B
Stheno · bartowski
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite 3.3 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
ME
adaptllm_medicine-llm
Medicine · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
NE
Nemotron Mini 4B Instruct
Nemotron · bartowski
Params4.19B
Context4K
Q4_K_M2.7 GB
GGUF
QW
Qwen3.5 9B Unredacted MAX
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen2 VL 7B Instruct Abliterated
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
QW
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16
Qwen · nuofang
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
LI
LittleInstructionMaker 4B v0.1
Littleinstructionmaker · mradermacher
Params3.96B
Context8K
Q4_K_M2.4 GB
Tool use
PO
Polaris VGA 9B Post1.0e
Polaris · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF

Other sizes

Keep browsing