Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama Guard 3 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 8B Q8 0 full v2
Qwen · RacerMiner
Params8.19B
Context40K
Smallest8.7 GB
GGUF
GR
Ibm Granite_Granite 4.0 H Tiny
Granite · lmstudio-community
Params6.94B
Context1024K
Q4_K_M4.2 GB
GGUF
PH
Athene Phi 3.5 mini instruct orpo i1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
NE
NeoHorse 1 9B Math SFT
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
LL
codellama_codellama-7b-python-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
ZE
Zephyr 7B Beta Harmless
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
LogDoctor Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
QW
qwen3.5 9B super coder
Qwen · jica98
Params9.2B
Context256K
Smallest5.5 GB
CodingVision
DI
discoresearch_discolm_german_7b_v1
Discolm · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3Vl 4b Thinking
Qwen · Qwen
Params4B
Context256K
Q4_K_M2.5 GB
ReasoningVision
SA
SauerkrautLM 7b LaserChat
Sauerkrautlm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
CO
Cosmos-Reason1-7B
Cosmos · unsloth
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3 Vl 4b Heretic
Qwen · DreamFast
Params4B
Context256K
Q4_K_M2.5 GB
Vision
GE
Gemma 3 4b It Qat
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
QW
Qwen3.5 4B Heretic
Qwen · Biomanticus
Params4.21B
Context256K
Smallest2.7 GB
GGUF
TU
Tulu 2 7B Helpful
Tulu · mradermacher
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
QW
Huihui ai.Qwen3 8B Abliterated
Qwen · DevQuasar
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
NO
normistral 7b warm instruct
Normistral · ltg
Params7.25B
Context2K
Q4_K_M4.4 GB
Coding
RE
Reggie v2 7b
Reggie · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
ReasoningVision
NE
NeoHorse 1 9B Uncensored GSQ RCO
Neohorse · abenzerps
Params8.95B
Context256K
Smallest3.1 GB
ReasoningTool use
CO
Yi Coder 9B Chat
Coder · bartowski
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
HU
HuatuoGPT O1 7B
Huatuogpt · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Reasoning
MI
sciphi_sciphi-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZD
TaichuAI/ZDTaichu5.0-9B
Zdtaichu · ngquocvinh
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
LO
LongCat Image Edit Turbo
Longcat · vantagewithai
Params6.29B
Context—
Q4_K_M3.9 GB
GGUF
SE
sexeh time testing
Sexeh · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 E4B xReadable zhTW OCR v1
Gemma · xCloudinfo
Params7.46B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
WI
ehartford_wizard-vicuna-7b-uncensored
Wizard · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF

Other sizes

Keep browsing