Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
LL
Llama Guard 3 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
Qwen3 8B Q8 0 full v2
Qwen · RacerMiner
Params8.19B
Context40K
Smallest8.7 GB
GR
Ibm Granite_Granite 4.0 H Tiny
Granite · lmstudio-community
Params6.94B
Context1024K
Q4_K_M4.2 GB
PH
Athene Phi 3.5 mini instruct orpo i1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
NE
NeoHorse 1 9B Math SFT
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
codellama_codellama-7b-python-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
ZE
Zephyr 7B Beta Harmless
Zephyr · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
LogDoctor Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
QW
qwen3.5 9B super coder
Qwen · jica98
Params9.2B
Context256K
Smallest5.5 GB
DI
discoresearch_discolm_german_7b_v1
Discolm · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3Vl 4b Thinking
Qwen · Qwen
Params4B
Context256K
Q4_K_M2.5 GB
SA
SauerkrautLM 7b LaserChat
Sauerkrautlm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
CO
Cosmos-Reason1-7B
Cosmos · unsloth
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Qwen3 Vl 4b Heretic
Qwen · DreamFast
Params4B
Context256K
Q4_K_M2.5 GB
GE
Gemma 3 4b It Qat
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
DE
DeepSeek R1 Distill Qwen 7B Abliterated v2
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
QW
Qwen3.5 4B Heretic
Qwen · Biomanticus
Params4.21B
Context256K
Smallest2.7 GB
TU
Tulu 2 7B Helpful
Tulu · mradermacher
Params6.74B
Context8K
Q4_K_M4.1 GB
QW
Huihui ai.Qwen3 8B Abliterated
Qwen · DevQuasar
Params8.19B
Context40K
Q4_K_M5.0 GB
NO
normistral 7b warm instruct
Normistral · ltg
Params7.25B
Context2K
Q4_K_M4.4 GB
RE
Reggie v2 7b
Reggie · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
NE
NeoHorse 1 9B Uncensored GSQ RCO
Neohorse · abenzerps
Params8.95B
Context256K
Smallest3.1 GB
CO
Yi Coder 9B Chat
Coder · bartowski
Params8.83B
Context128K
Q4_K_M5.3 GB
HU
HuatuoGPT O1 7B
Huatuogpt · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
sciphi_sciphi-mistral-7b-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
ZD
TaichuAI/ZDTaichu5.0-9B
Zdtaichu · ngquocvinh
Params8.95B
Context256K
Q4_K_M5.6 GB
LO
LongCat Image Edit Turbo
Longcat · vantagewithai
Params6.29B
Context—
Q4_K_M3.9 GB
SE
sexeh time testing
Sexeh · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GE
Gemma 4 E4B xReadable zhTW OCR v1
Gemma · xCloudinfo
Params7.46B
Context128K
Q4_K_M5.3 GB
WI
ehartford_wizard-vicuna-7b-uncensored
Wizard · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB