Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
CO
Yi Coder 9B
Coder · RichardErkhov
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
QW
Huihui Qwen3.5 9B Claude 4.6 Opus Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
DE
DeepHat V1 7B
Deephat · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
RO
RoMistral-7b-Instruct
Romistral · legraphista
Params7.24B
Context32K
Smallest1.6 GB
GGUF
YI
Yi 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5 9B The Defiant Fable DARK ROAST Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
CodingReasoningRoleplay
QW
shisa v2.1 qwen3 8b
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
YU
quantized-yue2-3b
Yue · ngquocvinh
Params3.63B
Context—
Q4_K_M2.9 GB
GGUF
JA
Jan-Nano-128K
Jan · unsloth
Params4B
Context128K
Q4_K_M2.5 GB
GGUF
PH
Phi 3.5 Mini instruct_Uncensored
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
ZE
huggingfaceh4_zephyr-7b-beta
Zephyr · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 4B Instruct 2507
Qwen · ggml-org
Params4B
Context256K
Smallest4.3 GB
GGUF
LL
Meta-Llama-3-8B-Instruct-imatrix
Llama · lmstudio-community
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
AB
ABD3ID CodeMentor
Abd · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Coding
DE
deepseek llm 7B chat
DeepSeek · TheBloke
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · modularai
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MY
Mythos 9b Unhinged
Mythos · King3Djbl
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningRoleplayTool use
DE
deepseek-ai_deepseek-coder-5.7bmqa-base
DeepSeek · TheBloke
Params5.7B
Context16K
Q4_K_M3.4 GB
Coding
QW
Qwen3 VL Embedding 8B
Qwen · VesNFF
Params7.6B
Context256K
Smallest6.2 GB
EmbeddingVision
QW
Qwen3.5 9B Gemini 3.1 Pro Reasoning Distill
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.7 GB
ReasoningVision
LL
Llama-3-ELYZA-JP-8B
Llama · mmnga
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 Swallow 8B RL v0.2
Qwen · mmnga-o
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
QW
sakura-8b-qwen3-v1.1
Qwen · SakuraLLM
Params8.2B
Context32K
Smallest4.6 GB
GGUF
GL
glm-4-9b-chat
GLM · legraphista
Params9.4B
Context128K
Smallest4.0 GB
GGUF
PH
Phi 3.1 mini 128k instruct
Phi · bartowski
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
GE
Gemma-2-9B-It-SPPO-Iter3
Gemma · legraphista
Params9.24B
Context8K
Smallest2.4 GB
GGUF
QW
Qwen3.5 4B GGUF shards
Qwen · Arki05
Params4B
Context—
Smallest0.0 GB
GGUF
LL
Llama 3.1 Swallow 8B Instruct v0.5
Llama · mmnga
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Vl 7b Cam Motion
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
DE
DeepSeek R1 Distill Llama 8B Heretic
DeepSeek · saidutta69
Params8.03B
Context128K
Q4_K_M4.9 GB
ReasoningTool use

Other sizes

Keep browsing