Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3 8B
Qwen · Aldaris
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
LA
latxa-7b-v1.2
Latxa · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GE
gemma 4 E4B it heretic
Gemma · Abiray
Params7.5B
Context128K
Q4_K_M5.3 GB
CodingVision
AP
APUS Reflex 9B
Apus · apus-ailab
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
mistralai_mistral-7b-v0.1
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
TwIL LM3 Pro
Granite · webAI-Official
Params3.66B
Context128K
Q4_K_M2.2 GB
ReasoningTool use
QW
Qwen3 4B
Qwen · lmstudio-community
Params4B
Context32K
Q4_K_M2.5 GB
GGUF
JE
Jebadiah 9b v2
Jebadiah · frontier-infra
Params9.2B
Context256K
Q4_K_M5.8 GB
GGUF
OP
openchat-3.6-8b-20240522
Openchat · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
OL
allenai_olmOCR 2 7B 1025
OLMo · lmstudio-community
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Qwen3 Sex
Qwen · ystemsrx
Params4B
Context256K
Smallest4.3 GB
GGUF
QW
Qwen3.5 4B Claude 4.6 Opus Reasoning Distilled v2
Qwen · Jackrong
Params4.2B
Context256K
Q4_K_M2.7 GB
ReasoningVision
HU
Hunyuan-MT-7B
Hunyuan · Mungert
Params7.5B
Context256K
Q4_K_M4.7 GB
GGUF
QW
Qwen Image 2.1 Text Encoder
Qwen · darrellbest
Params8.2B
Context256K
Q4_K_M5.7 GB
CodingVision
MI
Mistral-7B-Instruct-v0.3
Mistral · lmstudio-community
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
FL
mindview-t2i
Flux · mohsenvand
Params4.65B
Context—
Smallest1.2 GB
GGUF
QW
Qwen3.5 9B Heretic
Qwen · darrellbest
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
QW
Qwen2.5 7B Instruct
Qwen · paultimothymooney
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
IN
InternVL3_5 8B
Internvl · bartowski
Params8.19B
Context40K
Q4_K_M5.0 GB
Vision
GE
gemma 4 E2B it
Gemma · AtomicChat
Params4.65B
Context128K
Q4_K_M3.4 GB
ReasoningTool use
GE
Gemma 2 9b It
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
CA
causallm_7b
TheBloke
Params7.72B
Context8K
Q4_K_M4.8 GB
Tool useVision
QW
Qwen3.5 9B Heretic v2
Qwen · filvyb
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
QW
Qwen Image 2.1 Turbo Quantized
Qwen · abenzerps
Params7.1B
Context—
Q4_K_M4.1 GB
GGUF
NE
NeoHorse-1-9B
Neohorse · TokenRhythm
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
DE
deepseek-ai_deepseek-coder-6.7b-base
DeepSeek · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
JA
JackOD 9B Coder
Jackod · ManniX-ITA
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningTool use
QW
Qwen2.5 7B Instruct Uncensored
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3.5 9B GLM5.1 Distill v1
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
HO
K2 Horizon 7B Heretic
Horizon · mradermacher
Params9B
Context512K
Q4_K_M5.6 GB
GGUF

Other sizes

Keep browsing