Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta Llama 3 8B Instruct
Llama · MaziyarPanahi
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Huihui Qwen3 8B Abliterated v2
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
GE
Gemma 3 4b It
Gemma · lmstudio-community
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
QW
Qwen3 Embedding 4B
Qwen · Qwen
Params4B
Context40K
Q4_K_M2.5 GB
Embedding
QW
Qwen Image 2.1 Turbo
Qwen · Abiray
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
QW
Qwen_Qwen3 VL 8B Instruct
Qwen · lmstudio-community
Params8.19B
Context256K
Q4_K_M5.0 GB
Vision
IN
Index Translate 9B
Index · IndexTeam
Params9.2B
Context256K
Q4_K_M5.8 GB
Vision
QW
Huihui Qwythos 9B Claude Mythos 5 1M Abliterated MTP
Qwythos · huihui-ai
Params9.2B
Context1024K
Smallest5.8 GB
ReasoningTool useVision
WA
Wald-4B v1.2
Wald · org2ai
Params4.2B
Context256K
Q4_K_M2.7 GB
Vision
GE
Gemma-3N-E2B-It
Gemma · unsloth
Params4.46B
Context32K
Q4_K_M3.0 GB
Vision
LL
Llama3.3 8B Instruct Thinking Heretic Uncensored Claude 4.5 Opus High Reasoning
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
ReasoningRoleplay
GE
Models Google Gemma 3 4b It
Gemma · MaziyarPanahi
Params3.88B
Context128K
Q4_K_M2.5 GB
GGUF
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · lmstudio-community
Params8.19B
Context128K
Q4_K_M5.0 GB
Reasoning
QW
Qwen Image 2.1 viggle 4 steps turbo
Qwen · Abiray
Params7.12B
Context—
Q4_K_M4.2 GB
GGUF
KE
Kev 4B
Kev · ggml-org
Params4.2B
Context256K
Q4_K_M3.0 GB
GGUF
QW
Qwen3.8 9B Heretic Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen Image 2.1 PE I2I
Qwen · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
WA
Wan2.2 TI2V 5B Turbo
Wan · hum-ma
Params5B
Context—
Q4_K_M3.4 GB
GGUF
NE
Nemotron 3.5 Bolt E4B
Nemotron · nvidia
Params7.52B
Context64K
Q4_K_M6.2 GB
ReasoningTool use
MI
models--mistralai--Mistral-7B-Instruct-v0.3
Mistral · MaziyarPanahi
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
CO
C4Ai Command R7B 12 2024
Command R · bartowski
Params8B
Context8K
Q4_K_M5.1 GB
GGUF
QW
Qween3 4B thinking finetune
Qween · pramodlohra
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
LI
Ling 3.0 Tiny
Ling · bartowski
Params7.89B
Context128K
Q4_K_M4.9 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · MaziyarPanahi
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
VO
Voxtral Mini 3B 2507
Voxtral · ggml-org
Params4B
Context128K
Q4_K_M2.5 GB
Vision
FL
flux 3 image klein
Flux · leejet
Params5.55B
Context—
Smallest2.3 GB
GGUF
MI
Ministral 3 8B Instruct 2512
Mistral · mistralai
Params8.49B
Context256K
Q4_K_M5.2 GB
Vision
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · bartowski
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
QW
Qwen3.5 9B DeepSeek V4 Flash
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
QW
Qwen2.5 VL 7B Instruct
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision

Other sizes

Keep browsing