Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Gmcoder Bf16
Qwen · emperorofrome
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoning
GR
Granite 3.1 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Hermes-2-Theta-Llama-3-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
Tool use
LL
Llama 3.3 8B Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
DeepSeek V4 Pro Qwen3.5 4B MTP Public 20260801 HF
Qwen · Jackrong
Params4.33B
Context256K
Q4_K_M2.8 GB
ReasoningVision
PE
Peach 2.0 9B 8k Roleplay
Peach · QuantFactory
Params8.8B
Context16K
Q4_K_M5.3 GB
Roleplay
MI
argilla_capybarahermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
TE
Tev1 4B Experimental
Tev · bartowski
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
QW
Qwen2.5 VL 7B NSFW Caption V3 Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
Vision
QW
Mica v0.5 4B
Qwen · sky7350
Params4.21B
Context256K
Smallest2.5 GB
ReasoningTool use
ST
StartLux Decision v1 4B Q4 K M
Startlux · startlux-models
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
JA
Jan v2 VL med
Jan · janhq
Params8.2B
Context256K
Q4_K_M5.0 GB
Vision
AM
amalia-llm/AMALIA-9B-0626-DPO
Amalia · layerx-labs
Params9.15B
Context32K
Q4_K_M5.6 GB
GGUF
QW
Qwen3.5 9b Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool use
LL
Meta Llama 3.1 8B Instruct
Llama · joshnader
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
CH
AA Chameleon 7b Base
Chameleon · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
QW
MiMo V2.6 Distill Qwen 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
CodingReasoningRoleplay
HU
Hunyuan 7B Instruct
Hunyuan · bartowski
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
GE
Gemma 3n E4B It
Gemma · bartowski
Params6.9B
Context32K
Q4_K_M4.2 GB
GGUF
LL
CodeLlama 7b Python Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
AR
Artex Coder 7b
Artex · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
MT
Mtrini SVL 2.0
Mtrini · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
NO
nousresearch_nous-capybara-7b-v1.9
Nous · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 4.2 8b
Granite · bartowski
Params8.79B
Context128K
Q4_K_M5.5 GB
ReasoningTool use
MI
ehartford_dolphin-2.0-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OR
Ornith-1.5-9B
Ornith · Mungert
Params9.2B
Context256K
Q4_K_M6.1 GB
GGUF
LE
LeetCodeWizard 7B V1.1 i1
Leetcodewizard · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
GE
gemma 3n E4B it text
Gemma · lmstudio-community
Params6.9B
Context32K
Q4_K_M4.2 GB
GGUF
QW
Qwen Image 2.1 Anime Fusal
Qwen · LocalAI-io
Params7.12B
Context256K
Q4_K_M5.0 GB
Vision
YI
01 ai Yi 6B 200K
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB
GGUF

Other sizes

Keep browsing