Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Gmcoder Bf16
Qwen · emperorofrome
Params8.95B
Context256K
Q4_K_M5.6 GB
GR
Granite 3.1 8b Instruct
Granite · bartowski
Params8.17B
Context128K
Q4_K_M4.9 GB
LL
Hermes-2-Theta-Llama-3-8B
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
LL
Llama 3.3 8B Instruct
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
QW
DeepSeek V4 Pro Qwen3.5 4B MTP Public 20260801 HF
Qwen · Jackrong
Params4.33B
Context256K
Q4_K_M2.8 GB
PE
Peach 2.0 9B 8k Roleplay
Peach · QuantFactory
Params8.8B
Context16K
Q4_K_M5.3 GB
MI
argilla_capybarahermes-2.5-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
TE
Tev1 4B Experimental
Tev · bartowski
Params4.33B
Context256K
Q4_K_M2.8 GB
QW
Qwen2.5 VL 7B NSFW Caption V3 Abliterated
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
QW
Mica v0.5 4B
Qwen · sky7350
Params4.21B
Context256K
Smallest2.5 GB
ST
StartLux Decision v1 4B Q4 K M
Startlux · startlux-models
Params4.21B
Context256K
Q4_K_M2.7 GB
JA
Jan v2 VL med
Jan · janhq
Params8.2B
Context256K
Q4_K_M5.0 GB
AM
amalia-llm/AMALIA-9B-0626-DPO
Amalia · layerx-labs
Params9.15B
Context32K
Q4_K_M5.6 GB
QW
Qwen3.5 9b Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Meta Llama 3.1 8B Instruct
Llama · joshnader
Params8B
Context128K
Q4_K_M4.9 GB
CH
AA Chameleon 7b Base
Chameleon · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
QW
MiMo V2.6 Distill Qwen 9B Heretic
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
HU
Hunyuan 7B Instruct
Hunyuan · bartowski
Params7.5B
Context256K
Q4_K_M4.6 GB
GE
Gemma 3n E4B It
Gemma · bartowski
Params6.9B
Context32K
Q4_K_M4.2 GB
LL
CodeLlama 7b Python Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
AR
Artex Coder 7b
Artex · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
MT
Mtrini SVL 2.0
Mtrini · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
NO
nousresearch_nous-capybara-7b-v1.9
Nous · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 4.2 8b
Granite · bartowski
Params8.79B
Context128K
Q4_K_M5.5 GB
MI
ehartford_dolphin-2.0-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
Ornith-1.5-9B
Ornith · Mungert
Params9.2B
Context256K
Q4_K_M6.1 GB
LE
LeetCodeWizard 7B V1.1 i1
Leetcodewizard · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
GE
gemma 3n E4B it text
Gemma · lmstudio-community
Params6.9B
Context32K
Q4_K_M4.2 GB
QW
Qwen Image 2.1 Anime Fusal
Qwen · LocalAI-io
Params7.12B
Context256K
Q4_K_M5.0 GB
YI
01 ai Yi 6B 200K
Yi · RichardErkhov
Params6.06B
Context195K
Q4_K_M3.7 GB