Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
PA
pipizhao_pandalyst-7b-v1.1
Pandalyst · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
IN
InternVL3 8B Instruct
Internvl · ggml-org
Params7.61B
Context32K
Q4_K_M4.7 GB
TO
ToriiGate v0.4 7B
Toriigate · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Llama 3.1 8B Stheno v3.4
Llama · Lewdiculous
Params8B
Context128K
Q4_K_M4.9 GB
MI
Mistral Pygmalion 7B
Mistral · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen3.5 9B Harmonic
Qwen · DJLougen
Params8.95B
Context256K
Q4_K_M5.6 GB
HO
K2 Horizon 3.7B Heretic
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
HO
K2 Horizon 7B Candor Uncensored
Horizon · candor-lab
Params9B
Context512K
Q4_K_M5.6 GB
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
MI
OpenHermes-2.5-Mistral-7B-x2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
MiniCPM V 2 6
MiniCPM · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
QW
Qwythos 9B v2
Qwythos · mradermacher
Params9.2B
Context1024K
Q4_K_M5.8 GB
ID
ideogram4 turbo
Ideogram · ChrisColeTech
Params9.28B
Context256K
Q4_K_M5.0 GB
MI
severian_anima-phi-neptune-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite Guardian 4.1 8b
Granite · ibm-granite
Params8.38B
Context128K
Q4_K_M5.1 GB
QW
PopiT Qwen3 4B Medical SFT 1128
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
NE
NeoHorse 1 9 Uncensored
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
UN
ajibawa-2023_uncensored-jordan-7b
Uncensored · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
Llama 3.2 3B Instruct
Llama · mradermacher
Params3.61B
Context128K
Q4_K_M2.2 GB
GL
Huihui GLM 4.6V Flash abliterated
GLM · AliBilge
Params9.4B
Context128K
Q4_K_M6.2 GB
GE
gemma 4 E2B it UD japanese imatrix
Gemma · dahara1
Params4.65B
Context128K
Q4_K_M3.1 GB
QW
Qwen3-Vl-8B-Thinking-1M
Qwen · unsloth
Params8.19B
Context977K
Q4_K_M5.0 GB
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
QW
WhereOPD Qwen3.5 4B
Qwen · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
LL
Llama-3-Instruct-Neurona-8b-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
ST
L3-8B-Stheno-v3.2
Stheno · DavidAU
Params8B
Context8K
Q4_K_M4.9 GB
LL
nousresearch_yarn-llama-2-7b-64k
Llama · TheBloke
Params6.74B
Context64K
Q4_K_M4.1 GB
QW
Qwen3 4B
Qwen · enacimie
Params4.02B
Context40K
Q4_K_M5.0 GB
HO
Sakura K2 Horizon 7B
Horizon · webmp3
Params9B
Context512K
Smallest3.8 GB
DE
Qwen2.5 MOE 2X1.5B DeepSeek Uncensored Censored 4B
DeepSeek · DavidAU
Params4.1B
Context128K
Q4_K_M2.5 GB