Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
HA
Harpy 2 7B TIES
Harpy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
UC
UCLA AGI zephyr 7b sft full SPIN iter2
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NI
NicholasCorrado zephyr 7b uf rlced conifer dpo 2e
Nicholascorrado · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 Sex
Qwen · Wlc7758
Params4B
Context256K
Smallest4.3 GB
UN
Unity Coder 7b
Unity · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
PH
Phi 3.1 mini 4k instruct
Phi · lmstudio-community
Params3.82B
Context4K
Q4_K_M2.4 GB
GL
GLM4 9B Neon v2
GLM · bartowski
Params9.4B
Context32K
Q4_K_M6.2 GB
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Smallest1.9 GB
NE
Nemotron 3 Embed 8B BF16
Nemotron · Aqua00
Params7.95B
Context256K
Q4_K_M4.9 GB
BL
Blossom V6.2 8B
Blossom · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
SE
SeaLLM-7B-v2-bonito-v1-slerp
Seallm · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Dolphin 3.0 Llama 3.2 3B
Llama · QuantFactory
Params3.61B
Context128K
Q4_K_M2.2 GB
SH
shadow clown 7B slerp
Shadow · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
FL
Flow Judge v0.1
Flow · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
ME
Megamind v2 VL high
Megamind · digitranslab
Params8.19B
Context256K
Q4_K_M5.0 GB
LL
cognitivecomputations dolphin 2.9 llama3 8b
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
QW
LLDS A GRPO Qwen2.5 7B Ins i1
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
YA
models--automerger--YamExperiment26-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 8B Instruct RoleVerse SFT DPO
Llama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mixtral AI AlphaTron
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
SW
Swallow-7b-instruct-v0.1
Swallow · mradermacher
Params6.83B
Context4K
Q4_K_M4.1 GB
FL
flux 3 image klein sd
Flux · akhaliq
Params5.55B
Context—
Q4_K_M3.3 GB
FR
fredithefish_openzephyrchat
Fredithefish · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 Sex
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
MP
Mpt 7b 8k Chat
Mpt · mradermacher
Params6.65B
Context8K
Q4_K_M4.3 GB
CH
AA Chameleon 7b Base
Chameleon · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
MI
KawaiiMahou-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma Kimu 9b It
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
NO
notus 7b v1
Notus · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · diablomike20
Params8.95B
Context256K
Q4_K_M6.8 GB