Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HA
Harpy 2 7B TIES
Harpy · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter2
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NI
NicholasCorrado zephyr 7b uf rlced conifer dpo 2e
Nicholascorrado · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Sex
Qwen · Wlc7758
Params4B
Context256K
Smallest4.3 GB
GGUF
UN
Unity Coder 7b
Unity · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
PH
Phi 3.1 mini 4k instruct
Phi · lmstudio-community
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
GL
GLM4 9B Neon v2
GLM · bartowski
Params9.4B
Context32K
Q4_K_M6.2 GB
GGUF
MI
Minitron 4B Base
Minitron · legraphista
Params4.19B
Context4K
Smallest1.9 GB
GGUF
NE
Nemotron 3 Embed 8B BF16
Nemotron · Aqua00
Params7.95B
Context256K
Q4_K_M4.9 GB
Embedding
BL
Blossom V6.2 8B
Blossom · mradermacher
Params8.19B
Context32K
Q4_K_M5.0 GB
GGUF
SE
SeaLLM-7B-v2-bonito-v1-slerp
Seallm · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Dolphin 3.0 Llama 3.2 3B
Llama · QuantFactory
Params3.61B
Context128K
Q4_K_M2.2 GB
CodingTool use
SH
shadow clown 7B slerp
Shadow · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
FL
Flow Judge v0.1
Flow · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
ME
Megamind v2 VL high
Megamind · digitranslab
Params8.19B
Context256K
Q4_K_M5.0 GB
ReasoningTool useVision
LL
cognitivecomputations dolphin 2.9 llama3 8b
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
LLDS A GRPO Qwen2.5 7B Ins i1
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
YA
models--automerger--YamExperiment26-7B
Yam · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 8B Instruct RoleVerse SFT DPO
Llama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
RoleplayTool use
MI
Mixtral AI AlphaTron
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SW
Swallow-7b-instruct-v0.1
Swallow · mradermacher
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
FL
flux 3 image klein sd
Flux · akhaliq
Params5.55B
Context—
Q4_K_M3.3 GB
GGUF
FR
fredithefish_openzephyrchat
Fredithefish · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Sex
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
MP
Mpt 7b 8k Chat
Mpt · mradermacher
Params6.65B
Context8K
Q4_K_M4.3 GB
Coding
CH
AA Chameleon 7b Base
Chameleon · mradermacher
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
MI
KawaiiMahou-mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma Kimu 9b It
Gemma · mradermacher
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
NO
notus 7b v1
Notus · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
Qwen · diablomike20
Params8.95B
Context256K
Q4_K_M6.8 GB
CodingReasoningRoleplay

Other sizes

Keep browsing