Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HU
HunyuanVideo 1.5 T2V 720p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
GGUF
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-1-7b
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PH
FC PHI 3.5 Mini Merged 1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
LE
leolm_leo-hessianai-7b-chat
Leo · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
LL
SEVENLLM-Llama2-7B-CoT
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
DE
DeepSeek R1 0528 Qwen3 8B Heretic
DeepSeek · mradermacher
Params8.19B
Context128K
Q4_K_M5.0 GB
Reasoning
QW
OpenSpatial Qwen2.5 VL 7B
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
IN
InfinToppyKuno DARE 7bv2
Infintoppykuno · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Negentropy claude opus 4.7 9B
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningVision
LL
llama-3-sqlcoder-8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
Coding
PH
Phi 3 Mini 4k Instruct
Phi · SanctumAI
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
EX
models--automerger--Experiment26Mergerix-7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PE
Peach 2.0 9B 8k Roleplay
Peach · fengpeisheng1
Params8.83B
Context16K
Smallest4.8 GB
Roleplay
LL
Llama 3.1 8B xLAM Agent
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
GR
Granite 3.0 8b Lab Community
Granite · bartowski
Params8.17B
Context4K
Q4_K_M4.9 GB
GGUF
NE
abideen_nexonimbus-7b
Nexonimbus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
L3 SthenoMaidBlackroot 8B V1
Sthenomaidblackroot · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TH
TheSpice-7b-FT-v0.3.1
Thespice · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GA
Garryvik-0.1-7b-Linear
Garryvik · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GE
Gemma4 E4B Mahou Nsfw
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
Roleplay
LL
llama 2 7b chat sexed version5
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PA
Palette RP 9B 2609 v0.05
Palette · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
Roleplay
QW
Qwen3.5 9B GLM5.1 Distill v1
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
SC
ScienceRSI 8b
Sciencersi · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
JU
juanako 7b UNA i1
Juanako · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma-2-9B-It-SPPO-Iter3
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
HE
HermesLimaRP L2 7B
Hermeslimarp · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
Tool use

Other sizes

Keep browsing