Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
HU
HunyuanVideo 1.5 T2V 720p
Hunyuanvideo · jayn7
Params8.33B
Context—
Q4_K_M5.1 GB
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-1-7b
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
FC PHI 3.5 Mini Merged 1
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.3 GB
LE
leolm_leo-hessianai-7b-chat
Leo · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
LL
SEVENLLM-Llama2-7B-CoT
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen2.5 7B Instruct Abliterated v2
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
FR
FrogNano 4B 2609
Frognano · mradermacher
Params4.33B
Context256K
Q4_K_M2.8 GB
DE
DeepSeek R1 0528 Qwen3 8B Heretic
DeepSeek · mradermacher
Params8.19B
Context128K
Q4_K_M5.0 GB
QW
OpenSpatial Qwen2.5 VL 7B
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
IN
InfinToppyKuno DARE 7bv2
Infintoppykuno · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
QW
Negentropy claude opus 4.7 9B
Qwen · Jackrong
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
llama-3-sqlcoder-8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 Mini 4k Instruct
Phi · SanctumAI
Params3.82B
Context4K
Q4_K_M2.4 GB
EX
models--automerger--Experiment26Mergerix-7B
Experiment · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
PE
Peach 2.0 9B 8k Roleplay
Peach · fengpeisheng1
Params8.83B
Context16K
Smallest4.8 GB
LL
Llama 3.1 8B xLAM Agent
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GR
Granite 3.0 8b Lab Community
Granite · bartowski
Params8.17B
Context4K
Q4_K_M4.9 GB
NE
abideen_nexonimbus-7b
Nexonimbus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
L3 SthenoMaidBlackroot 8B V1
Sthenomaidblackroot · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
TH
TheSpice-7b-FT-v0.3.1
Thespice · duyntnet
Params7.24B
Context32K
Q4_K_M4.4 GB
GA
Garryvik-0.1-7b-Linear
Garryvik · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
PiSSA Llama 2 7b R128
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GE
Gemma4 E4B Mahou Nsfw
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
LL
llama 2 7b chat sexed version5
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
PA
Palette RP 9B 2609 v0.05
Palette · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
QW
Qwen3.5 9B GLM5.1 Distill v1
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
SC
ScienceRSI 8b
Sciencersi · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
JU
juanako 7b UNA i1
Juanako · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma-2-9B-It-SPPO-Iter3
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
HE
HermesLimaRP L2 7B
Hermeslimarp · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB