Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
IM
Z Image Turbo
Image · tsqn
Params6.15B
Context—
Q4_K_M5.0 GB
QW
GPT neo Custom 7B
Qwen · Szymon-Wachelko
Params7.6B
Context32K
Q4_K_M4.7 GB
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Q4_K_M5.3 GB
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
gemma 4 E4B it heretic
Gemma · igorls
Params7.5B
Context128K
Q4_K_M5.3 GB
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
QW
Qwen3.8 9B Function Calling xLAM Unsloth
Qwen · ermiaazarkhalili
Params9.2B
Context256K
Q4_K_M5.8 GB
NO
nomic ai nomic embed code
Nomic · bartowski
Params7.1B
Context32K
Q4_K_M4.4 GB
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
MI
Huihui MiniCPM V 4 5 abliterated
MiniCPM · huihui-ai
Params8.19B
Context40K
Q4_K_M5.0 GB
AI
aiXcompact 4B
Aixcompact · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
LL
LLaVA-NeXT-Video-7B-DPO
LLaVA · ManishThota
Params6.74B
Context4K
Smallest7.2 GB
PH
Phi 3 Mini 4k Instruct
Phi · second-state
Params3.82B
Context4K
Q4_K_M2.4 GB
RW
rwitz_go-bruins
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
dolphin 2.9.4 llama3.1 8b Q4 K M
Llama · yaneivan
Params8B
Context128K
Q4_K_M4.9 GB
NE
models--automerger--NeuralsirkrishnaInex12-7B
Neuralsirkrishna · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
QW
Qwen3.5 9b Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
QW
Qwen3.5 4B
Qwen · Smoffyy
Params4.21B
Context256K
Q4_K_M2.7 GB
SH
Shaylah Darija 4b Merged
Shaylah · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ZO
Zoro i1
Zoro · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
QW
Qwen1.5-7B-Chat-AWQ-fp16
Qwen · Qwen
Params7.72B
Context32K
Q4_K_M4.8 GB
GE
Gemma 3 4B it Uncensored
Gemma · lemuralabs
Params3.88B
Context128K
Smallest1.2 GB
HE
Hermes 2 DPO WestLake 7B v2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
QW
Twi Qwen v1 Merged
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ME
MechVL 4B RL
Mechvl · mradermacher
Params4.41B
Context256K
Q4_K_M2.7 GB
LL
Nous Hermes Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB