Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IM
Z Image Turbo
Image · tsqn
Params6.15B
Context—
Q4_K_M5.0 GB
GGUF
QW
GPT neo Custom 7B
Qwen · Szymon-Wachelko
Params7.6B
Context32K
Q4_K_M4.7 GB
Coding
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
gemma 4 E4B it heretic
Gemma · igorls
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.8 9B Function Calling xLAM Unsloth
Qwen · ermiaazarkhalili
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
NO
nomic ai nomic embed code
Nomic · bartowski
Params7.1B
Context32K
Q4_K_M4.4 GB
CodingEmbedding
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
MI
Huihui MiniCPM V 4 5 abliterated
MiniCPM · huihui-ai
Params8.19B
Context40K
Q4_K_M5.0 GB
CodingVision
AI
aiXcompact 4B
Aixcompact · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoningTool use
LL
LLaVA-NeXT-Video-7B-DPO
LLaVA · ManishThota
Params6.74B
Context4K
Smallest7.2 GB
EmbeddingVision
PH
Phi 3 Mini 4k Instruct
Phi · second-state
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
RW
rwitz_go-bruins
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
dolphin 2.9.4 llama3.1 8b Q4 K M
Llama · yaneivan
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
NE
models--automerger--NeuralsirkrishnaInex12-7B
Neuralsirkrishna · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Qwen3.5 9b Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool useVision
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.5 4B
Qwen · Smoffyy
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
SH
Shaylah Darija 4b Merged
Shaylah · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
ZO
Zoro i1
Zoro · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool use
QW
Qwen1.5-7B-Chat-AWQ-fp16
Qwen · Qwen
Params7.72B
Context32K
Q4_K_M4.8 GB
GGUF
GE
Gemma 3 4B it Uncensored
Gemma · lemuralabs
Params3.88B
Context128K
Smallest1.2 GB
GGUF
HE
Hermes 2 DPO WestLake 7B v2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
QW
Twi Qwen v1 Merged
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
ME
MechVL 4B RL
Mechvl · mradermacher
Params4.41B
Context256K
Q4_K_M2.7 GB
ReasoningVision
LL
Nous Hermes Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
Tool use

Other sizes

Keep browsing