Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ME
models--automerger--MeliodasM7-7B
Meliodas · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
pocketdoc_dans-totsirocco-7b
Dans · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.8 Flash Next
Qwen · kingjones777
Params3.88B
Context256K
Q4_K_M2.8 GB
GGUF
QW
dolphin 2.9.2 qwen2 7b
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
CodingTool use
MI
MistralLite 7B
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwythos 9B v2 Heretic
Qwythos · WaveCut
Params8.95B
Context1024K
Q4_K_M5.6 GB
GGUF
OP
weyaxi_openhermes-2.5-neural-chat-7b-v3-2-7b
Openhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
SH
OS Shepherd 9B
Shepherd · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Vision
QW
Qwen3.5 9B
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
SA
GGUF.chatgpt gpt codex V2
Safetensors · mondk
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
GL
LongWriter Glm4 9b
GLM · bartowski
Params9.4B
Context1024K
Q4_K_M6.3 GB
GGUF
GE
Gemma2 Gutenberg Doppel 9B
Gemma · bartowski
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
Huihui Qwen3.5 9B abliterated
Qwen · nuofang
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
IC
IceBlendedLatteRP-7b
Iceblendedlatterp · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ELYZA-japanese-Llama-2-7b-fast-instruct
Llama · mmnga
Params6.85B
Context4K
Q4_K_M4.2 GB
GGUF
CO
Concept 7b V1 Full
Concept · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
SO
Sorjin1.1 7B Instruct
Sorjin · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3.5 9B Unredacted MAX
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Reasoning
NE
mlabonne_neuralpipe-7b-slerp
Neuralpipe · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
EI
Einstein-v4-7B
Einstein · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
CodingTool use
IN
Internlm3 8b Instruct
InternLM · bartowski
Params8.8B
Context32K
Q4_K_M5.4 GB
GGUF
SY
maywell_synatra-v0.1-7b-instruct
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · qwp4w3hyb
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
IN
InternVL3_5 8B
Internvl · lmstudio-community
Params8.19B
Context40K
Q4_K_M5.0 GB
Vision
SA
SambaLingo Hungarian Chat
Sambalingo · mradermacher
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
PH
calme-2.1-phi3.5-4b
Phi · mradermacher
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
TO
tokyotech llm Swallow 7b instruct hf
Tokyotech · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Lebanese Llama 3.1 8b
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
NE
NeonLLM-Base
Neonllm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
Tool use
DE
DeepPresenter 9B
Deeppresenter · Forceless
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF

Other sizes

Keep browsing