Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
ME
malhajar_meditron-7b-chat
Meditron · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
IN
internlm3 8b instruct
InternLM · internlm
Params8.8B
Context32K
Q4_K_M5.4 GB
GGUF
QW
Qwen3.5 9B Gemini 3.1 Pro Reasoning Distill
Qwen · Tet3u
Params8.95B
Context256K
Q4_K_M5.7 GB
ReasoningVision
LL
liberalis-cogitator-llama-3.1-8b-dpo
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
DE
Qwen3.5 9B DeepSeek V4 Flash MTP HF
DeepSeek · Jackrong
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool use
KR
Kreatix K1
Kreatix · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
NE
neural chat 7b v3 3
Neural · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OL
Olmo 3 7B Instruct SFT
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Tool use
LL
llama 2 7b chat sexed version3
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.2 GB
GGUF
LL
llm-compiler-7b-ftd
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
GGUF
GE
Aura v1.1
Gemma · UncannyEcho
Params7.46B
Context128K
Smallest5.2 GB
Vision
QW
Qwen Image 2.1 Turbo
Qwen · off-grid-ai
Params7.12B
Context—
Smallest4.1 GB
GGUF
LL
Llama 3.1 8B Instruct Jbliterated v2
Llama · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Mistral 7B Instruct v0.3 Heretic
Mistral · saidutta69
Params7.25B
Context32K
Q4_K_M4.4 GB
Tool use
ON
OneDecision VisionGuard 4B SFT
Onedecision · prithivMLmods
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
GE
Gemma 3 4B VL It GLM 4.7 Flash Heretic Uncensored Thinking
Gemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
ReasoningVision
IN
AlchemistCoder-DS-6.7B
InternLM · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
SP
Spark X2.5 4b
Spark · Myric
Params4.11B
Context1024K
Q4_K_M2.5 GB
GGUF
QW
Qwable 9B Claude Fable 5 OBLITERATED
Qwable · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
CO
Codegeex4 All 9b
Codegeex · mradermacher
Params9.4B
Context128K
Q4_K_M6.3 GB
GGUF
GA
udkai_garrulus
Garrulus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
Nemotron H 8B Reasoning 128K
Nemotron · bartowski
Params8.1B
Context1024K
Q4_K_M5.0 GB
Reasoning
MI
Mistral 7B Instruct v0.2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 4B Cybersecurity Heretic 16bit
Qwen · sillykiwi
Params4B
Context256K
Q4_K_M2.5 GB
GGUF
QW
Qwen3 Sex
Qwen · mradermacher
Params4.02B
Context256K
Q4_K_M2.5 GB
Tool use
SK
Skywork OR1 7B Preview
Skywork · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
GE
Aidc 5k Laos Gemma 3 4b It v2
Gemma · mradermacher
Params4.55B
Context128K
Q4_K_M2.9 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · duyntnet
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
LL
Code-Llama-Bagel-8B
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
Coding
ON
OneDecision VisionGuard 9B SFT
Onedecision · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision

Other sizes

Keep browsing