Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.5-4B
Qwen · Manojb
Params4.21B
Context256K
Smallest2.9 GB
Vision
LL
Openthai Llama Pretrained 7B
Llama · RichardErkhov
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
LL
LLaMA Pro 8B Instruct
Llama · TheBloke
Params8.36B
Context4K
Q4_K_M5.1 GB
GGUF
ST
Storm-7B
Storm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
AC
AceInstruct 7B
Aceinstruct · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
Coding
GE
Zero Gemma4 E4B OpenZero
Gemma · shafire
Params7.5B
Context128K
Smallest5.9 GB
Tool use
LL
llama-3-8b-samantha
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CO
CodeQwen1.5-7B
Codeqwen · bartowski
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
LL
CaptureTheFlag CypherMindLLM XRLAB
Llama · ErenAta00
Params8B
Context128K
Smallest6.6 GB
GGUF
IN
Inv Konstanta V4 Alpha 7B
Inv · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
BL
BlackMagic-7B
Blackmagic · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
llama 2 7b chat catala
Llama · xaviviro
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
T3Q Mistral Orca Math DPO
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 Furry 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Roleplay
QW
Haru Qwen3 4b
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Roleplay
MI
ANIMA Cognitive Mistral v1 i1
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AH
Ahma-3B
Ahma · mradermacher
Params3.63B
Context2K
Q4_K_M2.8 GB
GGUF
ST
Starcoder2 7b
StarCoder · QuantFactory
Params7.17B
Context16K
Q4_K_M4.5 GB
Coding
LL
CodeLlama 7b Python Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
ME
epfl-llm_meditron-7b
Meditron · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen_Qwen2.5 Coder 7B Instruct
Qwen · smarttasks
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
ME
Merlin Agent
Merlin · bartowski
Params8.95B
Context256K
Q4_K_M5.9 GB
Coding
MI
mistralai_mistral-7b-instruct-v0.1
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GL
Huihui GLM 4.6V Flash Abliterated
GLM · seanbailey518
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
FL
FLUX.2 klein 9B
Flux · drends
Params9.08B
Context—
Q4_K_M5.9 GB
GGUF
QW
OPSD Qwen3 8B
Qwen · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
RA
RakutenAI 7B
Rakutenai · mradermacher
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LL
WizardLM Mistral CodeLlama 7b
Llama · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
IN
Internvl3-8B
Internvl · unsloth
Params7.61B
Context32K
Q4_K_M4.7 GB
CodingVision

Other sizes

Keep browsing