Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HI
HiDream O1 Image Dev
Hidream · smthem
Params8.8B
Context—
Smallest9.9 GB
GGUF
AN
sao10k_ana-v1-m7
Ana · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
Deepseek Coder 6.7B Instruct
DeepSeek · second-state
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
Mistral 7B Instruct DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
JA
JapanRailwayLLM
Japanrailwayllm · mradermacher
Params9.2B
Context8K
Q4_K_M5.8 GB
GGUF
LL
CodeLlama 7b Hf
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QW
Qwen3 8B Q8 0 full v2
Qwen · luckypoolio
Params8.19B
Context40K
Smallest8.7 GB
GGUF
GE
Gemma 4 E4B IT ABLITERATED UNCENSORED PHILADELPHIA CLASS
Gemma · mradermacher
Params7.46B
Context128K
Q4_K_M5.3 GB
GGUF
BO
Boptruth Agatha 7B i1
Boptruth · mradermacher
Params7.24B
Context128K
Q4_K_M4.4 GB
GGUF
EM
Emotion Companion 7b
Emotion · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Fireball Meta Llama 3.2 8B Instruct agent 003 128k code DPO i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Coding
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
Vision
ON
OneJev 9B
Onejev · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Vision
NE
Nemotron 3.5 Bolt E4B Heretic
Nemotron · mradermacher
Params7.52B
Context64K
Q4_K_M5.3 GB
ReasoningTool useVision
QW
Qwen2.5 7B Instruct Heretic
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen3.5 4B RpRMax v1
Qwen · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
GGUF
GE
Gemma 3 4b It Abliterated v2
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
OG
models--automerger--OgnoExperiment27-7B
Ogno · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
VideoGuard Qwen3.5 4B Safety RL Uncensored
Qwen · prithivMLmods
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningVision
IC
IceEspressoRPv6 i1
Iceespressorpv · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.6 9B Heretic Uncensored Thinking Sweet Madness
Qwen · Prototype3550
Params8.63B
Context256K
Q4_K_M5.5 GB
ReasoningRoleplayVision
FL
FLUX.3 Image Klein
Flux · DogukanUrker
Params5.55B
Context—
Smallest2.5 GB
GGUF
LL
Llama 3.1 Tulu 3 8B
Llama · lmstudio-community
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
TH
Threnystril v2.0 7B slerp
Threnystril · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama-3-Instruct-8B-SPPO-Iter3
Llama · legraphista
Params8B
Context8K
Smallest2.0 GB
GGUF
LL
Emma 500 Llama2 7b
Llama · bartowski
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
AS
astrollama 3 8b chat aic i1
Astrollama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
SE
Seed X PPO 7B
Seed · mradermacher
Params7.51B
Context32K
Q4_K_M4.6 GB
GGUF
QW
Qwen3 VL 4B Instruct Unredacted MAX
Qwen · prithivMLmods
Params4B
Context256K
Smallest4.3 GB
Vision

Other sizes

Keep browsing