Tiny GGUF models

62726 GGUF models with 3B parameters or fewer. Sizes run from 1M to 3.5B parameters. The longest trained context is 795898K tokens.

Most downloadedNewestLargestLongest contextSmallest
YU
YuE2 audio tokenizer, the v9 head of Mothersuperior, on MERT-v2-FullSong
Yue · Serveurperso
Params632M
Context—
Smallest0.0 GB
GGUF
QW
Qwen3.8-27B iMatrix NVFP4 MTP
Qwen · cdiamond
Params461M
Context256K
Smallest17.1 GB
Vision
OR
Tiel Coder 35B A3B GGUF MTP
Ornith · peculiar-ragdoll
Params447M
Context256K
Smallest12.7 GB
CodingVision
QW
Qwen3.8-27B
Qwen · ggml-org
Params3B
Context256K
Q4_K_M19.0 GB
Vision
MX
mxbai-embed-large-v1
Mxbai · mixedbread-ai
Params334M
Context512
Smallest0.7 GB
Embedding
PA
Parakeet Unified EN 0.6B
Parakeet · handy-computer
Params618M
Context—
Q4_K_M0.5 GB
GGUF
NE
nemotron 3.5 asr streaming 0.6b
Nemotron · handy-computer
Params638M
Context—
Q4_K_M0.5 GB
GGUF
VI
VieNeu-TTS-v3-Turbo-staging
Vieneu · pnnbao-ump
Params146M
Context—
Smallest0.2 GB
GGUF
CO
Cohere Transcribe
Cohere · handy-computer
Params2.05B
Context—
Q4_K_M1.6 GB
GGUF
HY
HyperCLOVAX SEED Text Instruct 1.5B
Hyperclovax · rippertnt
Params1.81B
Context128K
Q4_K_M1.1 GB
GGUF
NE
nemotron-3.5-asr-streaming-0.6b.q8_0
Nemotron · nvidia
Params648M
Context—
Smallest0.7 GB
GGUF
PA
Parakeet TDT 0.6B v3
Parakeet · handy-computer
Params627M
Context—
Q4_K_M0.5 GB
GGUF
MI
MiniMax-Music3 flow matching DiT
Minimax · Serveurperso
Params2.4B
Context—
Q4_K_M1.4 GB
GGUF
QW
Qwen3-TTS-Tokenizer-12Hz
Qwen · Serveurperso
Params1.9B
Context—
Q4_K_M0.3 GB
GGUF
WH
Whisper Medium
Whisper · handy-computer
Params764M
Context—
Q4_K_M0.5 GB
GGUF
GE
gemma 4 E2B it qat mobile
Gemma · unsloth
Params92M
Context128K
Smallest0.1 GB
Vision
PA
parakeet-tdt-0.6b-v3.oss-align.q8_0
Parakeet · nvidia
Params637M
Context—
Smallest0.7 GB
GGUF
WH
Whisper Large v3 Turbo
Whisper · handy-computer
Params809M
Context—
Q4_K_M0.5 GB
GGUF
JI
jinen-v1-small-hf
Jinen · togatogah
Params110M
Context1K
Smallest0.1 GB
GGUF
SU
surya ocr 2
Surya · datalab-to
Params632M
Context256K
Smallest1.3 GB
Vision
NO
nomic-embed-text-v1.5
Nomic · nomic-ai
Params137M
Context2K
Q4_K_M0.1 GB
Embedding
LF
LFM2.5 2.6B
Lfm · LiquidAI
Params2.7B
Context128K
Q4_K_M1.7 GB
GGUF
AD
AdaVaR 3B
Adavar · mradermacher
Params3.09B
Context125K
Q4_K_M1.9 GB
Vision
QW
Qwen3.8 2B
Qwen · empero-ai
Params1.94B
Context256K
Q4_K_M1.3 GB
Reasoning
OR
Ornith 1.5 35B A3B APEX MTP
Ornith · mudler
Params447M
Context256K
Smallest14.4 GB
Vision
MO
models moved
Models · ggml-org
Params34M
Context—
Smallest0.0 GB
GGUF
WH
Whisper Large v3
Whisper · handy-computer
Params1.54B
Context—
Q4_K_M1.0 GB
GGUF
QW
Qwen3.5-2B
Qwen · unsloth
Params1.88B
Context256K
Q4_K_M1.3 GB
Vision
QW
qwen2.5-1.5b-instruct
Qwen · Qwen
Params1.78B
Context32K
Q4_K_M1.1 GB
GGUF
EM
Embeddinggemma 300m
Embeddinggemma · ggml-org
Params308M
Context2K
Smallest0.3 GB
Embedding

What counts as a tiny model?

On this site tiny means 3B parameters or fewer. A Q4_K_M file is usually 2 GB or less, so these models run on almost any GPU, on a phone-class device or on a CPU. 62,726 models fit this class. The most downloaded are YuE2 audio tokenizer, the v9 head of Mothersuperior, on MERT-v2-FullSong, Qwen3.8-27B iMatrix NVFP4 MTP and Tiel Coder 35B A3B GGUF MTP.

Their Q4_K_M files run 0.0 to 444.7 GB. The families you will see most are Qwen (12810), Llama (7735) and Gemma (4035). The size class sorts models by parameter count, so a model with more experts or a longer context can need more memory than its class suggests. The model pages show the real file sizes.

Frequently asked questions

How much VRAM does a tiny GGUF model need?

A Q4_K_M file is usually 2 GB or less, so these models run on almost any GPU, on a phone-class device or on a CPU.

Which tiny GGUF model is the most popular?

By Hugging Face downloads, YuE2 audio tokenizer, the v9 head of Mothersuperior, on MERT-v2-FullSong.

Other sizes

Keep browsing