Tiny GGUF models

62726 GGUF models with 3B parameters or fewer. Sizes run from 1M to 3.5B parameters. The longest trained context is 795898K tokens.

Most downloadedNewestLargestLongest contextSmallest
CA
Carbon 3B
Carbon · HuggingFaceBio
Params3.45B
Context32K
Q4_K_M2.3 GB
GGUF
MI
MiniCPM5 1B Claude Opus Fable5 Thinking
MiniCPM · GnLOLot
Params1.08B
Context128K
Q4_K_M0.7 GB
Reasoning
QW
Qwen3Vl 2b Instruct
Qwen · Qwen
Params1.72B
Context256K
Q4_K_M1.1 GB
Vision
LL
Llama 3.2 3B Instruct
Llama · bartowski
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
GE
gemma 4 26B A4B it qat
Gemma · unsloth
Params430M
Context128K
Smallest0.3 GB
Vision
QW
Qwen3 1.7B Instruct
Qwen · Qwen
Params1.72B
Context40K
Smallest1.8 GB
GGUF
GR
Parable Granite 4.1 3B Claude Fable 5
Granite · AnkitAI
Params3.4B
Context128K
Q4_K_M2.1 GB
ReasoningTool use
LL
Llama-3.2-3B-Instruct
Llama · unsloth
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
QW
Qwen3-ASR 0.6B
Qwen · handy-computer
Params782M
Context—
Q4_K_M0.6 GB
GGUF
BG
bge-small-en-v1.5
BGE · CompendiumLabs
Params33M
Context512
Q4_K_M0.0 GB
Embedding
QW
Qwen3 Embedding 0.6b
Qwen · Qwen
Params596M
Context32K
Smallest0.6 GB
Embedding
LL
Llama 3.2 1B Instruct
Llama · bartowski
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
PA
Parakeet TDT 0.6B v2
Parakeet · handy-computer
Params618M
Context—
Q4_K_M0.5 GB
GGUF
AL
all-MiniLM-L6-v2
All · second-state
Params23M
Context512
Q4_K_M0.0 GB
Embedding
QW
Qwen3-0.6B
Qwen · unsloth
Params596M
Context40K
Q4_K_M0.4 GB
GGUF
AL
all-MiniLM-L6-v2
All · leliuga
Params23M
Context512
Q4_K_M0.0 GB
CodingEmbedding
EM
Embeddinggemma 2
Embeddinggemma · unsloth
Params271M
Context256K
Smallest0.2 GB
EmbeddingVision
GE
gemma 4 E4B it qat
Gemma · unsloth
Params93M
Context128K
Smallest0.1 GB
Vision
NE
Nemotron-3-Diarization.q8_0
Nemotron · nvidia
Params99M
Context—
Smallest0.1 GB
GGUF
QW
Qwen3.6-35B-A3B
Qwen · ggml-org
Params1.9B
Context256K
Q4_K_M20.4 GB
Vision
SN
snowflake arctic embed m v1.5
Snowflake · Snowflake
Params109M
Context512
Smallest0.1 GB
Embedding
BG
bge-large-en-v1.5
BGE · CompendiumLabs
Params334M
Context512
Q4_K_M0.2 GB
Embedding
QW
Qwen3 1.7B
Qwen · ggml-org
Params2.03B
Context40K
Q4_K_M1.3 GB
GGUF
QW
Qwen2.5 VL 3B Instruct
Qwen · ggml-org
Params3.09B
Context125K
Q4_K_M1.9 GB
Vision
VI
VieNeu TTS 0.3B q4
Vieneu · pnnbao-ump
Params294M
Context4K
Smallest0.2 GB
GGUF
BG
bge-base-en-v1.5
BGE · CompendiumLabs
Params109M
Context512
Q4_K_M0.1 GB
Embedding
QW
Qwen3 Reranker 0.6B
Qwen · ggml-org
Params596M
Context40K
Smallest0.6 GB
Embedding
TE
test model router download
Test · ggml-org
Params1M
Context2K
Smallest0.0 GB
GGUF
QW
Qwen2.5 0.5B Instruct
Qwen · bartowski
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
MI
MiniCPM V 4.7 1B
MiniCPM · openbmb
Params1.01B
Context256K
Q4_K_M0.7 GB
ReasoningTool useVision

Other sizes

Keep browsing