Small GGUF models

91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3 4B
Qwen · bartowski
Params4B
Context32K
Q4_K_M2.5 GB
GGUF
SE
SeedVR2 comfyUI
Seedvr · AInVFX
Params7.8B
Context—
Q4_K_M2.0 GB
GGUF
GE
gemma-4-E4B-it
Gemma · ggml-org
Params7.5B
Context128K
Smallest0.1 GB
Vision
NE
Nemotron 3 Embed 8B BF16
Nemotron · Abiray
Params7.95B
Context256K
Q4_K_M4.9 GB
Embedding
QW
Qwen2.5 Omni 7B
Qwen · ggml-org
Params7.6B
Context32K
Q4_K_M4.7 GB
Vision
TA
UI-TARS-1.5-7B
Tars · Mungert
Params7.6B
Context125K
Q4_K_M4.8 GB
Vision
QW
Qwen3-8B
Qwen · unsloth
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
FL
flux 3 image klein
Flux · unsloth
Params5.55B
Context—
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5 4B
Qwen · bartowski
Params4.33B
Context256K
Q4_K_M3.0 GB
Vision
QW
Qwen Image 2.1
Qwen · leejet
Params7.12B
Context—
Smallest2.6 GB
GGUF
FL
Flux2 Klein 9B True V2
Flux · wikeeyang
Params9.08B
Context—
Smallest5.7 GB
GGUF
GR
Granite 4.2 3B BE
Granite · taronaeo
Params3.66B
Context—
Q4_K_M2.2 GB
ReasoningTool use
IM
Z Image Turbo
Image · leejet
Params6.15B
Context—
Smallest2.6 GB
GGUF
GE
Gemma 4 E2B It
Gemma · bartowski
Params4.65B
Context128K
Q4_K_M3.5 GB
Vision
PH
Phi 4 Mini Instruct
Phi · unsloth
Params3.84B
Context128K
Q4_K_M2.5 GB
Coding
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · pottokao
Params8.2B
Context256K
Q4_K_M5.0 GB
CodingVision
GE
Gemma-3-4B-It
Gemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
BO
Bonsai-8B
Bonsai · prism-ml
Params8.2B
Context64K
Smallest1.2 GB
GGUF
GE
gemma 7b
Gemma · google
Params8.54B
Context—
Smallest34.2 GB
GGUF
GE
Gemma 4 E2B
Gemma · lmstudio-community
Params4.65B
Context128K
Q4_K_M3.4 GB
Vision
QW
Qwen3.5 4B
Qwen · SoAIHQ
Params4.33B
Context256K
Q4_K_M2.8 GB
ReasoningTool useVision
QW
Qwen3.5-4B
Qwen · unsloth
Params4.33B
Context256K
Q4_K_M2.8 GB
Vision
LF
LFM2.5 8B A1B
Lfm · LiquidAI
Params8.47B
Context125K
Q4_K_M5.2 GB
GGUF
PH
Phi 4 Mini Instruct
Phi · bartowski
Params3.84B
Context128K
Q4_K_M2.5 GB
GGUF
LT
Sulphur 2 base
Ltx · SulphurAI
Params9.2B
Context256K
Smallest9.5 GB
Vision
FL
FLUX.2 klein 4B
Flux · leejet
Params3.88B
Context—
Smallest2.5 GB
GGUF
MI
MiniCPM o 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Q4_K_M5.0 GB
Vision
CP
CPH Community 7B
Cph · CypherTroopers
Params7B
Context—
Q4_K_M0.0 GB
GGUF
VI
VibeVoice-ASR-Streaming-7B
Vibevoice · audio-cpp
Params7B
Context—
Smallest5.9 GB
GGUF

Other sizes

Keep browsing