Small GGUF models
91171 GGUF models with roughly 4B to 9B parameters. Sizes run from 3.51B to 9.5B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3 4B
Qwen · bartowski
Params4B
Context32K
Q4_K_M2.5 GB
SE
SeedVR2 comfyUI
Seedvr · AInVFX
Params7.8B
Context—
Q4_K_M2.0 GB
GE
gemma-4-E4B-it
Gemma · ggml-org
Params7.5B
Context128K
Smallest0.1 GB
NE
Nemotron 3 Embed 8B BF16
Nemotron · Abiray
Params7.95B
Context256K
Q4_K_M4.9 GB
QW
Qwen2.5 Omni 7B
Qwen · ggml-org
Params7.6B
Context32K
Q4_K_M4.7 GB
TA
UI-TARS-1.5-7B
Tars · Mungert
Params7.6B
Context125K
Q4_K_M4.8 GB
QW
Qwen3-8B
Qwen · unsloth
Params8.19B
Context40K
Q4_K_M5.0 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · bartowski
Params7.62B
Context128K
Q4_K_M4.7 GB
FL
flux 3 image klein
Flux · unsloth
Params5.55B
Context—
Q4_K_M3.7 GB
QW
Qwen3.5 4B
Qwen · bartowski
Params4.33B
Context256K
Q4_K_M3.0 GB
QW
Qwen Image 2.1
Qwen · leejet
Params7.12B
Context—
Smallest2.6 GB
FL
Flux2 Klein 9B True V2
Flux · wikeeyang
Params9.08B
Context—
Smallest5.7 GB
GR
Granite 4.2 3B BE
Granite · taronaeo
Params3.66B
Context—
Q4_K_M2.2 GB
IM
Z Image Turbo
Image · leejet
Params6.15B
Context—
Smallest2.6 GB
GE
Gemma 4 E2B It
Gemma · bartowski
Params4.65B
Context128K
Q4_K_M3.5 GB
PH
Phi 4 Mini Instruct
Phi · unsloth
Params3.84B
Context128K
Q4_K_M2.5 GB
QW
Qwen Image 2.1 Text Encoder Heretic
Qwen · pottokao
Params8.2B
Context256K
Q4_K_M5.0 GB
GE
Gemma-3-4B-It
Gemma · unsloth
Params3.88B
Context128K
Q4_K_M2.5 GB
BO
Bonsai-8B
Bonsai · prism-ml
Params8.2B
Context64K
Smallest1.2 GB
GE
gemma 7b
Gemma · google
Params8.54B
Context—
Smallest34.2 GB
GE
Gemma 4 E2B
Gemma · lmstudio-community
Params4.65B
Context128K
Q4_K_M3.4 GB
QW
Qwen3.5 4B
Qwen · SoAIHQ
Params4.33B
Context256K
Q4_K_M2.8 GB
QW
Qwen3.5-4B
Qwen · unsloth
Params4.33B
Context256K
Q4_K_M2.8 GB
LF
LFM2.5 8B A1B
Lfm · LiquidAI
Params8.47B
Context125K
Q4_K_M5.2 GB
PH
Phi 4 Mini Instruct
Phi · bartowski
Params3.84B
Context128K
Q4_K_M2.5 GB
LT
Sulphur 2 base
Ltx · SulphurAI
Params9.2B
Context256K
Smallest9.5 GB
FL
FLUX.2 klein 4B
Flux · leejet
Params3.88B
Context—
Smallest2.5 GB
MI
MiniCPM o 4 5
MiniCPM · openbmb
Params8.2B
Context40K
Q4_K_M5.0 GB
CP
CPH Community 7B
Cph · CypherTroopers
Params7B
Context—
Q4_K_M0.0 GB
VI
VibeVoice-ASR-Streaming-7B
Vibevoice · audio-cpp
Params7B
Context—
Smallest5.9 GB