GGUF models by ggml-org

224 GGUF models quantized by ggml-org on Hugging Face. Sizes run from 1M to 1050B parameters. The longest trained context is 10240K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
DeepSeek V4 Flash Vision Exp
DeepSeek · ggml-org
Params284B
Context—
Smallest10.8 GB
GGUF
QW
Qwen2.5 Coder 3B
Qwen · ggml-org
Params3.09B
Context32K
Smallest3.3 GB
Coding
QW
Qwen2.5 Coder 7B
Qwen · ggml-org
Params7.62B
Context32K
Smallest8.1 GB
Coding
MI
Voxtral-Mini-4B-Realtime-2602
Mistral · ggml-org
Params3.43B
Context128K
Q4_K_M2.6 GB
Vision
MI
MiniCPM V 4.6
MiniCPM · ggml-org
Params752M
Context256K
Q4_K_M0.5 GB
Vision
NO
Nomic Embed Text V2
Nomic · ggml-org
Params475M
Context2K
Smallest0.5 GB
Embedding
SM
SmolLM2-135M
SmolLM · ggml-org
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
NE
NVIDIA-Nemotron-3-Nano-Omni-30B-A3B
Nemotron · ggml-org
Params31.6B
Context1024K
Q4_K_M22.4 GB
ReasoningVision
GE
gemma 3n E2B it
Gemma · ggml-org
Params4.46B
Context32K
Smallest4.8 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · ggml-org
Params1.54B
Context32K
Smallest1.6 GB
Coding
GR
granite docling 258M
Granite · ggml-org
Params164M
Context8K
Smallest0.2 GB
Vision
IN
InternVL3 2B Instruct
Internvl · ggml-org
Params1.78B
Context32K
Q4_K_M1.1 GB
Vision
GE
gemma 3n E4B it
Gemma · ggml-org
Params6.9B
Context32K
Smallest7.4 GB
GGUF
MI
Ministral 3 3B Instruct 2512
Mistral · ggml-org
Params3.43B
Context256K
Smallest3.7 GB
Vision
AU
AutoGLM Phone 9B
Autoglm · ggml-org
Params9.4B
Context64K
Q4_K_M6.2 GB
Vision
LL
Meta Llama 3.1 8B Instruct
Llama · ggml-org
Params8.03B
Context128K
Smallest6.0 GB
GGUF
JI
jina embeddings v2 base code Q8 0
Jina · ggml-org
Params160M
Context8K
Smallest0.2 GB
CodingEmbedding
GE
gemma-4-12B
Gemma · ggml-org
Params11.9B
Context256K
Smallest12.7 GB
GGUF
WA
WavTokenizer Large Speech 75token
Wavtokenizer · ggml-org
Params65M
Context8K
Smallest0.1 GB
GGUF
QI
Qianfan OCR
Qianfan · ggml-org
Params4.42B
Context32K
Smallest4.7 GB
ReasoningTool useVision
DE
DeepSeek V4 Flash
DeepSeek · ggml-org
Params284B
Context—
Smallest4.7 GB
GGUF
QW
Qwen2 VL 7B Instruct
Qwen · ggml-org
Params7.62B
Context32K
Q4_K_M4.7 GB
Vision
GE
Gemma 3 270m
Gemma · ggml-org
Params268M
Context32K
Smallest0.3 GB
GGUF
IN
InternVL3 14B Instruct
Internvl · ggml-org
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool useVision
GE
gemma-4-E2B
Gemma · ggml-org
Params4.65B
Context128K
Smallest5.0 GB
GGUF
MI
Mistral Large 4 1T A52B NVFP4
Mistral · ggml-org
Params1050B
Context1024K
Smallest4.2 GB
ReasoningTool useVision
FU
Functiongemma 270m It
Functiongemma · ggml-org
Params268M
Context32K
Smallest0.3 GB
Tool use
QW
Qwen3 30B A3B
Qwen · ggml-org
Params30.5B
Context40K
Q4_K_M18.6 GB
GGUF
QW
Qwen2.5 Coder 3B Instruct
Qwen · ggml-org
Params3.09B
Context32K
Smallest3.3 GB
CodingTool use
QW
Qwen2.5 Coder 7B Instruct
Qwen · ggml-org
Params7.62B
Context32K
Smallest8.1 GB
CodingTool use

Keep browsing