GGUF models by ggml-org
224 GGUF models quantized by ggml-org on Hugging Face. Sizes run from 1M to 1050B parameters. The longest trained context is 10240K tokens.
MI
Mistral-Small-3.1-24B-Instruct-2503
Mistral · ggml-org
Params23.6B
Context128K
Smallest8.8 GB
QW
Qwen2.5 Coder 14B
Qwen · ggml-org
Params14.8B
Context32K
Smallest15.7 GB
CH
chatterbox-turbo
Chatterbox · ggml-org
Params427M
Context8K
Smallest0.5 GB
CH
chatterbox
Chatterbox · ggml-org
Params525M
Context128K
Smallest0.6 GB
NE
NVIDIA Nemotron 3 Nano 4B BF16
Nemotron · ggml-org
Params3.97B
Context1024K
Smallest4.2 GB
JI
jina reranker v1 turbo en
Jina · ggml-org
Params38M
Context8K
Smallest0.1 GB
DE
DeepSeek R1 Distill Qwen 32B
DeepSeek · ggml-org
Params32.8B
Context128K
Smallest34.8 GB
TI
tinyLFM3 Omni for testing
Tinylfm · ggml-org
Params35M
Context125K
Smallest0.0 GB
GE
Gemma 3 1b Pt
Gemma · ggml-org
Params1B
Context32K
Smallest2.0 GB
GE
gemma-1.1-2b-it
Gemma · ggml-org
Params2.51B
Context8K
Smallest2.7 GB
GE
gemma-1.1-7b-it
Gemma · ggml-org
Params8.54B
Context8K
Q4_K_M5.3 GB
MI
Devstral 2 123B Instruct 2512
Mistral · ggml-org
Params125B
Context256K
Smallest132.9 GB
OM
d1-Omni-2B
Omni · ggml-org
Params1.73B
Context125K
Smallest1.8 GB
QW
Qwen3 VL 30B A3B Instruct
Qwen · ggml-org
Params30.5B
Context256K
Smallest32.5 GB
QW
Qwen2.5 Coder 1.5B 32B speculative
Qwen · ggml-org
Params1.78B
Context32K
Q4_K_M1.1 GB
QW
Qwen3 235B A22B
Qwen · ggml-org
Params235B
Context40K
Q4_K_M142.2 GB
SE
sesame csm 1b
Sesame · ggml-org
Params96M
Context—
Smallest0.2 GB
QW
Qwen3 30B A3B Instruct 2507
Qwen · ggml-org
Params30.5B
Context256K
Smallest32.5 GB
QW
LoRA Qwen2.5 1.5B Instruct Abliterated
Qwen · ggml-org
Params94M
Context—
Smallest0.4 GB
QW
Qwen3 4B Thinking 2507
Qwen · ggml-org
Params4.02B
Context256K
Smallest4.3 GB
PH
LoRA Phi 4 Abliterated
Phi · ggml-org
Params119M
Context—
Smallest0.5 GB
LF
LFM2 Audio 1.5B
Lfm · ggml-org
Params1.17B
Context125K
Smallest1.2 GB
SM
e5 small v2 Q8 0
Small · ggml-org
Params33M
Context512
Smallest0.0 GB
BE
Bert Base Uncased
Bert · ggml-org
Params109M
Context512
Smallest0.1 GB
LO
LongCat Flash Chat
Longcat · ggml-org
Params563B
Context128K
Smallest0.8 GB
QW
Qwen2.5 Coder 1.5B
Qwen · ggml-org
Params1.54B
Context32K
Smallest0.7 GB
JI
jina embeddings v2 base en Q8 0
Jina · ggml-org
Params137M
Context8K
Smallest0.1 GB
GT
Gte Small
Gte · ggml-org
Params33M
Context512
Smallest0.0 GB
LL
LoRA Llama 3 70B Instruct Abliterated
Llama · ggml-org
Params424M
Context—
Smallest1.7 GB
QW
LoRA Qwen2.5 3B Instruct Abliterated
Qwen · ggml-org
Params140M
Context—
Smallest0.6 GB