General GGUF models

134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Models Meta Llama Llama 3.3 70B Instruct
Llama · MaziyarPanahi
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
MI
Mistral-7B-Instruct-v0.3
Mistral · thesven
Params7B
Context32K
Q4_K_M4.4 GB
GGUF
MI
models--mistralai--mathstral-7B-v0.1
Mistral · MaziyarPanahi
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
TP
tpn004 cand v3f aim
Tpn · tatjr13
Params23.6B
Context128K
Smallest14.7 GB
GGUF
TP
tpn004 cand v4f l1im
Tpn · tatjr13
Params23.6B
Context128K
Smallest18.3 GB
GGUF
MI
Rocinante X 12B v1
Mistral · TheDrummer
Params12.2B
Context128K
Q4_K_M7.5 GB
GGUF
TP
tpn004 cand v3f l1im
Tpn · tatjr13
Params23.6B
Context128K
Smallest18.3 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · hugging-quants
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Models Meta Llama Meta Llama 3.1 70B Instruct
Llama · MaziyarPanahi
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
TP
tpn004 cand v4f aim
Tpn · tatjr13
Params23.6B
Context128K
Smallest14.7 GB
GGUF
TP
tpn004 cand v1f aim
Tpn · tatjr13
Params23.6B
Context128K
Smallest14.7 GB
GGUF
GL
GLM 5.3 Flash GGUF DGX Spark
GLM · autotrust
Params279B
Context1024K
Smallest84.9 GB
GGUF
QW
Qwen Image 2.1 PE T2I Heretic
Qwen · pottokao
Params8.95B
Context256K
Q4_K_M5.9 GB
GGUF
QW
models--Qwen--Qwen2-7B-Instruct
Qwen · MaziyarPanahi
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
Models Mistralai Mistral Small Instruct 2409
Mistral · MaziyarPanahi
Params22.2B
Context32K
Q4_K_M13.3 GB
GGUF
LL
Models Meta Llama Meta Llama 3.1 8B Instruct
Llama · MaziyarPanahi
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Models Mistralai Mistral Small 24B Instruct 2501
Mistral · MaziyarPanahi
Params23.6B
Context32K
Q4_K_M14.3 GB
GGUF
SA
Sakura GalTransl 7B v3.7
Sakura · SakuraLLM
Params7.72B
Context128K
Smallest4.3 GB
GGUF
TE
test model stories260K
Test · ggml-org
Params—
Context—
Smallest0.0 GB
GGUF
GE
Models Google Gemma 3 27b It
Gemma · MaziyarPanahi
Params27B
Context128K
Q4_K_M16.5 GB
GGUF
MI
Models Mistralai Mistral Large Instruct 2411
Mistral · MaziyarPanahi
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
DE
DeepSeek V3 0324
DeepSeek · MaziyarPanahi
Params671B
Context160K
Smallest133.6 GB
GGUF
IN
Models PrimeIntellect INTELLECT 2
Intellect · MaziyarPanahi
Params32.8B
Context40K
Q4_K_M19.9 GB
GGUF
LL
Models Meta Llama Meta Llama 3.1 405B Instruct
Llama · MaziyarPanahi
Params410B
Context128K
Smallest151.2 GB
GGUF
PH
phi 4
Phi · MaziyarPanahi
Params14.7B
Context16K
Q4_K_M9.1 GB
GGUF
QW
Huihui Qwen3.6 35B A3B Abliterated
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
GGUF
LL
models--fireworks-ai--firefunction-v2
Llama · MaziyarPanahi
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
SM
SmolLM3 3B
SmolLM · bartowski
Params3.08B
Context64K
Q4_K_M1.9 GB
GGUF
LL
Llama 3.3 70B Instruct Abliterated
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
NE
NVIDIA Nemotron 3.5 Lightning 30B A3B
Nemotron · bartowski
Params32.9B
Context1024K
Q4_K_M25.5 GB
GGUF

Other uses

Keep browsing