General GGUF models

134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mixtral 8x7B Instruct v0.1
Mistral · MaziyarPanahi
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
MI
Mistral Nemo Instruct 2407 Absolute Heresy
Mistral · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
GGUF
NO
Nova-1.1-0.8B
Nova · mradermacher
Params1.01B
Context256K
Q4_K_M0.7 GB
Tool use
FI
Fimbulvetr 40 i1
Fimbulvetr · mradermacher
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
SO
SOLAR Platypus 10.7B v2
Solar · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 9B Abliterated
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
GGUF
MI
models--automerger--T3qInex12-7B
Mistral · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AU
Augmental ReMM 13B
Augmental · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
CH
Chaotic Order 24B V3
Chaotic · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
GP
gpt2
Gpt · QuantFactory
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
CH
chujiezheng Starling LM 7B beta ExPO
Chujiezheng · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 2507 4B Instruct Haiku 4.5 Merged
Qwen · bartowski
Params4B
Context256K
Q4_K_M2.6 GB
GGUF
QW
Qwen2 1.5B Instruct
Qwen · Goekdeniz-Guelmez
Params1.78B
Context32K
Q4_K_M1.1 GB
GGUF
SE
Seed OSS 36B Instruct
Seed · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
GGUF
QW
Qwen3 0.6B Abliterated
Qwen · bartowski
Params596M
Context40K
Q4_K_M0.4 GB
GGUF
LF
LFM2.5 24B A2B
Lfm · WhiskyAKM
Params24B
Context128K
Smallest13.6 GB
GGUF
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · crusoeai
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
QW
Qwen3 30B A3B ArliAI RpR v4 Fast
Qwen · mradermacher
Params30.5B
Context40K
Q4_K_M18.6 GB
GGUF
SO
Solar Open 100B
Solar · mradermacher
Params103B
Context128K
Q4_K_M62.3 GB
GGUF
MA
openaccess-ai-collective_manticore-13b
Manticore · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
RH
rho-math-1b-interpreter-v0.1
Rho · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
ZI
idea-ccnl_ziya-coding-34b-v1.0
Ziya · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
GGUF
QW
Qwen3.6 35B A3B DFlash2
Qwen · aminya
Params526M
Context256K
Q4_K_M0.3 GB
GGUF
MI
MiniCPM3 4B
MiniCPM · openbmb
Params4.07B
Context32K
Q4_K_M2.5 GB
GGUF
WI
Wizard Vicuna 30B Uncensored
Wizard · mradermacher
Params32.5B
Context2K
Q4_K_M19.6 GB
GGUF
QW
Qwen2 Audio 7B
Qwen · NexaAI
Params7.75B
Context8K
Q4_K_M4.8 GB
GGUF
MI
UCLA AGI Mistral7B PairRM SPPO Iter3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AI
bhenrym14_airoboros-l2-13b-2.1-yarn-64k
Airoboros · TheBloke
Params13B
Context64K
Q4_K_M7.9 GB
GGUF

Other uses

Keep browsing