General GGUF models

134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
OR
Orca 2 13b
Orca · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
LLaDA 1.5
Llada · mradermacher
Params8.02B
Context4K
Q4_K_M4.9 GB
GGUF
GE
gemma 3n E2B it
Gemma · ggml-org
Params4.46B
Context32K
Smallest4.8 GB
GGUF
AI
airoboros l2 7B 2.2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
GR
Granite 3.2 8b Instruct
Granite · lmstudio-community
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
DR
Draco 8x7B
Draco · mradermacher
Params46.7B
Context8K
Q4_K_M28.6 GB
Tool use
QW
BiggerCoQ Qwen3 10b
Qwen · mradermacher
Params10.9B
Context40K
Q4_K_M6.6 GB
GGUF
SU
SuperNova Medius
Supernova · bartowski
Params14.8B
Context128K
Q4_K_M9.0 GB
GGUF
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GE
gemma 4 26B A4B it MTP
Gemma · ironbcc
Params420M
Context256K
Q4_K_M0.3 GB
GGUF
NE
neutts air
Neutts · neuphonic
Params750M
Context—
Smallest1.5 GB
GGUF
GE
Fallen Gemma3 12B v1
Gemma · bartowski
Params11.8B
Context128K
Q4_K_M7.3 GB
GGUF
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Tool use
ZE
ZeusLabs Chronos Divergence 33B
Zeuslabs · RichardErkhov
Params32.5B
Context16K
Q4_K_M19.6 GB
GGUF
GR
Granite Switch 4.1 8b
Granite · ibm-granite
Params8.97B
Context128K
Q4_K_M6.4 GB
Tool use
MX
MXLewd L2 20B
Mxlewd · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
PI
maywell_pivot-0.1-early
Pivot · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
StartLux Decision v1 2B Q8 0
Startlux · startlux-models
Params1.88B
Context256K
Smallest2.0 GB
GGUF
UP
upstage SOLAR 10.7B Instruct v1.0
Upstage · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Infinity Instruct 7M Gen Llama3_1 8B
Llama · bartowski
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
Gemma 4 12B It Qat Q4_0 Unquantized Heretic v1.1
Gemma · igorls
Params11.9B
Context256K
Smallest7.0 GB
GGUF
GL
zai org GLM 4.6
GLM · bartowski
Params357B
Context198K
Q4_K_M217.7 GB
GGUF
LL
Llama3-ChatQA-1.5-70B
Llama · mradermacher
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
Llama 3 13B Instruct v0.1
Llama · MaziyarPanahi
Params13.3B
Context8K
Q4_K_M8.1 GB
GGUF
LL
Llama 3.3 70B Instruct ablated i1
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
LL
Openthai Llama Pretrained 7B
Llama · RichardErkhov
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
GE
Gemma4 Gutenberg Literotica 12B
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
GGUF
GE
Gemma 2 27b It
Gemma · ThomasBaruzier
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF

Other uses

Keep browsing