IQ2_S GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 26079 models have a IQ2_S file. Sizes run from 2M to 1030B parameters. The longest trained context is 10240K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
miqu evil dpo iMat
Miqu · InferenceIllusionist
Params69B
Context32K
Smallest14.5 GB
GGUF
LL
Dobby Mini Leashed Llama 3.1 8B i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
YI
SG Raccoon Yi 55B 200k i1
Yi · mradermacher
Params55.6B
Context195K
Q4_K_M33.3 GB
GGUF
LL
Llama 3.1 8B EZO 1.1 it
Llama · mmnga
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3.1 8B Instruct CODE Python English Asistant 16bit v2 i1
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Coding
LL
Llama 7b Hf
Llama · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B v0.3 Middo Alpaca 4o Mini
Mistral · mradermacher
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
CodeLlama 7B KStack Clean
Llama · mradermacher
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
AN
Anansi 35B A3B
Anansi · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningRoleplay
LL
Prox-Llama-3-8B-abliterated-orpo
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
Coding
TH
AM Thinking v1
Thinking · bartowski
Params32.8B
Context128K
Q4_K_M19.9 GB
Reasoning
MI
MiroThinker 1.7 Mini Heretic
Mirothinker · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · mmnga
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 mini 4k instruct iMat
Phi · qwp4w3hyb
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
SE
SeaLLMs v3 7B
Seallms · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
WI
WizardLM-2-7B-abliterated
Wizardlm · bartowski
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
SvS LLama 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GR
Granite Speech 5.0 2b
Granite · mradermacher
Params1.84B
Context16K
Q4_K_M1.1 GB
GGUF
LE
LemonadeRP-4.5.3
Lemonaderp · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Roleplay
LL
Bioinspired Base Hermes 3 Llama 3.1 8B
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
CodingTool useVision
NO
Nous Hermes 2 Mixtruct v0.1 8x7B DPO DARE TIES iMat
Nous · InferenceIllusionist
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
LL
Linkbricks-Horizon-AI-Ko-llama3-Instruct-dpo-8B-base
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
WE
WeirdCompound v1.7 24b
Weirdcompound · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
AS
ASearcher Local 14B
Asearcher · mradermacher
Params14.8B
Context128K
Q4_K_M9.0 GB
Tool use
GP
OpenAi GPT Oss 36B BrainStorm20x Uncensored
gpt-oss · mradermacher
Params36.6B
Context128K
Q4_K_M27.5 GB
CodingReasoning
CA
CataLlama-v0.2-Instruct-SFT
Catallama · mradermacher
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama3.1 Heretic Unsensored
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
CodingTool use
AS
Assistant_Pepe_32B
Assistant · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
CY
Cyberdonia 24B v11
Cyberdonia · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF

Other quants

Keep browsing