Q4_K_S GGUF models

4-bit k-quant, small. Slightly smaller and a little lower quality than Q4_K_M. 104639 models have a Q4_K_S file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
HO
K2 Horizon 3.7B Heretic
Horizon · mradermacher
Params5.06B
Context512K
Q4_K_M3.2 GB
GGUF
LL
code Llama 2 13B i1
Llama · mradermacher
Params13B
Context16K
Q4_K_M7.9 GB
Coding
PA
Pantheon Proto RP 1.8 30B A3B
Pantheon · bartowski
Params30.5B
Context32K
Q4_K_M18.6 GB
Roleplay
QW
Qwen3.8 9B Distill Uncen Heretic
Qwen · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
ReasoningTool useVision
LL
Llama 3_1 Nemotron 51B Instruct
Llama · bartowski
Params51.5B
Context128K
Q4_K_M31.0 GB
GGUF
MI
Solstice Mixtral v1 i1
Mistral · mradermacher
Params46.7B
Context32K
Q4_K_M28.6 GB
GGUF
GE
Gemma 4 26B A4B It SOMPOA Heresy
Gemma · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
MI
OpenHermes-2.5-Mistral-7B-x2
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
MiniCPM V 2 6
MiniCPM · bartowski
Params7.6B
Context32K
Q4_K_M4.7 GB
CodingVision
QW
Qwen3-14B-128K
Qwen · unsloth
Params14.8B
Context128K
Q4_K_M9.0 GB
GGUF
QW
Qwythos 9B v2
Qwythos · mradermacher
Params9.2B
Context1024K
Q4_K_M5.8 GB
Reasoning
QW
Dongfangshuo Qwen3.8 27B NSFW
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
RoleplayVision
SK
Skyfall 31B v4.2 Heretic
Skyfall · mradermacher
Params31.4B
Context128K
Q4_K_M19.0 GB
Coding
MI
severian_anima-phi-neptune-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite Guardian 4.1 8b
Granite · ibm-granite
Params8.38B
Context128K
Q4_K_M5.1 GB
ReasoningTool use
QW
PopiT Qwen3 4B Medical SFT 1128
Qwen · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
NE
NeoHorse 1 9 Uncensored
Neohorse · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
DA
Dark Scarlett 27B v3.0
Dark · ReadyArt
Params27.3B
Context256K
Q4_K_M23.5 GB
Roleplay
EK
Ekbasis 27B
Ekbasis · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
UN
ajibawa-2023_uncensored-jordan-7b
Uncensored · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
ST
stable diffusion 3.5 medium
Stable · city96
Params2.47B
Context—
Q4_K_M1.8 GB
GGUF
KA
KansenSakura Eclipse RP 12b
Kansensakura · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
Roleplay
OP
openbuddy-yi1.5-34b-v21.3-32k
Openbuddy · mradermacher
Params34.4B
Context128K
Q4_K_M20.7 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · mradermacher
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
GE
Gemma 4 12B It Heretic_Decensored
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
Vision
GL
Huihui GLM 4.6V Flash abliterated
GLM · AliBilge
Params9.4B
Context128K
Q4_K_M6.2 GB
Vision
GE
gemma 4 E2B it UD japanese imatrix
Gemma · dahara1
Params4.65B
Context128K
Q4_K_M3.1 GB
Vision
MI
MiniCPM V 4.7 1B Heretic
MiniCPM · mradermacher
Params1.01B
Context256K
Q4_K_M0.7 GB
RoleplayVision
QW
Qwen3-Vl-8B-Thinking-1M
Qwen · unsloth
Params8.19B
Context977K
Q4_K_M5.0 GB
ReasoningVision
GR
Granite 8b Code Instruct 4k
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding

Other quants

Keep browsing