Q4_K_S GGUF models
4-bit k-quant, small. Slightly smaller and a little lower quality than Q4_K_M. 104639 models have a Q4_K_S file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.8 27B OBLITERATED Mythos Class Agentic
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
GE
Gemma 4 Writers 31B
Gemma · mradermacher
Params30.7B
Context256K
Q4_K_M18.7 GB
FI
FireRed Image Edit 1.1
Firered · vantagewithai
Params20.4B
Context—
Q4_K_M13.1 GB
AI
Airoboros L2 7B 2.1
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
CY
CyberSecQwen 4B
Cybersecqwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
LL
codellama_codellama-7b-hf
Llama · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
PH
gretelai Phi 3 mini 128k instruct
Phi · RichardErkhov
Params1.9B
Context—
Q4_K_M0.0 GB
AR
AREX 2
Arex · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
QW
Huihui Qwen3 VL 30B A3B Instruct Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
SC
SCAIL Preview
Scail · vantagewithai
Params16.4B
Context—
Q4_K_M11.3 GB
GP
Gpt Oss 120b Abliterated
gpt-oss · mradermacher
Params117B
Context128K
Q4_K_M87.9 GB
QW
Qwen3.5 9B Claude 4.6 OS Auto Variable HERETIC UNCENSORED THINKING MAX NEOCODE Imatrix
Qwen · DavidAU
Params8.95B
Context256K
Q4_K_M6.8 GB
DE
DeepSeek V3.1
DeepSeek · bartowski
Params670B
Context160K
Q4_K_M409.2 GB
MI
cognitivecomputations_dolphin-2.6-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LF
lfm2.5 2.6b fable5 coding agent heretic
Lfm · saidutta69
Params2.7B
Context128K
Q4_K_M1.7 GB
HU
Huihui Ornith 1.5 35B A3B Abliterated
Huihui · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
FL
flux2 dev
Flux · gguf-org
Params17.8B
Context—
Q4_K_M20.0 GB
GE
Gemma 3 4b It Abliterated
Gemma · bartowski
Params3.88B
Context128K
Q4_K_M2.5 GB
OR
Orion 26B A4B v1
Orion · bartowski
Params25.2B
Context256K
Q4_K_M17.0 GB
FU
Functiongemma-270M-It
Functiongemma · unsloth
Params268M
Context32K
Q4_K_M0.3 GB
LL
Llama 2 70B Chat
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
QW
Qwen Image 2.1
Qwen · chfm
Params7.12B
Context—
Q4_K_M4.2 GB
MI
Mistral MOE 4X7B Dark MultiVerse Uncensored Enhanced32 24B
Mistral · DavidAU
Params24.2B
Context32K
Q4_K_M14.7 GB
ON
OneJev 4B
Onejev · bartowski
Params4.84B
Context256K
Q4_K_M3.2 GB
GE
TangYuGe Gemma 4 12b It Qat
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
LL
calme 2.4 llama3 70b
Llama · MaziyarPanahi
Params70.6B
Context8K
Q4_K_M42.5 GB
LI
LightOnOCR 3 4B
Lightonocr · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
AT
Athena v1
Athena · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
HE
Hermes-4-70B
Hermes · unsloth
Params70.6B
Context128K
Q4_K_M42.5 GB
QW
Qwythos 9B Claude Mythos 5 1M Uncensored Heretic
Qwythos · llmfan46
Params8.95B
Context1024K
Q4_K_M6.2 GB