Q4_K_S GGUF models

4-bit k-quant, small. Slightly smaller and a little lower quality than Q4_K_M. 104639 models have a Q4_K_S file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
GGUF
HU
Huihui CyberStrike OffSec 35B Abliterated
Huihui · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
Tool useVision
SH
Shaylah Darija 4b Merged
Shaylah · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
ZO
Zoro i1
Zoro · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool use
FR
Froopert 31B
Froopert · bartowski
Params30.7B
Context256K
Q4_K_M19.6 GB
ReasoningRoleplayVision
MI
Mistral Large Instruct 2407
Mistral · bartowski
Params123B
Context128K
Q4_K_M73.2 GB
GGUF
NE
Nex N2.5 Pro
Nex · bartowski
Params396B
Context256K
Q4_K_M250.8 GB
Vision
TR
TranslatePsy AfriSLM 0.8B
Translatepsy · mradermacher
Params1.01B
Context256K
Q4_K_M0.7 GB
ReasoningTool use
DO
DoItYourself v1 2B
Doityourself · theprint
Params1.94B
Context256K
Q4_K_M1.3 GB
GGUF
HE
Hermes 2 DPO WestLake 7B v2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
QW
Twi Qwen v1 Merged
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool use
CY
CyberStrike OffSec 35B
Cyberstrike · ahmedandaloes
Params35.5B
Context256K
Q4_K_M21.7 GB
GGUF
QW
Qwen3 VL 32B Instruct Abliterated v1
Qwen · mradermacher
Params32.8B
Context256K
Q4_K_M19.8 GB
Vision
ME
MechVL 4B RL
Mechvl · mradermacher
Params4.41B
Context256K
Q4_K_M2.7 GB
ReasoningVision
GR
Granite 4.0 H Small
Granite · bartowski
Params32.2B
Context1024K
Q4_K_M19.7 GB
GGUF
LL
Nous Hermes Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
Tool use
SO
SOR ColdBrew 12B Think RP Base
Sor · mradermacher
Params12.2B
Context1000K
Q4_K_M7.5 GB
ReasoningRoleplay
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
GE
Gemma 4 26B A4B It Heretic Ara v2
Gemma · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
GGUF
SW
Swallow-7b-NVE-hf
Swallow · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
maywell_pivot-10.7b-mistral-v0.2-rp
Mistral · TheBloke
Params10.7B
Context32K
Q4_K_M6.5 GB
Roleplay
GE
Gemma-4-26B-A4B-It
Gemma · Kremon96
Params25.2B
Context256K
Q4_K_M16.9 GB
ReasoningTool useVision
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Hermes-2-Pro-Mistral-7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Qwen3.8 27B OBLITERATED Mythos Class Agentic
Qwen · mradermacher
Params27.3B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
GP
Gpt Oss 20b Unsloth Puzzle V2
gpt-oss · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
Reasoning
TR
TranslatePsy AfriSLM 4B
Translatepsy · mradermacher
Params4.84B
Context256K
Q4_K_M3.1 GB
ReasoningTool use
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ON
OneJev 0.8B
Onejev · bartowski
Params1.01B
Context256K
Q4_K_M0.7 GB
Vision
HO
K2 Horizon MoVA 36B A4B
Horizon · ngquocvinh
Params37.4B
Context512K
Q4_K_M22.4 GB
GGUF

Other quants

Keep browsing