Q4_K_S GGUF models

4-bit k-quant, small. Slightly smaller and a little lower quality than Q4_K_M. 104639 models have a Q4_K_S file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
YI
EVA Yi 1.5 9B 32K V1
Yi · bartowski
Params8.83B
Context32K
Q4_K_M5.3 GB
GGUF
EU
Euryale 1.3 longLORA 70b rope8 32k fp16 i1
Euryale · mradermacher
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
IN
Internlm2 Math Plus 7b
InternLM · mradermacher
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
AR
Artemis 31B v1.1 Heretic v2
Artemis · mradermacher
Params30.7B
Context256K
Q4_K_M18.7 GB
Coding
ER
ERNIE 4.5 21B A3B PT
Ernie · bartowski
Params21.8B
Context128K
Q4_K_M13.5 GB
GGUF
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking
Qwen · FrosT-LyRo
Params39.1B
Context256K
Q4_K_M23.9 GB
CodingReasoningRoleplay
QW
Qwen3-Vl-2B-Thinking-1M
Qwen · unsloth
Params1.72B
Context977K
Q4_K_M1.1 GB
ReasoningVision
CA
CarbonVillain En 10.7B v4
Carbonvillain · mradermacher
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Collaiborator-MEDLLM-Llama-3-8B-v2
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
TH
Thaki ASR KO Dialects v1 i1
Thaki · mradermacher
Params1.72B
Context64K
Q4_K_M1.1 GB
GGUF
NO
nomic ai nomic embed code
Nomic · bartowski
Params7.1B
Context32K
Q4_K_M4.4 GB
CodingEmbedding
AM
AmberChat
Amberchat · mradermacher
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
EK
Ekbasis Skopos 27B
Ekbasis · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
GGUF
AI
aiXcompact 4B
Aixcompact · mradermacher
Params4.21B
Context256K
Q4_K_M2.7 GB
CodingReasoningTool use
IN
Internvl3-78B-Instruct
Internvl · unsloth
Params72.7B
Context32K
Q4_K_M47.4 GB
CodingVision
LL
Llama 2 70B OASST 1 200
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
IN
Internlm2 Math Plus 1_8b
InternLM · mradermacher
Params1.89B
Context8K
Q4_K_M1.2 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · second-state
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
LU
luxia 21.4b alignment v1.0
Luxia · MaziyarPanahi
Params21.4B
Context32K
Q4_K_M12.9 GB
GGUF
GE
KaLM Embedding Gemma3 12B 2511
Gemma · mradermacher
Params11.8B
Context128K
Q4_K_M7.3 GB
Embedding
RW
rwitz_go-bruins
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
Medgemma 27b Text It Abliterated
Medgemma · mradermacher
Params27B
Context128K
Q4_K_M16.5 GB
GGUF
KA
karakuri-lm-8x7b-instruct-v0.1
Karakuri · mmnga
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
NE
models--automerger--NeuralsirkrishnaInex12-7B
Neuralsirkrishna · MaziyarPanahi
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AS
AstaBrief_8B
Astabrief · mradermacher
Params8.2B
Context40K
Q4_K_M5.0 GB
GGUF
QW
Huihui Qwen3 30B A3B Thinking 2507 Abliterated
Qwen · mradermacher
Params30.5B
Context256K
Q4_K_M18.6 GB
Reasoning
WA
WaifuGemma4 26b A4B v1
Waifugemma · mradermacher
Params25.2B
Context256K
Q4_K_M16.8 GB
RoleplayVision
QW
Qwen3.5 9b Uncensored
Qwen · mradermacher
Params8.95B
Context256K
Q4_K_M5.6 GB
Tool useVision
MI
Magistral Small 2509
Mistral · bartowski
Params23.6B
Context128K
Q4_K_M14.3 GB
Reasoning

Other quants

Keep browsing