Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · IMUGLYHUH
Params1.86B
Context256K
Smallest0.9 GB
GGUF
PA
parakeet ctc 1.1b
Parakeet · cstr
Params1.06B
Context—
Smallest0.7 GB
GGUF
PY
Pygmalion 3 12b
Pygmalion · PygmalionAI
Params12.2B
Context1000K
Smallest4.9 GB
GGUF
LL
DarkIdol Llama 3.1 8B Instruct 1.2 Uncensored
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
SA
saishf SOVLish Maid L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PR
PranavHarshan NarraLarge
Pranavharshan · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
BG
bge reranker v2 m3
BGE · cstr
Params568M
Context—
Smallest0.5 GB
GGUF
CO
Yi Coder 9B Chat Instruct TIES
Coder · RichardErkhov
Params28.3B
Context128K
Q4_K_M17.1 GB
Coding
LL
namespace Pt Llama 3 8B Instruct 80K QLoRA Merged
Llama · RichardErkhov
Params8.03B
Context80K
Q4_K_M4.9 GB
GGUF
YI
Yi-1.5-9B-Chat
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
WA
wav2vec2 large xlsr 53 italian
Wav · cstr
Params315M
Context—
Smallest0.2 GB
GGUF
WA
wav2vec2 xls r 300m cs 250
Wav · cstr
Params315M
Context—
Smallest0.2 GB
GGUF
ST
Starling-LM-11B-alpha
Starling · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
QW
Qwen3.8 Flash Next GSQ RCO Coder 192E
Qwen · oisee
Params101B
Context256K
Smallest3.0 GB
CodingReasoningTool use
LL
Configurable-Llama-3-8B-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
ReAligned Qwen3.5 0.8B
Qwen · Lazarus-Ai
Params752M
Context256K
Smallest0.4 GB
ReasoningTool use
AN
andykcheng unitrend model 8b vllm
Andykcheng · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
Gemma-4-26B-A4B-It
Gemma · Ninnix96
Params25.2B
Context256K
Smallest11.5 GB
ReasoningTool use
ST
stt nl fastconformer hybrid ctc large
Stt · cstr
Params109M
Context—
Smallest0.1 GB
GGUF
DE
deepseek-coder-1.3b-typescript
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
PE
PersianLLaMA 13B Instruct
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
GGUF
CO
CodeQwen1.5 7B Trojan Clas2024 Development
Codeqwen · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
QW
Josiefied Qwen2.5 1.5B Instruct Abliterated v3
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
YA
yanolja EEVE Korean Instruct 10.8B v1.0
Yanolja · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
DE
DeepSeek V2 Lite Chat
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
GGUF
MI
K2S3 Mistral 7b v1.2
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Llama3 KALE LM Chem 1.5 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
RN
Rnj 1 Instruct Heretic
Rnj · ZuzeTt
Params8.31B
Context32K
Q4_K_M5.1 GB
Tool use
QW
Qwen3.8 Flash Next GSQ RCO Coder 192E implant16
Qwen · oisee
Params101B
Context256K
Smallest3.0 GB
CodingReasoningTool use
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF

Other quants

Keep browsing