Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
ValiantLabs CodeLlama 13B Fireplace
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
GK
GKD
Gkd · Asher-1
Params151M
Context—
Smallest0.3 GB
GGUF
LL
Llama3 S Instruct v0.3 Checkpoint 7000 Phase 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0 128k
Solar · RichardErkhov
Params10.7B
Context128K
Q4_K_M6.5 GB
GGUF
KI
Chat Error Kimiko 10.7B v3
Kimi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
CodeLlama 13B Python Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ST
Starling LM 7B beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MI
IntelLabs sqft sparsepeft mistral 7b v0.3 60 gsm8k heu
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · botp
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool useVision
LL
smol_llama-4x220M-MoE
Llama · RichardErkhov
Params595M
Context2K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.2 1B Instruct Ja Version2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev oh v1.3 unnatural instructions x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QV
QVikhr 3 1.7B Instruction Noreasoning
Qvikhr · Vikhrmodels
Params1.72B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
NE
NESPED GEN TinyLlama schemaLinking min
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
aifeifei798 llama3 8B DarkIdol 2.2 Uncensored 1048K
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct STOCK_SOLAR Passthrough
Solar · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
GGUF
MI
jisukim8873 mistral 7B alpaca case 3 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ggml bakllava 1
LLaVA · mys
Params7.24B
Context32K
Smallest4.4 GB
Vision
BL
Bloom 1b7
Bloom · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DE
Final DEC Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
CH
chinese alpaca 2 7b 16k
Chinese · hfl
Params6.93B
Context16K
Smallest2.6 GB
GGUF
ML
mlfoundations dev oh v1 2 only unnatural instructions
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
VO
VoxCPM2 Q4 K M
Voxcpm · coolthor
Params1.62B
Context32K
Smallest1.0 GB
GGUF
MI
Codestral-22B-v0.1-hf
Mistral · thesven
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
MI
Hebrew Mistral 7B 200K
Mistral · RichardErkhov
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
IN
Instant RP Noodles 12B v1.4
Instant · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
RoleplayTool use
TH
Themis
Themis · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing