Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
mlfoundations dev llama3 1 8b oh v3.1 wo gpteacher
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Finnish NLP llama 7b finnish instruct v0.2
Llama · RichardErkhov
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
RF
rfdetr cpp base
Rfdetr · mudler
Params31M
Context—
Smallest0.0 GB
GGUF
LL
Magpie Align Llama 3 8B Magpie Align v0.3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Smoothie Qwen3 AIO
Qwen · prithivMLmods
Params596M
Context40K
Q4_K_M0.4 GB
ReasoningTool use
QW
Qwen3.6 27B
Qwen · eaddario
Params26.9B
Context256K
Smallest0.0 GB
GGUF
LL
posicube Llama2 chat AYB 13B
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
SA
Salamandra 7b
Salamandra · RichardErkhov
Params7.77B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Leo Hessianai 7b P1 Llama Errors P2 GEC
Llama · RichardErkhov
Params6.74B
Context8K
Q4_K_M4.1 GB
GGUF
HJ
Hjgugugjhuhjggg mergekit ties ysreuuq
Hjgugugjhuhjggg · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
YO
Youri 7b
Youri · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
wenlianghuang Phi3 finance Matt
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
JA
Japanese Stablelm 3b 4e1t Instruct
Japanese · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
CY
z image catalyst
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
LL
Llama3 70B Chinese Chat
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
GP
p208p2002 gpt2 squad nqg hl
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
EE
EEVE Korean 10.8B Instruct v4 Sft
Eeve · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
TR
travel_microsoft_DialoGPT Small
Travel · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
phi 2
Phi · andrijdavid
Params2.78B
Context2K
Q4_K_M1.8 GB
Coding
GP
gpt2-large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
DE
Deepseek Coder 6.7b Chat
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
CodingTool use
OL
akrishnan TOFU olmo 7b ft full 25epochs lr1e 05 seed42 run1
OLMo · RichardErkhov
Params6.89B
Context4K
Q4_K_M4.2 GB
GGUF
LL
AndreyRzhaksinskiy CDS CodeLlama 7b 20241112 pretrain
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
JJhooww Mistral 7B v0.2 Base ptbr
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
VO
vox jot models
Vox · IrieDinamik
Params4.43B
Context—
Q4_K_M2.9 GB
GGUF
ZE
Zephyr 7b Beta_Bio Tutor_Dpo
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DA
DaringMaid-20B-V1.1
Daringmaid · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
mistral-7b-dolphin-sft
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing