Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GP
Gpt2 Xl
Gpt · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF
GP
gpt2-medium-conversational
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
LL
Llama 3.2 1B Instruct Ja Gsm8K Layerswap
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ME
MentaLLaMA-chat-7B-hf
Mentallama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
AL
allknowingroger MultiverseMath 12B MoE
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.9
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama-3.2-3B-Instruct-cot-19-11-2
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
DE
KSU HW SEC Deepseek SFT RL Text DFG AST
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
QW
qwen3 embed 0.6b
Qwen · cstr
Params596M
Context32K
Smallest0.0 GB
GGUF
LL
llama-2-ko-7b-emb-dev
Llama · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.2 GB
GGUF
QW
Qwen3.6-27B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params26.9B
Context256K
Smallest10.0 GB
ReasoningTool useVision
ML
mlfoundations dev oh v1.3 evol instruct x2
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 8B Instruct 80K QLoRA Merged
Llama · Monor
Params8.03B
Context80K
Q4_K_M4.9 GB
GGUF
GP
Gpt2_Large_Tdata1_Fdata_30_Epochs
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
GP
Gpt Neox 20b Embeddings
Gpt · RichardErkhov
Params20.6B
Context2K
Q4_K_M13.1 GB
Embedding
JP
jpalgo EMP FT 3K V1
Jpalgo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
SO
SOLAR Platypus 10.7B v2
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
QW
Qwen 7B Chat Cantonese
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.9 GB
GGUF
CH
chinese alpaca 2 7b 16k
Chinese · shaowenchen
Params6.93B
Context16K
Smallest2.9 GB
GGUF
TE
TeleOCR
Teleocr · cstr
Params1.26B
Context—
Smallest1.2 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
MI
CodeActAgent Mistral 7b v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Phi 3.5 Mini Instruct
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
PH
Phi 2 Layla v1 Chatml
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
YE
yejingfu MN 12B Starcannon v2
Yejingfu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
YI
Faro Yi 34B
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
mia llm autotrain bloom 11b sft slm
Mia · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 8000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
JA
Jais Adapted 7b Chat
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
GGUF

Other quants

Keep browsing