Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
MinaMila llama 3 InstBase adult ImpFeat 1ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
NI
nimamegh roberta cnn legal
Nimamegh · RichardErkhov
Params354M
Context512
Q4_K_M0.2 GB
GGUF
GE
gemma-2b-orpo
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Internlm2 Math 20b Llama
Llama · RichardErkhov
Params19.9B
Context8K
Q4_K_M12.0 GB
GGUF
YI
CaPlatTessDolXaBoros Yi 34B 200K DARE Ties
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
mkebiclioglu mistralai Code Instruct Finetune sunday night test 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
ML
mlfoundations dev oh v1 w v3 opengpt
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RE
reflect_llm8B_llmMstlrg-om2-80k-SftT2_MgSpsdpT02_b0.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
IN
inflatebot MN 12B Mag Mell R1
Inflatebot · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Reasoning
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · khalifah66362666
Params1.86B
Context256K
Smallest0.9 GB
GGUF
XU
XueyingJia pythia 1b stage1 dpo SG merge
Xueyingjia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
RL
Rlhflow_Mixture_Iter2
Rlhflow · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
KaKee llama 2 7b full unknown various answer 7 stereo epoch1
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
TI
TinyLlama-1.1B-Chat-v0.2
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
hardikg2907 code llama html completion 1
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
BE
besimray miner id 3 72df7d62 e0d6 41b2 9153 9843320d6b82 1729802122
Besimray · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Qwen2 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
MI
K2S3 Mistral 7b v1.43
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
MI
WebraftAI synapsellm 7b mistral v0.4 preview3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh v1.3 alpaca x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LF
lfm2 colbert
Lfm · cstr
Params353M
Context—
Smallest0.0 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
XT
xTower13B
Xtower · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyLlama TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Nous-Hermes-Llama2-13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MELT-TinyLlama-1.1B-Chat-v1.0
Melt · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
OP
OpenELM 1_1B DPO Full Max Min Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF

Other quants

Keep browsing