Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
Llama2 13b Lima Sft Dpo
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
OpenCodeReasoning Nemotron 1.1 32B
Qwen · quantized4all
Params32.8B
Context64K
Q4_K_M19.9 GB
VI
Vistral 7B Chat
Vistral · RichardErkhov
Params7.29B
Context32K
Q4_K_M4.4 GB
QW
Gukbap Qwen2 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
Mistral7B PairRM SPPO Iter1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RA
RaiBP gpt2 openwebtext2 first 30 chunks ablation bilingual
Raibp · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
JP
jpacifico French Alpaca 7B Instruct beta
Jpacifico · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HE
0 hero Matter 0.1 Slim 7B preview
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3 Lumimaid 8b v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5-0.5B-SFT-2e-5-2ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
XL
xLAM 7b Fc R
Xlam · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
Karen_TheEditor_V2_STRICT_Mistral_7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
jacson llama 2 7b codewello
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
Llama 2 Koen 13b
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
LL
llama 2 13b langchain chat
Llama · shaowenchen
Params13B
Context2K
Smallest5.4 GB
LL
Llama3 TAIDE LX 8B Chat Alpha1
Llama · nctu6
Params8.03B
Context8K
Q4_K_M4.9 GB
RO
ronigold dictalm2.0 instruct fine tuned alpaca gpt4 hebrew
Ronigold · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
DE
Srijith rkr deepseek base 1e 3 NO cot only failed samples 100 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
YI
Yi 34b 200k Aezakmi Raw Toxic 2702
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
DU
duliadotio dulia 13b 8k alpha
Duliadotio · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
MI
klcsp mistral 7b 0.3 gpt4o 100k classification fft
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
AT
Atlas Chat 9B
Atlas · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GE
Gemma 2b Ru
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
DE
deepseek-coder-1.3b-python-peft
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
IG
Ignis 7B DPO
Ignis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB