Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
SH
shubham11goel gita text generation gpt2
Shubham · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
mobinln Llama 3.2 1B Instruct fa
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GD
gdshaji gd tp 5k v1
Gdshaji · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
simonycl llama 3.1 8b instruct armorm judge iter2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MO
Moderate_Everything
Moderate · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ET
EtherealRainbow v0.3 8B
Etherealrainbow · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
saransh03sharma mintrec llama 2 7b Folder 5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
mlfoundations dev hp ablations mistral epoch5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
rm_1014_70K_step_600_lr_1e6
Step · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
bunnycore Phi 3.5 mini TitanFusion 0.6
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
IN
inflatebot L3 8B Helium3 baseStheno
Inflatebot · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
Phi 3 medium 128k instruct
Phi · ZeroWw
Params14B
Context128K
Smallest10.5 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
QW
terry69 qwen 0.5 feedback dirty
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
PR
Preethi 1995 NEW MERGED MODEL
Preethi · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.37B
Context128K
Q4_K_M0.9 GB
PO
PowerLM 3b
Powerlm · RichardErkhov
Params3.51B
Context4K
Q4_K_M2.1 GB
YI
NobodyExistsOnTheInternet Yi 34B GiftedConvo merged
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral scheduler constant
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
rahuldshetty tiny starcoder instruct
StarCoder · RichardErkhov
Params164M
Context8K
Q4_K_M0.1 GB
MI
Nondzu Mistral 7B codealpaca lora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
DopeorNope Mistralopithecus v0.1 10.8B
Mistral · RichardErkhov
Params10.8B
Context32K
Q4_K_M6.5 GB
RE
Rendoo_Luman0002_05_45
Rendoo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
archangel_sft_llama7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
ME
mergekit community config smart ablit
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2-7B-Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB