Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
LL
codellama CodeLlama 7b Python hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Phi 3 Mini 128k Instruct LLaMAfied
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
abhishek2475 Llama 3.2 3B Instruct finetuned
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
ME
mergekit community Mathing 1.5B
Mergekit · RichardErkhov
Params1.59B
Context32K
Q4_K_M1.0 GB
MI
Mistral_Pro_8B_v0.1
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
QW
abdulmannan 01 qwen 2.5 3b finetuned for sql generation
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
FU
FuseChat 7B v2.0
Fusechat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
LDCC Instruct Llama 2 Ko 13B v4.1.8
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
LL
GNER LLaMA 7B
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
FINGU AI Qwen2.5 14B Instruct Fine Tuned v3
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
MI
mlfoundations dev hp ablations mistral adambeta1 0.85 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
MI
mlfoundations dev hp ablations grid mistral bsz512 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
QW
Qwen2.5 1b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
LL
realPCH 240103 llama test 2
Llama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
_Spydaz_Web_AI_Llava
LLaVA · RichardErkhov
Params7.24B
Context512K
Q4_K_M4.4 GB
RE
reflect_llm8B_llm-mstlrg-om2-80k_SFTt2_psdp-t02_b.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
mathstral-7B-v0.1
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
PH
yuhonghhh microsoft phi 2 fine tune alpaca chinese merged model
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
TI
TinyLlama TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
brianmatzelle llama3.2 3b instruct hasanpiker abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
NO
Nous Hermes Platypus2 13B QLoRA 0.80 Epoch
Nous · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
PY
Pythia 160m Deduped
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
LL
llama-7b-sft-float32
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
VI
Vinallama 2.7b Chat
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
PS
Psyonic Cetacean V1 20B Ultra Quality Float32
Psyonic · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
OP
OpenELM 1_1B DPO Full Max 8 Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr5e 7 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB