Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
DC
Dclm Perplexity Correlations Spearmanr No Samp 160m
Dclm · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
LL
ArliAI Llama 3 8B Instruct DPO v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GR
Granite 3.0 2b Instruct
Granite · RichardErkhov
Params2.63B
Context4K
Q4_K_M1.6 GB
GE
ecdev gemma 2b instruct ft medical qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GP
gpt2-medium-vaguely-human-dialogue
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
IS
ishmanish gpt2 autotrain text HrPolicy aug v2
Ishmanish · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
RO
RoamifyMML financial advice model
Roamifymml · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Llama 3.2 3b It Library ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
LL
Llama-3.2-1B-Instruct-APIGen-FC-v0.1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
TA
TAIDE LX 7B Chat
Taide · nctu6
Params6.94B
Context4K
Q4_K_M4.2 GB
RW
rwitz2 go bruins v2.1.1
Rwitz · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
saishf West Hermes 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 Math 7B
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
BR
BroAlanTaps GPT2 large 256 17250steps 1.2Btokens
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
llama-2-7b-guanaco-instruct-sharded
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · Unrestricted
Params3.97B
Context1024K
Q4_K_M2.8 GB
LL
meta llama Llama 2 13b hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
TA
TatvaJoshi AHS new tinyllama spo faq v5.3
Tatvajoshi · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
KU
Kukedlc NeuralFusion 7b Dare Ties
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
akshit Gupta llama1Bredmerged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GE
gemma-2b-it
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
mesolitica malaysian llama 3 8b instruct 16k
Llama · RichardErkhov
Params8.03B
Context32K
Q4_K_M4.9 GB
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
PI
piotr ai polanka 3b pretrain full v0.4
Piotr · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GH
Ghost 8b Beta
Ghost · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
mlfoundations dev mistral alpaca sft sample 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
abdulmannan 01 gemma 2 2b finetuned for json generation
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MI
Mistral 7B Instruct SimPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OP
OpenELM 270M SFT D_chosen HuggingFaceH4 Ultrafeedback_Binarized Xlarge
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB