Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
thirdeyeai qwen2.5 .5b uncensored
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
GE
General Persona Chatbot v1 09 09 24
General · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mixtral_7Bx4_MOE_24B
Mistral · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
LG
lgaalves gpt2 guanaco dolly platypus
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
CH
ChenWeiLi MedLlama 3 8B DARE
Chenweili · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BR
BroAlanTaps GPT2 large 128 20000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
Llama-2-7b-sft-model-ocra-500k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
LL
akshit Gupta llamaitnew merged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
Llama 3.1 SauerkrautLM 8b Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama-3-MAAL-8B-Instruct-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
amod plnyr Phi 3.5 mini quote planName SFT v2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GE
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
Gemma · jaromer
Params25.2B
Context256K
Q4_K_M16.8 GB
KA
karthikeyan r slm custom model 20k pt
Karthikeyan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Raj Maharajwala Open Insurance LLM Llama3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
grounded ai phi3.5 hallucination judge merge
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
Llama3 8b It Prometheus Ko
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
CodeLlama 7b Instruct Hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
Llama 3.2 1B Instruct Gsm8K
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
MI
cloudyu Mixtral 7Bx2 MoE 13B
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
LL
Llama 3.1 8B InitializedEmbeddings_with_Hermes 3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
SA
SambaLingo Arabic Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
MI
arnavgrg mistral 7b nf4 fp16 upscaled
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
saiga2 13b
Saiga · IlyaGusev
Params13B
Context2K
Smallest5.4 GB
AR
ArianAskari SOLID SFT DPO MixQV3 SOLIDChosen SFTRejected Zephyr 7b beta
Arianaskari · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
KSU HW SEC Deepseek SFT RL Text DFG
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
FA
FaizanPervaz stablecode alpha 3b finetuned
Faizanpervaz · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
OP
OpenHermes-2.5-Nebula-v2-7B
Openhermes · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PA
Pastiche Crown Clown 7b Dare
Pastiche · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB