Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
SO
SOLAR Platypus 10.7B v2
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen 7B Chat Cantonese
Qwen · RichardErkhov
Params7.72B
Context32K
Q4_K_M4.9 GB
CH
chinese alpaca 2 7b 16k
Chinese · shaowenchen
Params6.93B
Context16K
Smallest2.9 GB
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
MI
CodeActAgent Mistral 7b v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Phi 3.5 Mini Instruct
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
PH
Phi 2 Layla v1 Chatml
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
YE
yejingfu MN 12B Starcannon v2
Yejingfu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
YI
Faro Yi 34B
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
MI
mia llm autotrain bloom 11b sft slm
Mia · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 8000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
JA
Jais Adapted 7b Chat
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
CA
CarbonVillain en 10.7B v3
Carbonvillain · LiteLLMs
Params10.7B
Context4K
Q4_K_M6.5 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
SM
jbaron34 SmolLM2 135M Bebop Reranker
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
QW
Aether-Qwen2-0.5B-SFT-v0.0.2
Qwen · thesven
Params494M
Context128K
Q4_K_M0.4 GB
IN
interview eval zephyr 7b math train
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OR
Yi Ko 6B Orca Alpaca Gpt4 Math Lora
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
EL
Eleusis 12B
Eleusis · PygmalionAI
Params12.2B
Context1000K
Smallest4.9 GB
LL
RuterNorway Llama 2 13b chat norwegian
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
Mistral 7b v0.1 Layla v4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
ahmedheakl qwen2.5 0.5b llamafac 500k
Llama · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
QW
Huihui Qwen3.8 27B Abliterated
Qwen · Astark-40
Params27.3B
Context256K
Smallest10.9 GB
GP
gpt2-medium-650k-stable-diffusion-prompt-generator
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
MI
mistral small 3.1 24b instruct 2503
Mistral · mrfakename
Params23.6B
Context32K
Smallest8.9 GB
LL
georgesung llama2 7b chat uncensored
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
ST
Stablecode Completion Alpha 3b 4k
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
QW
Josiefied Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
rsrishi10 gemma text2sql Instruct Finetune test
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB