Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
ST
Starling-LM-11B-alpha
Starling · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Configurable-Llama-3-8B-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
ReAligned Qwen3.5 0.8B
Qwen · Lazarus-Ai
Params752M
Context256K
Smallest0.4 GB
AN
andykcheng unitrend model 8b vllm
Andykcheng · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DE
deepseek-coder-1.3b-typescript
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
PE
PersianLLaMA 13B Instruct
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
CO
CodeQwen1.5 7B Trojan Clas2024 Development
Codeqwen · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
QW
Josiefied Qwen2.5 1.5B Instruct Abliterated v3
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
YA
yanolja EEVE Korean Instruct 10.8B v1.0
Yanolja · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
DE
DeepSeek V2 Lite Chat
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
MI
K2S3 Mistral 7b v1.2
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
LL
Llama3 KALE LM Chem 1.5 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
RN
Rnj 1 Instruct Heretic
Rnj · ZuzeTt
Params8.31B
Context32K
Q4_K_M5.1 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LF
LFM2.5 1.2B Instruct
Lfm · ZuzeTt
Params1.17B
Context125K
Q4_K_M0.7 GB
LL
meta llama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
FL
Flux2 Klein 9B True V2
Flux · fsxedx
Params9.08B
Context—
Smallest5.7 GB
LL
llama 2 7b langchain chat
Llama · shaowenchen
Params6.74B
Context4K
Smallest2.8 GB
HO
K2 Horizon 7B
Horizon · eaddario
Params9B
Context512K
Smallest0.0 GB
QW
Ebai3.6 27b unscensored
Qwen · vathvathv
Params26.9B
Context256K
Smallest10.0 GB
MI
K2S3 Mistral 7b v1.0
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
IN
InternLM2
InternLM · legraphista
Params7.74B
Context256K
Smallest1.9 GB
BA
baichuan2 7b base
Baichuan · shaowenchen
Params7.51B
Context4K
Smallest3.3 GB
MI
Ministral 3 14B Reasoning 2512
Mistral · eaddario
Params13.5B
Context256K
Q4_K_M8.2 GB
PR
Prometheus 8x7b v2.0
Prometheus · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · zihuiliu7737
Params122B
Context256K
Q4_K_M74.2 GB
BI
bidirlm omni 2.5b
Bidirlm · cstr
Params2.45B
Context125K
Smallest0.0 GB
QW
Qwen3.8-27B
Qwen · bramburn
Params27.3B
Context256K
Q4_K_M16.6 GB
QW
Qwen3.6 Uncensored Aggressive
Qwen · tripolskypetr
Params26.9B
Context256K
Q4_K_M21.2 GB