Q5_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
GR
granite-8b-code-instruct
Granite · SanctumAI
Params8.05B
Context4K
Q4_K_M4.9 GB
LL
vicgalle Configurable Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2 7B Instruct Step DPO
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
LL
llama2-7b-chat-hf-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
llm-compiler-13b-ftd
Llm · legraphista
Params13B
Context16K
Smallest2.9 GB
SE
SeaLLMs v3 1.5B Chat
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
LA
Lawma 8b
Lawma · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
CH
chinese alpaca 2 1.3b rlhf
Chinese · hfl
Params1.26B
Context4K
Smallest0.6 GB
MA
magnum v2 12b
Magnum · anthracite-org
Params12.2B
Context1000K
Smallest5.7 GB
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
SE
SeaLLM 7B Chat
Seallm · parinzee
Params6.87B
Context4K
Q4_K_M4.2 GB
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
TI
TinySlime-1.1B-v1.0
Tinyslime · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GP
GPT OSS Cybersecurity Merged 20B Heretic Ara
gpt-oss · piotreknow02
Params20.9B
Context128K
Q4_K_M15.8 GB
LL
nirosunid Llama 3.1 8B bnb 4bit python v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
FL
Flux2 Klein 9B True V2
Flux · Leon1000
Params9.08B
Context—
Smallest5.7 GB
LL
Llama-3-Instruct-8B-SimPO
Llama · legraphista
Params8.03B
Context8K
Smallest2.0 GB
LL
llm-compiler-7b
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
QW
Qwen Image 2.1
Qwen · hhfw
Params8.19B
Context256K
Smallest3.3 GB
ML
mlfoundations dev stackoverflow 10000tasks 0p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ST
statking zephyr 7b sft full orpo
Statking · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
1 800 SHARED TASKS PHI3 Medium NLI 16bit
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
MI
Mistral 7B Merge 14 v0.1
Mistral · LiteLLMs
Params7.24B
Context32K
Q4_K_M4.4 GB
SU
Sungur 14B GGUF
Sungur · suayptalha
Params14.8B
Context40K
Q4_K_M9.0 GB
DM
dmis lab meerkat 7b v1.0
Dmis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 7B Instruct Abliterated v3
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · nurdich
Params1.86B
Context256K
Smallest0.9 GB
FL
Flux2 Klein 9B True V2
Flux · BIGJUTT
Params9.08B
Context—
Smallest5.7 GB