Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3 8B Abliterated
Qwen · ZeroWw
Params8.19B
Context40K
Smallest5.5 GB
ReasoningTool use
MI
mlfoundations dev hp ablations mistral adambeta2 0.95 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Instruct KTO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Tess 2.0 Mixtral 8x7B
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
OP
OpenLLM Ro RoMistral 7b Instruct 2024 05 17
Openllm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
mlfoundations dev mistral 7b 0 3 oh dcft v3.1 llama 3.1 8b
Llama · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
NU
AI MO NuminaMath 72B TIR
Numinamath · RichardErkhov
Params72.7B
Context128K
Q4_K_M47.4 GB
GGUF
QW
Pretrain Qwen 500M
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
LL
Llama 2 Ko 7b
Llama · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.2 GB
GGUF
LL
Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
GP
Gpt2 Context_Generator
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
martyn mixtral megamerge dare 8x7b v2
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M9.4 GB
GGUF
EN
ENERGY DRINK LOVE eeve leaderboard inst v1.5
Energy · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
VI
VibeThinker 3B
Vibethinker · usermma
Params3.09B
Context128K
Smallest1.3 GB
CodingReasoningTool use
PH
IntelLabs sqft phi 3 mini 4k 60 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
GI
Gille StrangeMerges 8 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Open_Llama_3B
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
ZE
zephyr-7b-dpo-full-prompt-extend-chosen-delete_0.5
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BR
broalantap GPT2 large 4 44000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
PH
amod plnyr Phi 3.5 mini quote planName SFT PLNYR
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
TR
Tristan dclm fasttext oh eli5 410m
Tristan · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
GGUF
RR
rrajvoyxa tinyllama helpdesk lang it v4
Rrajvoyxa · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
ML
mlfoundations dev oh v1.3 opengpt x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
NotASI Qwen2 0.5B Math
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
GO
google codegemma 2b
Google · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
jan hq Llama3.1 s instruct 2024 09 09 cp 1000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Mistral 7B Base SFT DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
bunnycore Phi 3.5 Mini Sonet RP V2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
Roleplay
MI
mlfoundations dev hp ablations mistral adambeta2 0.98 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing