Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
EX
Excalibur 7b
Excalibur · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DE
AymanTarig deepseek coder 1.3b FC v2
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
DO
DongfuJiang prm version2 subsample hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
kimleang123 qwen 7b v2 khmer qa 1000 v10 merge model attn flash 2
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
KO
JY623 KoSOLAR 10.7B merge v3.0
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
ruslandev llama 3 8b gpt 4o ru1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
SFR Iterative DPO LLaMA 3 8B R
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GL
Glm 4 9b Chat
GLM · RichardErkhov
Params9.4B
Context128K
Q4_K_M6.3 GB
Tool use
LL
yujiepan llama 3 tiny random
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
GGUF
SM
akhilfau fine tuned smolLM2 physics 10
SmolLM · RichardErkhov
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
IN
internlm2-limarp-chat-20b
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
GE
Gemma 2 2b Jpn It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
GP
gpt2-small-italian-embeddings
Gpt · RichardErkhov
Params132M
Context1K
Q4_K_M0.1 GB
Embedding
CP
cpt_st-vicuna-v1.3-1.5b-ppl
Cpt · RichardErkhov
Params1.48B
Context2K
Q4_K_M0.9 GB
GGUF
MI
Nondzu Mistral 7B code 16k qlora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
tyson0420 stack codellama 7b inst
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
KH
KhanhTTS OmniVoice
Khanhtts · doof-ferb
Params—
Context—
Smallest1.0 GB
GGUF
NO
nota ai st vicuna v1.3 10.5b taylor
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
QW
Qwen3.5 0.8B Base
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB
ReasoningTool use
SH
Shwetaupadhyay7878 gita text generation gpt2
Shwetaupadhyay · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Open_Hermes_Orca_Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
HA
harry_potter_chatbot
Harry · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
CS
csg-wukong-1B-chat-v0.1
Csg · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
PH
disi unibo nlp phi3.5 SFT medqa triples nographeval cot
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 math heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
meta llama CodeLlama 7b Instruct hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NF
nfaheem Marcoroni 7b DPO Merge
Nfaheem · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing