Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP GGUF q4kp
Qwen · lstari
Params1.86B
Context256K
Smallest0.9 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
FairMind Minerva 3B Instruct v1.0
Fairmind · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3 8B Instruct SPPO Score Iter2_gp_8b Table 0.002
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
hirushanan Code LLaMA 7b optimize code
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3 10b Wiki 240709 F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
GGUF
QW
Qwen1.5-1.8B-Chat
Qwen · RichardErkhov
Params1.84B
Context32K
Q4_K_M1.2 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
suzume-llama-3-8B-multilingual-orpo-borda-top25
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
LL
Youngwon Llama 3.2 1B info 4bit
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Dolphin 2.9 Llama3 8b 256k
Llama · RichardErkhov
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
LL
haqishen Llama 3 8B Japanese Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo camel ai chemistry
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Prosparse Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
PH
Phi 2 Electrical Engineering
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 3 epoch
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GR
Granite 3.0 8b Instruct
Granite · RichardErkhov
Params8.17B
Context4K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama 1.1B Intermediate Step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
GR
GritLM 8x7B
Gritlm · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
SA
SambaLingo Arabic Base 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
GGUF
MI
Mistral 7B Instruct v0.2 Function Calling
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
QwQ 32B
Qwen · dahara1
Params32.8B
Context128K
Q4_K_M19.9 GB
ReasoningTool use
SM
SmolLM3 3B
SmolLM · Bastion-AI
Params3.08B
Context64K
Q4_K_M1.9 GB
ReasoningTool use
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
QW
elinas Qwen2 11.3B
Qwen · RichardErkhov
Params11.3B
Context128K
Q4_K_M6.9 GB
GGUF
QW
cofeg Finetuned Xunzi Qwen2 1.5B for ancient text generation
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
MA
magnum v2 32b
Magnum · anthracite-org
Params32.5B
Context32K
Smallest14.7 GB
GGUF

Other quants

Keep browsing