Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
NI
Nitral AI Hathor Tahsin L3 8B v0.85
Nitral · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
genaitraining llama 2 7b domain tuned
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral_Tv Neural Marconroni
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 3.0 1b A400M Base
Granite · RichardErkhov
Params1.38B
Context4K
Q4_K_M0.9 GB
GGUF
JA
Jais Adapted 13b Chat
Jais · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
GGUF
MI
mlfoundations dev hp ablations grid mistral bsz2048 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BO
Bookworm-10.7B-v0.4-DPO
Bookworm · joongi007
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
OP
OpenELM 1_1B DPO Full Max 12 Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
OR
Orca2 13B Selfmerge 26B
Orca · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
IN
interview eval zephyr 7b math case 6 ep5
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
anakin87 Llama 3 8b ita ties
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
QW
Bharatdeep H portfolio query qwen 2.5 7b v3 cot
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
mlfoundations dev llama3 1 8b oh v3.1 wo caseus custom
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LE
Yi_lee v1 6B
Lee · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
QW
Replete LLM V2.5 Qwen 3b
Qwen · RichardErkhov
Params3.4B
Context32K
Q4_K_M2.1 GB
Tool use
QW
GermanEduScorer-Qwen2-1.5b
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
MA
Maestrale
Maestrale · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
rhaymison phi 3 portuguese tom cat 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
Tiny Random Llama 3
Llama · RichardErkhov
Params4M
Context4K
Q4_K_M0.0 GB
GGUF
MI
harshitv804 MetaMath Mistral 2x7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
Gemma · Ishowbackup
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool useVision
TI
tinyfrank 1.4B
Tinyfrank · andrijdavid
Params1.36B
Context2K
Q4_K_M0.8 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo5epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
IN
InfinityKumon 2x7B
Infinitykumon · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
BL
Bloomz 7b1 Mt Sft Chat
Bloomz · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
OP
OpenELM 1_1B DPO Full 1 5
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
SA
SauerkrautLM SOLAR Instruct
Sauerkrautlm · RichardErkhov
Params10.7B
Context8K
Q4_K_M6.5 GB
GGUF

Other quants

Keep browsing