Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · vanchi34200
Params1.86B
Context256K
Smallest0.9 GB
GGUF
MI
Mistral 7B SFT DPO v0
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyDolphin-2.8-1.1b_oasst2_chatML_all_Cluster_dare_ties_v1
Tinydolphin · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
GGUF
ZE
zephyr-7b-dpo-full-prompt-extend-chosen-delete_0.1
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
simonycl llama 3 8b instruct armorm judge
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
CR
crestf411 daybreak kunoichi 2dpo 7b
Crestf · RichardErkhov
Params7.24B
Context16K
Q4_K_M4.4 GB
GGUF
LL
Llama-3-Base-8B-SFT
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
dolphin-2.9.3-qwen2-1.5b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LL
LLM LAT llama2 7b chat lat removed backdoor5
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
SA
SambaLingo Slovenian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
AI
AIFT Ko Orca Plat Yi Ko 6b v1.2 Dpo
Aift · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
LL
shawon majid llama 3 8B Instruct test smell detection raw
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
AP
Apollo 1.8B
Apollo · RichardErkhov
Params1.84B
Context8K
Q4_K_M1.2 GB
GGUF
JA
Jais Adapted 70b
Jais · RichardErkhov
Params69.5B
Context4K
Q4_K_M41.8 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo gpt4 llm
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BL
Blexus awareness test
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
DE
DeepSeek V4 Flash Vision Exp
DeepSeek · tarruda
Params284B
Context1024K
Smallest0.5 GB
GGUF
GR
Grafted Hermetic Platypus D 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 0.3
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Merge 14 v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
TW
Twice-KoSOLAR-16.1B-instruct-test
Twice · RichardErkhov
Params16.1B
Context4K
Q4_K_M9.7 GB
GGUF
ST
Stopwolf DistilabelCerberus 7B slerp
Stopwolf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
Phi 3 mini mango 1
Phi · rhysjones
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
Llama3 Aloe 8B Alpha
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IN
interview eval zephyr 7b math case 4
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
flozi00 Llama 2 13b german assistant v5
Llama · RichardErkhov
Params13.1B
Context2K
Q4_K_M7.9 GB
GGUF
ML
mlfoundations dev oh v1 2 only alpaca
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing