Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
unclemusclez Unsloth Qwen2.5 Coder 1.5B Devinator v1
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
GE
Gemma4 26B A4B REAP 25
Gemma · crucible-labs
Params19.5B
Context256K
Smallest9.1 GB
ReasoningTool use
OG
Ogno Monarch Jaskier Merge 7b
Ogno · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 Storm 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
JU
juvi21 Hermes 2 Theta L3 Euryale Ties 0.8 70B
Juvi · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
LL
transitionGap PROPERTYMUTATION IN Llama3.1 8B smallset
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
NL
NLPinas yi bagel 2x34b
Nlpinas · RichardErkhov
Params34.4B
Context195K
Q4_K_M12.9 GB
GGUF
ZE
zephyr-7b-full-ut-sft
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · SSBOfficial
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
NO
Noor Fatima 01 gpt2 finetuned summary
Noor · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
Aspen77 llama 3 3b instruct extract pii
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
LL
broalantap Llama3 Compressor Adapter 4 12000steps
Llama · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
MI
Mistral 7B Base SFT CPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
nayem ng mdjannatulnayem llama2 7b finetuned casuallm lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
LL
Orenguteng Llama 3.1 8B Lexi Uncensored V2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama-3-70B-Instruct-Gradient-1048k
Llama · RichardErkhov
Params70.6B
Context1024K
Q4_K_M42.5 GB
GGUF
SC
scb10x typhoon 7b
Scb · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
GGUF
MO
MoE-Girl-1BA-7BT
Moe · allura-org
Params6.92B
Context8K
Q4_K_M4.2 GB
GGUF
GE
Gemma 2B Tele It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev OH original wo evol instruct 140k
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 3.1 8b ITA
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
OP
OpenELM-1_1B-Instruct
Openelm · LiteLLMs
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
AT
Atlas Chat 2B
Atlas · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
grounded ai phi3 toxicity judge merge
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF

Other quants

Keep browsing