Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2 0.5B Instruct
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
GGUF
MI
DeepKarkhanis Mistral Passthrough 8L 10B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Neural-grok-dolphin-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
StarCoder
StarCoder · RichardErkhov
Params164M
Context8K
Q4_K_M0.1 GB
Coding
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
HA
harkov000 hermes dolphin 8B
Harkov · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr5e 7 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral bsz1024 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
ChaoticNeutrals Poppy Porpoise v0.6 L3 8B
Chaoticneutrals · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
sonthenguyen NeuralHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
JO
JOSIEv4o 8b Stage1 Beta1
Josiev · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
IL
ilsp Meltemi 7B v1
Ilsp · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
GGUF
GE
Gemma 2 Baku 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
QW
leniachat-qwen2-1.5B-v0
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
LL
abhinand Llama 3 Galen 8B 32k v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ON
Only Nvidia_12684
Only · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
Tool use
QW
Qwen3.5 397B A17B
Qwen · ubergarm
Params396B
Context256K
Smallest1.1 GB
GGUF
MI
nbeerbower mistral nemo gutenberg 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
MI
Radintloom Mistral 7b Fusion
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AI
AIdenU SOLAR 10.7b ko Y24 v0.1
Aidenu · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
DE
KSU HW SEC Deepseek SFT RL AST only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
lotfsadek llama3.1 finetuned py v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Weyaxi MetaMath NeuralHermes 2.5 Mistral 7B Linear
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PH
Ganidu Phi 3 mini 4k OLScience v02
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
BL
Blexus Quble test model v1 INSTRUCT v1
Blexus · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
FL
flux1 kontext dev quants
Flux · marduk191
Params11.9B
Context—
Smallest4.0 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use

Other quants

Keep browsing