Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
Tiny-Pirate-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
YI
dolphin-2.9.1-yi-1.5-34b
Yi · RichardErkhov
Params34.4B
Context8K
Q4_K_M20.7 GB
GGUF
LL
BramVanroy llama2 13b ft mc4 nl cleaned tiny
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
LDCC Instruct Llama 2 Ko 13B v4.1.2
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
FU
FusionNet
Fusionnet · RichardErkhov
Params21.2B
Context4K
Q4_K_M12.7 GB
GGUF
DI
Diksha2001 VLLM new
Diksha · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
cais HarmBench Llama 2 13b cls multimodal behaviors
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
LL
vicgalle Configurable Llama 3 8B v0.3
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mixtral-8x7B-Instruct-v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
GE
Gemma 7b Aps It
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
GE
Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
Gemma · braydenh563
Params4.65B
Context128K
Q4_K_M3.5 GB
ReasoningTool use
CH
Christoph LSN gpt2 finetuned on metadata
Christoph · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
TI
TinyLlama-1.1B-intermediate-step-240k-503b
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
BU
BugTraceAI Apex G4 26B Master
Bugtraceai · piotreknow02
Params25.2B
Context256K
Q4_K_M16.8 GB
ReasoningTool use
RU
RuishiCh0314 code mix model finetuned
Ruishich · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
Coding
ST
starcoder2-7b
StarCoder · RichardErkhov
Params7.17B
Context16K
Q4_K_M4.5 GB
Coding
KU
Kunoichi-DPO-7B
Kunoichi · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh teknium scaling down ratiocontrolled 0.6
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo collective cognition
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
ST
starcoder2-15b-instruct-v0.1
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
Coding
ML
mlfoundations dev oh v1.3 camel chemistry x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
CH
K2-Chat
Chat · legraphista
Params65.3B
Context8K
Smallest14.2 GB
GGUF
PH
Phi 3 medium 128k instruct
Phi · thesven
Params14B
Context128K
Q4_K_M8.6 GB
Coding
LL
Bllossom70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
SM
SmolLlama-1.5B
Smolllama · RichardErkhov
Params1.54B
Context2K
Q4_K_M0.9 GB
GGUF
QW
Qwen3.6 35B A3B MTP
Qwen · td-builder
Params35.5B
Context256K
Smallest13.2 GB
ReasoningTool useVision
MI
Mistral 7B Instruct V0.2_Bio Tutor_Dpo
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
smol_llama-220M-GQA-fineweb_edu
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
GGUF
MY
Myrrh_solar_10.7b_3.0
Myrrh · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF

Other quants

Keep browsing