Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 7B Instruct Uncensored
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Meta Llama 3 70B Instruct
Llama · hermes42
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
LL
vicgalle Humanish Roleplay Llama 3.1 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
FL
flux1 tests
Flux · TylorShine
Params11.9B
Context—
Q4_K_M6.9 GB
GGUF
BA
baichuan2 7b chat
Baichuan · shaowenchen
Params7.51B
Context4K
Smallest3.3 GB
GGUF
GE
Turkish Gemma 9b T1
Gemma · ytu-ce-cosmos
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
ZE
gguf zephyr 220m dpo full
Zephyr · Felladrin
Params218M
Context2K
Q4_K_M0.2 GB
GGUF
LL
Llama 2 7b Ebft Sparsegpt Outlier Wiki Block Outlier
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
vicgalle Configurable Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2 7B Instruct Step DPO
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
LL
llama2-7b-chat-hf-v3
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
llm-compiler-13b-ftd
Llm · legraphista
Params13B
Context16K
Smallest2.9 GB
GGUF
SE
SeaLLMs v3 1.5B Chat
Seallms · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LA
Lawma 8b
Lawma · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
CH
chinese alpaca 2 1.3b rlhf
Chinese · hfl
Params1.26B
Context4K
Smallest0.6 GB
GGUF
TI
TinyLlama 1.1B Chat v0.2
Tinyllama · kirp
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
SE
SeaLLM 7B Chat
Seallm · parinzee
Params6.87B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
TI
TinySlime-1.1B-v1.0
Tinyslime · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GP
GPT OSS Cybersecurity Merged 20B Heretic Ara
gpt-oss · piotreknow02
Params20.9B
Context128K
Q4_K_M15.8 GB
Tool use
LL
nirosunid Llama 3.1 8B bnb 4bit python v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama-3-Instruct-8B-SimPO
Llama · legraphista
Params8.03B
Context8K
Smallest2.0 GB
GGUF
LL
llm-compiler-7b
Llm · legraphista
Params6.74B
Context16K
Smallest1.5 GB
GGUF
QW
Qwen Image 2.1
Qwen · hhfw
Params8.19B
Context256K
Smallest3.3 GB
CodingTool useVision
ML
mlfoundations dev stackoverflow 10000tasks 0p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 7000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
ST
statking zephyr 7b sft full orpo
Statking · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
1 800 SHARED TASKS PHI3 Medium NLI 16bit
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
GGUF

Other quants

Keep browsing