Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
CA
cais Zephyr RMU
Cais · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
jointpreferences mistral 7b sft helpful
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3.5 4B
Qwen · eaddario
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool useVision
QW
VanillaKD Pretrain Qwen 200M
Qwen · RichardErkhov
Params203M
Context32K
Q4_K_M0.2 GB
GGUF
IN
MD Judge V0_2 Internlm2_7B
InternLM · RichardErkhov
Params7.74B
Context32K
Q4_K_M4.7 GB
GGUF
CH
chujiezheng tulu 2 dpo 13b ExPO
Chujiezheng · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
SA
Salamandra 2b
Salamandra · RichardErkhov
Params2.25B
Context8K
Q4_K_M1.5 GB
GGUF
GE
Gargaz gemma 2b romanian better
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
NE
NexaAIDev Octopus v2
Nexaaidev · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Cendol Llama2 13b Merged
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
CO
Yi Coder 1.5B Chat
Coder · legraphista
Params1.48B
Context128K
Smallest0.5 GB
Coding
CY
z image cyberrealistic v4
Cyberrealistic · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
GGUF
CR
crackthejob codingLLM
Crackthejob · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama-1.1B-step-50K-105b
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Na0s Llama 3.1 8B Pruned 4 Layers LoRA PEFT 3.0
Llama · RichardErkhov
Params6.94B
Context128K
Q4_K_M4.3 GB
GGUF
QW
Qwen3.8-27B
Qwen · Sh200126
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
LL
winglian llama 3 8b 256k PoSE
Llama · RichardErkhov
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
BR
Breeze-7B-Base-v1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
BI
Bifrost-27B-GGUF
Bifrost · Joumdane
Params27B
Context128K
Q4_K_M16.5 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · KilomAkel
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
SE
sense voice
Sense · lovemefan
Params234M
Context—
Smallest0.2 GB
GGUF
QW
Josiefied Qwen2.5 7B Instruct Abliterated v2
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
GL
GLM 4.6V Flash
GLM · eaddario
Params9.4B
Context128K
Q4_K_M6.2 GB
ReasoningTool useVision
GE
GeM2 Llamion 14B Base
Gem · RichardErkhov
Params14.5B
Context4K
Q4_K_M8.8 GB
GGUF
AU
AutoCoder_S_6.7B
Autocoder · legraphista
Params6.74B
Context16K
Smallest1.5 GB
Coding
YI
Yi-1.5-9B-Chat-16K
Yi · RichardErkhov
Params8.83B
Context16K
Q4_K_M5.3 GB
GGUF
AR
archangel_sft_pythia2-8b
Archangel · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
QW
Qwen2.5 14B Instruct Abliterated
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
QW
Qwen2.5 Coder 1.5B Instruct Abliterated
Qwen · URWI08RUS
Params1.78B
Context32K
Q4_K_M1.1 GB
CodingTool use

Other quants

Keep browsing