Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
TechxGenus CursorCore QW2.5 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
CH
chinese alpaca 2 7b
Chinese · hfl
Params6.93B
Context4K
Smallest2.6 GB
GGUF
ML
mlfoundations dev oh teknium scaling down random 1.0
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
zelk12 MT5 Gen1 gemma 2 9B
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
OG
Ogno Monarch Jaskier Merge 7b OH PREF DPO
Ogno · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
TowerInstruct Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
granite 4.1 30b
Granite · eaddario
Params28.9B
Context128K
Smallest0.0 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · heideng90
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
LL
Vikhr-Llama-3.2-1B-instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
EU
euclaise Memphis CoT 3B
Euclaise · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
TI
TinyLlama-1.1B-2.5T-chat-and-function-calling
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Tool use
DE
Srijith rkr deepseek base 1e 4 NO cot only failed samples 300 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
ST
rm_1014_20K_70K_step_400
Step · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
at2507 gemma 2b instruct ft python code instructions
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
Coding
ZE
Zephyr 7b Beta
Zephyr · RichardErkhov
Params7.2B
Context128K
Q4_K_M4.4 GB
GGUF
ML
mlfoundations dev oh v1.3 unnatural instructions x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
PH
aifeifei798 Phi 3 song lyrics 1.1
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · Riyan200324200324
Params1.86B
Context256K
Smallest0.9 GB
GGUF
MI
AlexeyRyzhikov mistral7b text to json v4.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AndreyRzhaksinskiy JS CodeLlama 7b 20241210
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
QV
Vikhrmodels/QVikhr-2.5-1.5B-Instruct-r
Qvikhr · Vikhrmodels
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
MI
MiniMax M3
Minimax · prometheusAIR
Params426B
Context1024K
Smallest171.7 GB
ReasoningTool useVision
MI
DPO_mistral_v01_7b_ultra_0130_1k
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Fireball Alpaca Llama3.1.07 8B Philos Math
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
TE
5techlab sft fine tuned model gpt2
Techlab · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
EC
Eclipse 13B Dpo
Eclipse · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.6
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
LLaMAntino-3-ANITA-8B-Inst-DPO-ITA
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PI
pippafeet-11B-0.1
Pippafeet · RichardErkhov
Params11.6B
Context32K
Q4_K_M7.0 GB
GGUF

Other quants

Keep browsing