Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
HA
Hastagaras L3.2 JametMini 3B MK.III
Hastagaras · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GE
nooynoos gemma2 2b M.O.M gemma sprint
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
AN
anthonymeo full train
Anthonymeo · RichardErkhov
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
KA
karthikeyan r calculation model 6k
Karthikeyan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
RE
RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4
Redpajama · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
AE
aetherwiing MN 12B Starcannon v2
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M3.4 GB
GGUF
LL
omniquad del llama3.2 text sql fp16 merged v1.1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ON
OneReward
Onereward · befox
Params11.9B
Context—
Smallest4.0 GB
GGUF
LL
OpenRLHF Llama 2 7b sft model ocra 500k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeActAgent Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
CodeLlama 13B Fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
BR
broalantap GPT2 large 4 16000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
GGUF
TR
Tristan dclm fasttext oh eli5 1b
Tristan · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 8
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama-3-8B-Instruct-function-calling-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
GE
christinechagas gemma 2b instruct ft health bot
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
sanchit gandhi distil mistral 1.5B v0.1
Mistral · RichardErkhov
Params1.57B
Context32K
Q4_K_M1.0 GB
GGUF
QW
masa research Qwen2 0.5B 20240829 175401
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
GGUF
XA
xammi final mp model merged
Xammi · RichardErkhov
Params1.1B
Context4K
Q4_K_M0.7 GB
GGUF
LL
llama-2-70B-instruct
Llama · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
synapsellm-7b-mistral-v0.4-preview2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
tinyllama-1.1b-sum-dpo-full_LR1e-7_2epochs
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MA
Marcoroni Neural Chat 7B v2
Marcoroni · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
juhwanlee llmdo Mistral 7B case 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch3 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
20240105_mistral Step50
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
hongzoh Yi 6B Open Platypus v2
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
PH
myfi parser phi 3.5 mini instruct epochs 4
Phi · RichardErkhov
Params3.82B
Context8K
Q4_K_M2.3 GB
GGUF
PY
Pythia 2.8b
Pythia · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF

Other quants

Keep browsing