Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
s3nh nsfw noromaid mistral instruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SH
Shieldgemma 2b
Shieldgemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DP
Yi Ko 6B DPO v2
Dpo · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
MI
Mixtral_7Bx2_MoE_13B
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
NO
nothingiisreal L3 8B Celeste v1
Nothingiisreal · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
AL
allknowingroger MultiverseEx26 Neurallaymons 12B MoE
Allknowingroger · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
GE
yhkim9362 gemma en ko 7b v0.1
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
DE
yiran wang3 clean mining sft split deepseek packing sft
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
AS
asiansoul SmartLlama 3 Ko 8B 256k PoSE
Asiansoul · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
thusinh1969 Guardian V0.1 LLaMA3.1 8B 5G 6Oct2024 epoch1.3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
GE
AXCXEPT EZO Humanities 9B gemma 2 it
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SA
saucam Arithmo Wizard 2 7B
Saucam · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
IN
InternLM2
InternLM · RichardErkhov
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
VI
Vidyayini gita text generation gpt2
Vidyayini · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
openinstruct-mistral-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
OpenELM 1_1B DPO Full 2 5
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
LL
chinese llama 2 7b 16k
Llama · hfl
Params6.93B
Context16K
Smallest2.6 GB
GGUF
DA
H2O Danube3 500m Base
Danube · RichardErkhov
Params514M
Context8K
Q4_K_M0.3 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
ricepaper vi gemma 2b RAG
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
TI
Tiny-Cowboy-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GE
Lion Gemma 7b Cn
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
MI
UNA-dolphin-2.6-mistral-7b-dpo-laser
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MeghanathReddy gita text generation gpt2
Meghanathreddy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
OP
OpenLLM Ro RoLlama3 8b Instruct 2024 06 28
Openllm · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Other quants

Keep browsing