Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
XT
xTower13B
Xtower · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
Mistral 7B v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
TinyLlama TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
Nous-Hermes-Llama2-13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
MELT-TinyLlama-1.1B-Chat-v1.0
Melt · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
OP
OpenELM 1_1B DPO Full Max Min Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
QW
Rombos LLM V2.5 Qwen 32b
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
LL
ReBatch Llama 3 8B dutch
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
klcsp llama3 8b fft summarization 11 v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
mlfoundations dev llama2 oh teknium scaling down random 0.2
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B Instruct RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
microsoft Phi 3 mini 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
VI
Vizzy CA plates fine tuned gpt2 minimal test
Vizzy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
GraySwanAI Mistral 7B Instruct RR
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OL
OLMoE 1B 7B 0924
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
SY
Synatra_TbST02M_IN01
Synatra · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
Zhao Ching TWLLM Llama2 Extend VTLoss Enhanced
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
ME
MetaMath Cybertron
Metamath · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
MelangeB-70b
Melangeb · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GE
jstredacted gemma 2b instruct trivia qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
HU
ai human lab EEVE Korean Instruct 10.8B expo
Human · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
LL
Gemma 2b
Llama · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
nsmagar llama 2 7b law korean
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
OP
Openchat_3.5 Gpt 4 80k
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
nbeerbower llama3.1 gutenberg 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
TU
Tulu 2 Dpo 13b
Tulu · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
QW
Ostrich 27B Qwen3.6 260603
Qwen · etemiz
Params27.3B
Context256K
Q4_K_M16.8 GB
SW
v000000 SwallowMaid 8B L3 SPPO abliterated
Swallowmaid · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB