Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
PE
perlthoughts neural chat v3 3 8x7b MoE
Perlthoughts · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
llama-3-chinese-8b-instruct-v2
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
ME
mergekit community mergekit model stock hxpoald
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
ZE
zephyr-7b-dpo-full-prompt-extend-bad
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AL
allknowingroger MultiverseEx26 7B slerp
Allknowingroger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
Aira-2-1B1
Aira · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
SFR-Iterative-DPO-LLaMA-3-8B-R
Llama · thesven
Params8.03B
Context8K
Q4_K_M4.9 GB
TO
tongliuphysics zephyr 7b ultra p 0.03
Tongliuphysics · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral epoch5 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
Phi 3 Mini 4k Instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
MI
skymizer mistral 7B v0.1 csft relufication stage 1 on slim orca
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 32B Instruct
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
LL
Meta Llama Guard 2 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Nayana_Qwen2_VL_OCR
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
selectorseb llama3.1 8b drone function calling chatformat
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
dat lequoc fast apply 16bit v0.v15.4 Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
mlfoundations dev hp ablations mistral lr8e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.2 1B Instruct SFT D_chosen Pref Mix4
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
JI
JiuZhang3.0 7B
Jiuzhang · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Wengwengwhale llama 3.1 8B Instruct finance analyst
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
KU
Kunoichi-7B
Kunoichi · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
ZE
Zephyr 7b Sft Full SPIN Iter1
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
DA
datarock labs telekom cpt quantized test
Datarock · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
Llama2 13b Lima Sft Dpo
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
QW
OpenCodeReasoning Nemotron 1.1 32B
Qwen · quantized4all
Params32.8B
Context64K
Q4_K_M19.9 GB
VI
Vistral 7B Chat
Vistral · RichardErkhov
Params7.29B
Context32K
Q4_K_M4.4 GB
QW
Gukbap Qwen2 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB