Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
QW
Qwen2.5 1b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
LL
realPCH 240103 llama test 2
Llama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
_Spydaz_Web_AI_Llava
LLaVA · RichardErkhov
Params7.24B
Context512K
Q4_K_M4.4 GB
RE
reflect_llm8B_llm-mstlrg-om2-80k_SFTt2_psdp-t02_b.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
mathstral-7B-v0.1
Mistral · legraphista
Params7.25B
Context32K
Smallest1.6 GB
PH
yuhonghhh microsoft phi 2 fine tune alpaca chinese merged model
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
TI
TinyLlama TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
brianmatzelle llama3.2 3b instruct hasanpiker abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
NO
Nous Hermes Platypus2 13B QLoRA 0.80 Epoch
Nous · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
PY
Pythia 160m Deduped
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
LL
llama-7b-sft-float32
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
VI
Vinallama 2.7b Chat
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
PS
Psyonic Cetacean V1 20B Ultra Quality Float32
Psyonic · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
OP
OpenELM 1_1B DPO Full Max 8 Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr5e 7 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SH
shubham11goel gita text generation gpt2
Shubham · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
BE
Beepo 22B
Beepo · concedo
Params22.2B
Context32K
Q4_K_M13.3 GB
LL
mobinln Llama 3.2 1B Instruct fa
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GD
gdshaji gd tp 5k v1
Gdshaji · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
simonycl llama 3.1 8b instruct armorm judge iter2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2b It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MO
Moderate_Everything
Moderate · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ET
EtherealRainbow v0.3 8B
Etherealrainbow · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
saransh03sharma mintrec llama 2 7b Folder 5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
mlfoundations dev hp ablations mistral epoch5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
rm_1014_70K_step_600_lr_1e6
Step · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
bunnycore Phi 3.5 mini TitanFusion 0.6
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
IN
inflatebot L3 8B Helium3 baseStheno
Inflatebot · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB