Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
martyn mixtral megamerge dare 8x7b v2
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M9.4 GB
EN
ENERGY DRINK LOVE eeve leaderboard inst v1.5
Energy · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
VI
VibeThinker 3B
Vibethinker · usermma
Params3.09B
Context128K
Smallest1.3 GB
PH
IntelLabs sqft phi 3 mini 4k 60 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GI
Gille StrangeMerges 8 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Open_Llama_3B
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
ZE
zephyr-7b-dpo-full-prompt-extend-chosen-delete_0.5
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
BR
broalantap GPT2 large 4 44000steps
Broalantap · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
PH
amod plnyr Phi 3.5 mini quote planName SFT PLNYR
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
TR
Tristan dclm fasttext oh eli5 410m
Tristan · RichardErkhov
Params405M
Context2K
Q4_K_M0.3 GB
RR
rrajvoyxa tinyllama helpdesk lang it v4
Rrajvoyxa · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
ML
mlfoundations dev oh v1.3 opengpt x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
NotASI Qwen2 0.5B Math
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GO
google codegemma 2b
Google · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
jan hq Llama3.1 s instruct 2024 09 09 cp 1000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral 7B Base SFT DPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
bunnycore Phi 3.5 Mini Sonet RP V2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
mlfoundations dev hp ablations mistral adambeta2 0.98 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-2-7b-dolphin-open_platypus-pruned_70
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GP
Gpt2Finetuned
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
TI
TinyAlpaca-1.1B
Tinyalpaca · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
RJ1200 llama 31b question
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
ML
mlfoundations dev oh v1 2 only metamath
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
OpenPipe mistral ft optimized 1218
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
TinyLlama-1.1B-intermediate-step-480k-1T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LG
lgaalves gpt2 platypus dolly guanaco
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
OP
OpenELM 3B SFT D_chosen HuggingFaceH4 Ultrafeedback_Binarized Xlarge
Openelm · RichardErkhov
Params3.04B
Context2K
Q4_K_M1.9 GB
LL
llama2-7b-hf-guanaco
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama 3.1 TAIDE LX 8B Chat
Llama · nctu6
Params8.52B
Context128K
Q4_K_M5.3 GB