Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
PS
Psyfighter2 Noromaid Ties Capybara 13B
Psyfighter · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
OP
OPEN SOLAR KO 10.7B
Open · RichardErkhov
Params10.9B
Context2K
Q4_K_M6.5 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
PH
jcorte real phi35 pd kb lora model
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
BR
BroAlanTaps GPT2 large 64 34000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
JU
MD Judge v0.1
Judge · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Hercules-2.0-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GA
garage bAInd Platypus 30B
Garage · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
danielgombas llama 1b step2 batch v7
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
mlfoundations dev llama3 1 8b webinstruct original 700k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
DO
DongfuJiang prm gsm all data bon 4 hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
VO
Voldemort 10B
Voldemort · RichardErkhov
Params10.7B
Context8K
Q4_K_M6.5 GB
IN
Intel neural chat 7b v3 3
Intel · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Higgs Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GE
Gemma 2 2b It
Gemma · dahara1
Params2.61B
Context8K
Q4_K_M1.7 GB
LL
BramVanroy Llama 2 13b chat dutch
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
AI
Airoboros 33b Gpt4 1.4.1 Lxctx PI 16384 Fp16
Airoboros · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
MI
Andrewwwwww Nous Hermes 2 Mixtral 8x7B DPO
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
SU
SUS Chat 34B
Sus · RichardErkhov
Params34.4B
Context8K
Q4_K_M20.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 7 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
openchat-3.6-8b-20240522
Llama · thesven
Params8.03B
Context8K
Q4_K_M4.9 GB
LU
Luban Marcoroni 13B v3
Luban · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
LL
Llama 3.1 0.5B Untrained
Llama · RichardErkhov
Params473M
Context8K
Q4_K_M0.4 GB
LL
open_llama_3b_600bt_preview
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
PH
rasyosef Phi 1 5 Instruct v0.1
Phi · RichardErkhov
Params1.41B
Context2K
Q4_K_M0.9 GB
LL
Llama 3.2 3B
Llama · RichardErkhov
Params1.94B
Context128K
Q4_K_M1.6 GB
LL
Llama 3.2 1B FC v2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB