Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
MI
Oh Mistral bs4096_lr5_00E 06_schedulerconstant_warmup1_00E 01_minlr
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
CH
Chatgpt2
Chatgpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GR
Granite Guardian 3.0 8b
Granite · RichardErkhov
Params8.17B
Context8K
Q4_K_M4.9 GB
ST
Starcoder2 15b
StarCoder · RichardErkhov
Params16B
Context16K
Q4_K_M9.9 GB
TI
Tinyllama_Alpaca_Sft_Sample
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
BI
bitnet_b1_58-large
Bitnet · RichardErkhov
Params729M
Context2K
Q4_K_M0.5 GB
SM
SmolLM 135M Instruct
SmolLM · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
LL
Llama 2 7b chat_FFT_Alpaca Gpt4 Zh
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
PH
dbands Phi 3.5 mini instruct pyomo pysim coder
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
LL
Llama 3.2 3B Instruct SFT D_chosen Pref Mix
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GE
vaniebermudez gemma 2b instruct ft squad qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
HE
Heejindo rationale model e10
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
mlfoundations dev llama3 1 8b oh dcft v1 wo camel chemistry gpt 4o mini
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MA
MathCoder L 7B
Mathcoder · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
CodeLlama 7b Hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MA
MarcusCosta Sumarizadors
Marcuscosta · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
QW
thirdeyeai qwen2.5 .5b uncensored
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
GE
General Persona Chatbot v1 09 09 24
General · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mixtral_7Bx4_MOE_24B
Mistral · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
LG
lgaalves gpt2 guanaco dolly platypus
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
CH
ChenWeiLi MedLlama 3 8B DARE
Chenweili · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BR
BroAlanTaps GPT2 large 128 20000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
Llama-2-7b-sft-model-ocra-500k
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
RI
Yi Ko 6B
Richarderkhov · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
LL
akshit Gupta llamaitnew merged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
Llama 3.1 SauerkrautLM 8b Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama-3-MAAL-8B-Instruct-v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
amod plnyr Phi 3.5 mini quote planName SFT v2
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GE
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
Gemma · jaromer
Params25.2B
Context256K
Q4_K_M16.8 GB
KA
karthikeyan r slm custom model 20k pt
Karthikeyan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB