Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
RO
RoamifyMML financial advice model
Roamifymml · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
Llama 3.2 3b It Library ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
LL
Llama-3.2-1B-Instruct-APIGen-FC-v0.1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
TA
TAIDE LX 7B Chat
Taide · nctu6
Params6.94B
Context4K
Q4_K_M4.2 GB
RW
rwitz2 go bruins v2.1.1
Rwitz · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
saishf West Hermes 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 Math 7B
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
BR
BroAlanTaps GPT2 large 256 17250steps 1.2Btokens
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
llama-2-7b-guanaco-instruct-sharded
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
NE
Nemotron3-Nano-4B-Uncensored-HauhauCS-Aggressive
Nemotron · Unrestricted
Params3.97B
Context1024K
Q4_K_M2.8 GB
LL
meta llama Llama 2 13b hf
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
TA
TatvaJoshi AHS new tinyllama spo faq v5.3
Tatvajoshi · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
KU
Kukedlc NeuralFusion 7b Dare Ties
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
akshit Gupta llama1Bredmerged FinetunedByAG
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GE
gemma-2b-it
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
mesolitica malaysian llama 3 8b instruct 16k
Llama · RichardErkhov
Params8.03B
Context32K
Q4_K_M4.9 GB
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
PI
piotr ai polanka 3b pretrain full v0.4
Piotr · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GH
Ghost 8b Beta
Ghost · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
mlfoundations dev mistral alpaca sft sample 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
abdulmannan 01 gemma 2 2b finetuned for json generation
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
MI
Mistral 7B Instruct SimPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
OP
OpenELM 270M SFT D_chosen HuggingFaceH4 Ultrafeedback_Binarized Xlarge
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
LL
DsnTgr llama 3.2 3b it Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
AL
ALMA-13B-R
Alma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GE
gemma-2b-it-tamil-v0.1-alpha
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MA
Manticore 30b Chat Pyg Alpha
Manticore · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
MI
thkim0305 Mistral 7B ours algo7s lyr20 n11 0.5 0.5 0.1 0.1 200steps full
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ME
Medicine Chat
Medicine · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB