GGUF models by Kondara
52 GGUF models quantized by Kondara on Hugging Face. Sizes run from 1.23B to 19.9B parameters. The longest trained context is 1000K tokens.
IN
Internlm2_5 20b
InternLM · Kondara
Params19.9B
Context256K
Q4_K_M12.0 GB
GE
Gemma 2 2b It
Gemma · Kondara
Params2.61B
Context8K
Smallest2.2 GB
MI
Mistral Nemo Base 2407
Mistral · Kondara
Params12.2B
Context1000K
Q4_K_M7.5 GB
GE
Gemma 2 9b It
Gemma · Kondara
Params9.24B
Context8K
Q4_K_M5.8 GB
KO
Komodo 7b Base
Komodo · Kondara
Params6.76B
Context4K
Q4_K_M4.1 GB
GE
Gemma 2 9b
Gemma · Kondara
Params9.24B
Context8K
Q4_K_M5.8 GB
SE
SeaLLMs v3 7B Chat
Seallms · Kondara
Params7.62B
Context128K
Q4_K_M4.7 GB
CE
Mt5 Large
Cendol · Kondara
Params1.23B
Context512
Q4_K_M0.8 GB
VI
Vicuna 7b v1.5 16k
Vicuna · Kondara
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama3 8b Cpt Sea Lionv2.1 Instruct
Llama · Kondara
Params8.03B
Context8K
Smallest8.5 GB
LL
Llama3 8b Cpt Sahabatai v1 Instruct
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama3 8b Cpt Sea Lionv2.1 Instruct
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Meta Llama 3 8B
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 Medium 4k Instruct
Phi · Kondara
Params14B
Context4K
Q4_K_M8.6 GB
MI
Mistral 7B Instruct v0.3
Mistral · Kondara
Params7.25B
Context32K
Q4_K_M4.4 GB
IN
Internlm2_5 1_8b Chat
InternLM · Kondara
Params1.89B
Context32K
Q4_K_M1.2 GB
IN
Internlm2_5 7b Chat 1m
InternLM · Kondara
Params7.74B
Context256K
Q4_K_M4.7 GB
IN
Internlm2_5 1_8b
InternLM · Kondara
Params1.89B
Context32K
Q4_K_M1.2 GB
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · Kondara
Params14.8B
Context128K
Q4_K_M9.0 GB
MI
Mistral NeMo Minitron 8B Base
Mistral · Kondara
Params8.41B
Context8K
Q4_K_M5.1 GB
LL
GBLM Pruner LLaMA 2 7B Chat
Llama · Kondara
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen2 7B Instruct
Qwen · Kondara
Params7.62B
Context32K
Q4_K_M4.7 GB