GGUF models by Kondara

52 GGUF models quantized by Kondara on Hugging Face. Sizes run from 1.23B to 19.9B parameters. The longest trained context is 1000K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
Internlm2_5 20b
InternLM · Kondara
Params19.9B
Context256K
Q4_K_M12.0 GB
GGUF
GE
Gemma 2 2b It
Gemma · Kondara
Params2.61B
Context8K
Smallest2.2 GB
GGUF
MI
Mistral Nemo Base 2407
Mistral · Kondara
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
GE
Gemma 2 9b It
Gemma · Kondara
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
KO
Komodo 7b Base
Komodo · Kondara
Params6.76B
Context4K
Q4_K_M4.1 GB
GGUF
GE
Gemma 2 9b
Gemma · Kondara
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
SE
SeaLLMs v3 7B Chat
Seallms · Kondara
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
CE
Mt5 Large
Cendol · Kondara
Params1.23B
Context512
Q4_K_M0.8 GB
GGUF
VI
Vicuna 7b v1.5 16k
Vicuna · Kondara
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2.1 Instruct
Llama · Kondara
Params8.03B
Context8K
Smallest8.5 GB
GGUF
LL
Llama3 8b Cpt Sahabatai v1 Instruct
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama3 8b Cpt Sea Lionv2.1 Instruct
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3 8B
Llama · Kondara
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Medium 4k Instruct
Phi · Kondara
Params14B
Context4K
Q4_K_M8.6 GB
Coding
MI
Mistral 7B Instruct v0.3
Mistral · Kondara
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
IN
Internlm2_5 1_8b Chat
InternLM · Kondara
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
IN
Internlm2_5 7b Chat 1m
InternLM · Kondara
Params7.74B
Context256K
Q4_K_M4.7 GB
GGUF
IN
Internlm2_5 1_8b
InternLM · Kondara
Params1.89B
Context32K
Q4_K_M1.2 GB
GGUF
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · Kondara
Params14.8B
Context128K
Q4_K_M9.0 GB
ReasoningTool use
MI
Mistral NeMo Minitron 8B Base
Mistral · Kondara
Params8.41B
Context8K
Q4_K_M5.1 GB
GGUF
LL
GBLM Pruner LLaMA 2 7B Chat
Llama · Kondara
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
Qwen2 7B Instruct
Qwen · Kondara
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF

Keep browsing