GGUF models by LiteLLMs
73 GGUF models quantized by LiteLLMs on Hugging Face. Sizes run from 272M to 141B parameters. The longest trained context is 1024K tokens.
AY
aya-23-8B
Aya · LiteLLMs
Params8.03B
Context8K
Q4_K_M5.1 GB
GE
gemma-1.1-7b-it
Gemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
CO
Codegemma 1.1 7b It
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
LL
openchat-3.6-8b-20240522
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen1.5-110B-Chat
Qwen · LiteLLMs
Params111B
Context32K
Q4_K_M67.2 GB
LL
Llama-3-Giraffe-70B
Llama · LiteLLMs
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
Llama 3 8B Instruct 262k
Llama · LiteLLMs
Params8.03B
Context256K
Q4_K_M4.9 GB
LL
Ko-Llama3-Luxia-8B
Llama · LiteLLMs
Params8.17B
Context8K
Q4_K_M5.0 GB
MA
mathstral 7B v0.1
Mathstral · LiteLLMs
Params7.25B
Context32K
Q4_K_M4.4 GB
GE
gemma-2b
Gemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
Llama 3 8B 16K
Llama · LiteLLMs
Params8.03B
Context16K
Q4_K_M4.9 GB
SH
Shieldgemma 27b
Shieldgemma · LiteLLMs
Params27.2B
Context8K
Q4_K_M16.6 GB
MI
Tess-2.0-Mixtral-8x22B
Mistral · LiteLLMs
Params141B
Context64K
Q4_K_M85.6 GB