GGUF models by LiteLLMs

73 GGUF models quantized by LiteLLMs on Hugging Face. Sizes run from 272M to 141B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
AY
aya-23-8B
Aya · LiteLLMs
Params8.03B
Context8K
Q4_K_M5.1 GB
GGUF
GE
gemma-1.1-7b-it
Gemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
CO
Codegemma 1.1 7b It
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
openchat-3.6-8b-20240522
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-110B-Chat
Qwen · LiteLLMs
Params111B
Context32K
Q4_K_M67.2 GB
GGUF
LL
Llama-3-Giraffe-70B
Llama · LiteLLMs
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
Llama 3 8B Instruct 262k
Llama · LiteLLMs
Params8.03B
Context256K
Q4_K_M4.9 GB
GGUF
LL
Ko-Llama3-Luxia-8B
Llama · LiteLLMs
Params8.17B
Context8K
Q4_K_M5.0 GB
GGUF
MA
mathstral 7B v0.1
Mathstral · LiteLLMs
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
GE
gemma-2b
Gemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Llama 3 8B 16K
Llama · LiteLLMs
Params8.03B
Context16K
Q4_K_M4.9 GB
GGUF
SH
Shieldgemma 27b
Shieldgemma · LiteLLMs
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
MI
Tess-2.0-Mixtral-8x22B
Mistral · LiteLLMs
Params141B
Context64K
Q4_K_M85.6 GB
GGUF

Keep browsing