GGUF models by LiteLLMs
73 GGUF models quantized by LiteLLMs on Hugging Face. Sizes run from 272M to 141B parameters. The longest trained context is 1024K tokens.
OP
OpenELM-270M-Instruct
Openelm · LiteLLMs
Params272M
Context2K
Q4_K_M0.2 GB
OP
OpenELM-270M
Openelm · LiteLLMs
Params272M
Context2K
Q4_K_M0.2 GB
LL
Turkish-Llama-8b-v0.1
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
OP
OpenELM-450M
Openelm · LiteLLMs
Params457M
Context2K
Q4_K_M0.3 GB
OP
OpenELM-1_1B
Openelm · LiteLLMs
Params1.08B
Context2K
Q4_K_M0.7 GB
MI
Mistral Nemo Instruct 2407
Mistral · LiteLLMs
Params12.2B
Context1000K
Q4_K_M7.5 GB
ST
L3-8B-Stheno-v3.1
Stheno · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
RH
Rhea-72b-v0.5
Rhea · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
SH
Shieldgemma 2b
Shieldgemma · LiteLLMs
Params2.61B
Context8K
Q4_K_M1.7 GB
MO
Smaug-72B-v0.1
Momo · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
AY
aya-23-35B
Aya · LiteLLMs
Params35B
Context8K
Q4_K_M21.5 GB
YI
Yi-1.5-34B-Chat
Yi · LiteLLMs
Params34.4B
Context4K
Q4_K_M20.7 GB
GE
gemma-2b-it
Gemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
CO
Codegemma 2b
Codegemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
Mixtral-8x22B-Instruct-v0.1
Mistral · LiteLLMs
Params141B
Context64K
Q4_K_M85.6 GB
QW
free-evo-qwen72b-v0.8-re
Qwen · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
WI
WizardLM-70B-V1.0
Wizardlm · LiteLLMs
Params69B
Context4K
Q4_K_M41.4 GB
FA
Falcon
Falcon · LiteLLMs
Params11.1B
Context2K
Q4_K_M6.8 GB
CO
Codegemma 1.1 2b
Codegemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
Meta-Llama-3-120B-Instruct
Llama · LiteLLMs
Params120B
Context8K
Q4_K_M73.2 GB
PH
Phi 3 mini 4k instruct
Phi · LiteLLMs
Params3.82B
Context4K
Q4_K_M2.4 GB
LL
Meta-Llama-Guard-2-8B
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
PH
Phi 3 medium 128k instruct
Phi · LiteLLMs
Params14B
Context128K
Q4_K_M8.6 GB
LL
openbuddy-llama3-8b-v21.1-8k
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
CO
Codegemma 7b
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
LL
Meta-Llama-3-70B
Llama · LiteLLMs
Params70.6B
Context8K
Q4_K_M42.5 GB
SH
Shieldgemma 9b
Shieldgemma · LiteLLMs
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
Codestral-22B-v0.1
Mistral · LiteLLMs
Params22.2B
Context32K
Q4_K_M13.3 GB
CO
Codegemma 7b It
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
LL
Llama3-ChatQA-1.5-8B
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB