GGUF models by LiteLLMs

73 GGUF models quantized by LiteLLMs on Hugging Face. Sizes run from 272M to 141B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
OP
OpenELM-270M-Instruct
Openelm · LiteLLMs
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
OP
OpenELM-270M
Openelm · LiteLLMs
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
LL
Turkish-Llama-8b-v0.1
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OP
OpenELM-450M
Openelm · LiteLLMs
Params457M
Context2K
Q4_K_M0.3 GB
GGUF
OP
OpenELM-1_1B
Openelm · LiteLLMs
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
MI
Mistral Nemo Instruct 2407
Mistral · LiteLLMs
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
ST
L3-8B-Stheno-v3.1
Stheno · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
RH
Rhea-72b-v0.5
Rhea · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
GGUF
SH
Shieldgemma 2b
Shieldgemma · LiteLLMs
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
MO
Smaug-72B-v0.1
Momo · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
GGUF
AY
aya-23-35B
Aya · LiteLLMs
Params35B
Context8K
Q4_K_M21.5 GB
GGUF
YI
Yi-1.5-34B-Chat
Yi · LiteLLMs
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
GE
gemma-2b-it
Gemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
CO
Codegemma 2b
Codegemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
Mixtral-8x22B-Instruct-v0.1
Mistral · LiteLLMs
Params141B
Context64K
Q4_K_M85.6 GB
GGUF
QW
free-evo-qwen72b-v0.8-re
Qwen · LiteLLMs
Params72.3B
Context32K
Q4_K_M43.8 GB
GGUF
WI
WizardLM-70B-V1.0
Wizardlm · LiteLLMs
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
FA
Falcon
Falcon · LiteLLMs
Params11.1B
Context2K
Q4_K_M6.8 GB
GGUF
CO
Codegemma 1.1 2b
Codegemma · LiteLLMs
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Meta-Llama-3-120B-Instruct
Llama · LiteLLMs
Params120B
Context8K
Q4_K_M73.2 GB
GGUF
PH
Phi 3 mini 4k instruct
Phi · LiteLLMs
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
LL
Meta-Llama-Guard-2-8B
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 medium 128k instruct
Phi · LiteLLMs
Params14B
Context128K
Q4_K_M8.6 GB
Coding
LL
openbuddy-llama3-8b-v21.1-8k
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CO
Codegemma 7b
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
Meta-Llama-3-70B
Llama · LiteLLMs
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
SH
Shieldgemma 9b
Shieldgemma · LiteLLMs
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Codestral-22B-v0.1
Mistral · LiteLLMs
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
CO
Codegemma 7b It
Codegemma · LiteLLMs
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
Llama3-ChatQA-1.5-8B
Llama · LiteLLMs
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

Keep browsing