GGUF models by mlabonne

38 GGUF models quantized by mlabonne on Hugging Face. Sizes run from 596M to 70.6B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama-3-8B-Instruct-abliterated-dpomix
Llama · mlabonne
Params8.03B
Context8K
Smallest8.5 GB
GGUF
TW
TwinLlama 3.1 8B
Twinllama · mlabonne
Params8.03B
Context128K
Smallest8.5 GB
GGUF
GE
Gemmalpaca-2B
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
NE
NeuralMarcoro14 7B
Neuralmarcoro · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
DatacampLlama 3.1 8B
Llama · mlabonne
Params8.03B
Context128K
Smallest8.5 GB
GGUF
EV
EvolCodeLlama 7b
Evolcodellama · mlabonne
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
PH
phi 2 orange v2
Phi · mlabonne
Params2.78B
Context2K
Smallest2.0 GB
GGUF
MO
Monarch 7B
Monarch · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing