GGUF models by mlabonne
38 GGUF models quantized by mlabonne on Hugging Face. Sizes run from 596M to 70.6B parameters. The longest trained context is 128K tokens.
LL
Llama-3-8B-Instruct-abliterated-dpomix
Llama · mlabonne
Params8.03B
Context8K
Smallest8.5 GB
TW
TwinLlama 3.1 8B
Twinllama · mlabonne
Params8.03B
Context128K
Smallest8.5 GB
GE
Gemmalpaca-2B
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.6 GB
NE
NeuralMarcoro14 7B
Neuralmarcoro · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
DatacampLlama 3.1 8B
Llama · mlabonne
Params8.03B
Context128K
Smallest8.5 GB
EV
EvolCodeLlama 7b
Evolcodellama · mlabonne
Params6.74B
Context16K
Q4_K_M4.1 GB
PH
phi 2 orange v2
Phi · mlabonne
Params2.78B
Context2K
Smallest2.0 GB
MO
Monarch 7B
Monarch · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB