GGUF models by mlabonne

38 GGUF models quantized by mlabonne on Hugging Face. Sizes run from 596M to 70.6B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · mlabonne
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 3 27b It Abliterated
Gemma · mlabonne
Params27B
Context128K
Q4_K_M16.5 GB
Vision
GE
Gemma 3 12b It Abliterated v2
Gemma · mlabonne
Params11.8B
Context128K
Q4_K_M7.3 GB
Vision
LL
Hermes 3 Llama 3.1 8B Lorablated
Llama · mlabonne
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
GE
Gemma 3 4b It Abliterated v2
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
GE
Gemma 3 4b It Abliterated
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
GE
Gemma 3 12b It Abliterated
Gemma · mlabonne
Params11.8B
Context128K
Q4_K_M7.3 GB
Vision
GE
Gemma 3 1b It Abliterated v2
Gemma · mlabonne
Params1B
Context32K
Q4_K_M0.8 GB
Vision
QW
Qwen3 0.6B Abliterated
Qwen · mlabonne
Params596M
Context40K
Q4_K_M0.4 GB
ReasoningTool use
GE
gemma-2b
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.5 GB
GGUF
GE
Gemma 3 1b It Abliterated
Gemma · mlabonne
Params1B
Context32K
Q4_K_M0.8 GB
Vision
GE
gemma-2b-it
Gemma · mlabonne
Params2.51B
Context8K
Q4_K_M1.5 GB
GGUF
LL
FineLlama 3.1 8B
Llama · mlabonne
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
Gemma 3 27b It Qat Abliterated
Gemma · mlabonne
Params27B
Context128K
Q4_K_M16.5 GB
Vision
LL
Llama 3.1 70B Instruct lorablated
Llama · mlabonne
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
GE
Gemma 3 12b It Qat Abliterated
Gemma · mlabonne
Params11.8B
Context128K
Q4_K_M7.3 GB
Vision
GE
gemma-7b-it
Gemma · mlabonne
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
GE
Gemma 3 4b It Qat Abliterated
Gemma · mlabonne
Params3.88B
Context128K
Q4_K_M2.5 GB
Vision
GE
Gemma 3 1b It Qat Abliterated
Gemma · mlabonne
Params1B
Context32K
Q4_K_M0.8 GB
Vision
AL
AlphaMonarch 7B
Alphamonarch · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
NeuralBeagle14 7B
Neuralbeagle · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
gemma-7b
Gemma · mlabonne
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
NE
NeuralDaredevil 8B abliterated
Neuraldaredevil · mlabonne
Params8B
Context—
Smallest0.3 GB
GGUF
DA
Daredevil-8B
Daredevil · mlabonne
Params8.03B
Context8K
Smallest8.5 GB
GGUF
KU
Beyonder 4x7B v3
Kunoichi · mlabonne
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF
DA
Daredevil-8B-abliterated
Daredevil · mlabonne
Params8.03B
Context8K
Smallest8.5 GB
GGUF
LL
Llama 3.1 8B
Llama · mlabonne
Params8.03B
Context128K
Smallest8.5 GB
GGUF
NE
NeuralMonarch 7B
Neuralmonarch · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
NeuralHermes 2.5 Mistral 7B
Mistral · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MI
NeuralHermes 2.5 Mistral 7B laser
Mistral · mlabonne
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use

About mlabonne on Hugging Face

mlabonne has published 38 GGUF models that ggufmodels.com tracks, across families such as Gemma, Llama and Mistral. The most downloaded are Meta Llama 3.1 8B Instruct Abliterated, Gemma 3 27b It Abliterated and Gemma 3 12b It Abliterated v2.

The quants mlabonne publishes most often are Q8_0 (34), Q5_K_M (31), Q4_K_M (30) and Q6_K (25). Their Q4_K_M files range from 0.4 to 42.5 GB. Parameter counts run from 596M to 70.6B, and the longest trained context window is 128K tokens.

Frequently asked questions

What GGUF models has mlabonne published?

38 models are listed here, mainly from the Gemma, Llama and Mistral families. The most downloaded is Meta Llama 3.1 8B Instruct Abliterated.

Which quants does mlabonne publish?

The most common are Q8_0 (34 models), Q5_K_M (31 models), Q4_K_M (30 models) and Q6_K (25 models).

Keep browsing