GGUF models by featherless-ai-quants

3092 GGUF models quantized by featherless-ai-quants on Hugging Face. Sizes run from 494M to 72.7B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
QW
huihui-ai/Qwen2.5-14B-Instruct-abliterated
Qwen · featherless-ai-quants
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
RO
TheDrummer/Rocinante-12B-v1.1
Rocinante · featherless-ai-quants
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
QW
huihui-ai/Qwen2.5-7B-Instruct-abliterated-v2
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
Sao10K/Llama-3.1-8B-Stheno-v3.4
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
huihui-ai/Qwen2.5-7B-Instruct-abliterated
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
QW
unsloth/Qwen2.5-Coder-7B-Instruct
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
QW
huihui-ai/Qwen2.5-32B-Instruct-abliterated
Qwen · featherless-ai-quants
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
LL
meta-llama/Llama-3.1-8B-Instruct
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
JA
stabilityai/japanese-stablelm-instruct-gamma-7b
Japanese · featherless-ai-quants
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
LL
meta-llama/Meta-Llama-3.1-8B-Instruct
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen/Qwen2.5-Coder-7B-Instruct
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
LU
Sao10K/L3-8B-Lunaris-v1
Lunaris · featherless-ai-quants
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
unsloth/Qwen2.5-Coder-7B
Qwen · featherless-ai-quants
Params7.62B
Context32K
Q4_K_M4.7 GB
Coding
LL
tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
Roleplay
QW
Qwen/Qwen2.5-Coder-1.5B
Qwen · featherless-ai-quants
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
DE
deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
DeepSeek · featherless-ai-quants
Params14.8B
Context128K
Q4_K_M9.0 GB
ReasoningTool use
QW
Qwen/Qwen2.5-Coder-14B-Instruct
Qwen · featherless-ai-quants
Params14.8B
Context32K
Q4_K_M9.0 GB
CodingTool use
LL
unsloth/Meta-Llama-3.1-8B
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
NE
mlabonne/NeuralDaredevil-8B-abliterated
Neuraldaredevil · featherless-ai-quants
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
meta-llama/Llama-Guard-3-8B
Llama · featherless-ai-quants
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
maximuslee07/llama-2-13b-rockwellautomation
Llama · featherless-ai-quants
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
QW
Qwen/Qwen2.5-Coder-32B-Instruct
Qwen · featherless-ai-quants
Params32.8B
Context32K
Q4_K_M19.9 GB
CodingTool use
VI
Vikhrmodels/Vikhr-Nemo-12B-Instruct-R-21-09-24
Vikhr · featherless-ai-quants
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
NousResearch/Meta-Llama-3-8B
Llama · featherless-ai-quants
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
tergel/gemma-2-2b-it-math-fs-gpt4o-bon
Gemma · featherless-ai-quants
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
AC
FreedomIntelligence/AceGPT-v2-8B-Chat
Acegpt · featherless-ai-quants
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CA
Nitral-AI/Captain-Eris_Violet-V0.420-12B
Captain · featherless-ai-quants
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
QW
Qwen/Qwen2-1.5B
Qwen · featherless-ai-quants
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LL
cognitivecomputations/Llama-3-8B-Instruct-abliterated-v2
Llama · featherless-ai-quants
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF

About featherless-ai-quants on Hugging Face

featherless-ai-quants has published 3,092 GGUF models that ggufmodels.com tracks, across families such as Llama, Qwen and Gemma. The most downloaded are huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated, huihui-ai/Qwen2.5-14B-Instruct-abliterated and TheDrummer/Rocinante-12B-v1.1.

The quants featherless-ai-quants publishes most often are Q8_0 (3092), Q6_K (3092), Q5_K_S (3092) and Q5_K_M (3092). Their Q4_K_M files range from 0.4 to 47.4 GB. Parameter counts run from 494M to 72.7B, and the longest trained context window is 1024K tokens.

Frequently asked questions

What GGUF models has featherless-ai-quants published?

3,092 models are listed here, mainly from the Llama, Qwen and Gemma families. The most downloaded is huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated.

Which quants does featherless-ai-quants publish?

The most common are Q8_0 (3092 models), Q6_K (3092 models), Q5_K_S (3092 models) and Q5_K_M (3092 models).

Keep browsing