GGUF models by NeverSleep
40 GGUF models quantized by NeverSleep on Hugging Face. Sizes run from 7.24B to 125B parameters. The longest trained context is 1000K tokens.
SO
SOLAR Maid 4x10.7B
Solar · NeverSleep
Params36.1B
Context16K
Q4_K_M20.4 GB
MI
NoromaidxOpenGPT4 1 GGUF iMatrix
Mistral · NeverSleep
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
NeverSleep_Llama-3-Lumimaid-70B-v0.1-alt
Llama · NeverSleep
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
MiquMaid v1 70B
Miqumaid · NeverSleep
Params69B
Context32K
Q4_K_M41.4 GB
MI
MiquMaid-v2-70B-DPO
Miqumaid · NeverSleep
Params69B
Context32K
Q4_K_M41.4 GB
LL
NeverSleep_Llama-3-Lumimaid-70B-v0.1-OAS
Llama · NeverSleep
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
MiquMaid v2 70B
Miqumaid · NeverSleep
Params69B
Context32K
Q4_K_M41.4 GB
MI
MiquMaid v2 2x70B
Miqumaid · NeverSleep
Params125B
Context32K
Smallest45.9 GB
MI
MiquMaid v3 70B
Miqumaid · NeverSleep
Params69B
Context32K
Smallest38.9 GB
LU
Lumi 8B v0.3 run3 e3.0
Lumi · NeverSleep
Params8.03B
Context8K
Q4_K_M4.9 GB