GGUF models by flyfishstudio
42 GGUF models quantized by flyfishstudio on Hugging Face. Sizes run from 494M to 9.24B parameters. The longest trained context is 128K tokens.
TI
tinyllama 1.1b chat v1.0 Q4 0 4 8
Tinyllama · flyfishstudio
Params1.1B
Context2K
Smallest0.6 GB
SM
SmolLM 1.7B
SmolLM · flyfishstudio
Params1.71B
Context2K
Smallest1.0 GB
LL
yarn llama 2 7b 128k Q4 0 4 8
Llama · flyfishstudio
Params6.74B
Context128K
Smallest3.8 GB
LL
Hermes 3 Llama 3.1 8B
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
SM
SmolLM 1.7B
SmolLM · flyfishstudio
Params1.71B
Context2K
Smallest1.0 GB
LL
llama 2 7b chat Q4 0 4 4
Llama · flyfishstudio
Params6.74B
Context4K
Smallest3.8 GB
MI
openhermes 2.5 mistral 7b Q4 0 4 8
Mistral · flyfishstudio
Params7.24B
Context32K
Smallest4.1 GB
LL
Meta Llama 3.1 8B Instruct
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
RO
rocket 3b Q4 0 4 8
Rocket · flyfishstudio
Params2.8B
Context4K
Smallest1.6 GB
MI
Mistral 7B Instruct v0.3 Q4 0 4 8
Mistral · flyfishstudio
Params7.25B
Context32K
Smallest4.1 GB
MI
Mistral 7B Instruct v0.3 Q4 0 4 4
Mistral · flyfishstudio
Params7.25B
Context32K
Smallest4.1 GB
LL
Meta Llama 3.1 8B Instruct
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB