GGUF models by flyfishstudio

42 GGUF models quantized by flyfishstudio on Hugging Face. Sizes run from 494M to 9.24B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
tinyllama 1.1b chat v1.0 Q4 0 4 8
Tinyllama · flyfishstudio
Params1.1B
Context2K
Smallest0.6 GB
GGUF
SM
SmolLM 1.7B
SmolLM · flyfishstudio
Params1.71B
Context2K
Smallest1.0 GB
GGUF
LL
yarn llama 2 7b 128k Q4 0 4 8
Llama · flyfishstudio
Params6.74B
Context128K
Smallest3.8 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
Tool use
SM
SmolLM 1.7B
SmolLM · flyfishstudio
Params1.71B
Context2K
Smallest1.0 GB
GGUF
LL
llama 2 7b chat Q4 0 4 4
Llama · flyfishstudio
Params6.74B
Context4K
Smallest3.8 GB
GGUF
MI
openhermes 2.5 mistral 7b Q4 0 4 8
Mistral · flyfishstudio
Params7.24B
Context32K
Smallest4.1 GB
Tool use
LL
Meta Llama 3.1 8B Instruct
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
GGUF
RO
rocket 3b Q4 0 4 8
Rocket · flyfishstudio
Params2.8B
Context4K
Smallest1.6 GB
GGUF
MI
Mistral 7B Instruct v0.3 Q4 0 4 8
Mistral · flyfishstudio
Params7.25B
Context32K
Smallest4.1 GB
GGUF
MI
Mistral 7B Instruct v0.3 Q4 0 4 4
Mistral · flyfishstudio
Params7.25B
Context32K
Smallest4.1 GB
GGUF
LL
Meta Llama 3.1 8B Instruct
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
GGUF

Keep browsing