GGUF models by flyfishstudio

42 GGUF models quantized by flyfishstudio on Hugging Face. Sizes run from 494M to 9.24B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 3B Instruct Q4 0 4 4
Qwen · flyfishstudio
Params3.09B
Context32K
Smallest1.8 GB
Tool use
QW
Qwen2.5 3B Instruct Q4 0 4 8
Qwen · flyfishstudio
Params3.09B
Context32K
Smallest1.8 GB
Tool use
PH
Phi 3.5 Mini instruct_Uncensored
Phi · flyfishstudio
Params3.82B
Context128K
Smallest2.2 GB
GGUF
GE
Gemma 2 2b It
Gemma · flyfishstudio
Params2.61B
Context8K
Smallest1.6 GB
GGUF
QW
qwen2-0_5b-instruct
Qwen · flyfishstudio
Params494M
Context32K
Smallest0.4 GB
GGUF
GE
Gemma 2 9b It
Gemma · flyfishstudio
Params9.24B
Context8K
Smallest5.4 GB
GGUF
QW
qwen2-0_5b-instruct
Qwen · flyfishstudio
Params494M
Context32K
Smallest0.4 GB
GGUF
GE
Gemma 2 2b It
Gemma · flyfishstudio
Params2.61B
Context8K
Smallest1.6 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · flyfishstudio
Params3.82B
Context128K
Smallest2.2 GB
GGUF
GE
Gemma 2 9b It
Gemma · flyfishstudio
Params9.24B
Context8K
Smallest5.4 GB
GGUF
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
GGUF
LL
yarn llama 2 7b 128k Q4 0 4 4
Llama · flyfishstudio
Params6.74B
Context128K
Smallest3.8 GB
GGUF
LL
Meta Llama 3.1 8B Instruct Abliterated
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
GGUF
PH
Phi 3.1 mini 4k instruct Q4 0 4 4
Phi · flyfishstudio
Params3.82B
Context4K
Smallest2.2 GB
GGUF
GE
gemma 2 2b it v2 Q4 0 4 4
Gemma · flyfishstudio
Params2.61B
Context8K
Smallest1.6 GB
GGUF
PH
Phi 3.5 Mini instruct_Uncensored
Phi · flyfishstudio
Params3.82B
Context128K
Smallest2.2 GB
GGUF
RO
rocket 3b Q4 0 4 4
Rocket · flyfishstudio
Params2.8B
Context4K
Smallest1.6 GB
GGUF
LL
Hermes 3 Llama 3.1 8B
Llama · flyfishstudio
Params8.03B
Context128K
Smallest4.7 GB
Tool use
OR
orca mini v3 7b Q4 0 4 4
Orca · flyfishstudio
Params6.74B
Context4K
Smallest3.8 GB
GGUF
GE
Gemma 2 Ataraxy 9B Q4 0 4 4
Gemma · flyfishstudio
Params9.24B
Context8K
Smallest5.4 GB
GGUF
GE
gemma 2 2b it v2 Q4 0 4 8
Gemma · flyfishstudio
Params2.61B
Context8K
Smallest1.6 GB
GGUF
PH
Phi 3.1 mini 4k instruct Q4 0 4 8
Phi · flyfishstudio
Params3.82B
Context4K
Smallest2.2 GB
GGUF
GE
Gemma 2 Ataraxy 9B Q4 0 4 8
Gemma · flyfishstudio
Params9.24B
Context8K
Smallest5.4 GB
GGUF
MI
openhermes 2.5 mistral 7b Q4 0 4 4
Mistral · flyfishstudio
Params7.24B
Context32K
Smallest4.1 GB
Tool use
TI
tinyllama 1.1b chat v1.0 Q4 0 4 4
Tinyllama · flyfishstudio
Params1.1B
Context2K
Smallest0.6 GB
GGUF
OR
orca mini v3 7b Q4 0 4 8
Orca · flyfishstudio
Params6.74B
Context4K
Smallest3.8 GB
GGUF
QW
Qwen1.5 1.8B Layla v4
Qwen · flyfishstudio
Params1.84B
Context32K
Smallest1.1 GB
GGUF
LL
llama 2 7b chat Q4 0 4 8
Llama · flyfishstudio
Params6.74B
Context4K
Smallest3.8 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · flyfishstudio
Params3.82B
Context128K
Smallest2.2 GB
GGUF
QW
Qwen1.5 1.8B Layla v4
Qwen · flyfishstudio
Params1.84B
Context32K
Smallest1.1 GB
GGUF

About flyfishstudio on Hugging Face

flyfishstudio has published 42 GGUF models that ggufmodels.com tracks, across families such as Llama, Gemma and Qwen. The most downloaded are Qwen2.5 3B Instruct Q4 0 4 4, Qwen2.5 3B Instruct Q4 0 4 8 and Phi 3.5 Mini instruct_Uncensored.

The quants flyfishstudio publishes most often are Q4_0 (42). Parameter counts run from 494M to 9.24B, and the longest trained context window is 128K tokens.

Frequently asked questions

What GGUF models has flyfishstudio published?

42 models are listed here, mainly from the Llama, Gemma and Qwen families. The most downloaded is Qwen2.5 3B Instruct Q4 0 4 4.

Which quants does flyfishstudio publish?

The most common are Q4_0 (42 models).

Keep browsing