GGUF models by sheldonrobinson
37 GGUF models quantized by sheldonrobinson on Hugging Face. Sizes run from 23M to 41.9B parameters. The longest trained context is 128K tokens.
LL
Llama 3.2 3B Instruct
Llama · sheldonrobinson
Params3.21B
Context128K
Smallest3.4 GB
PH
Phi 3.5 MoE Instruct
Phi · sheldonrobinson
Params41.9B
Context128K
Q4_K_M25.3 GB
LL
Llama 3.1 8B Instruct
Llama · sheldonrobinson
Params8.03B
Context128K
Smallest8.5 GB
TU
Turn Detector
Turn · sheldonrobinson
Params135M
Context8K
Q4_K_M0.1 GB
GE
Gemma 2 9b
Gemma · sheldonrobinson
Params9.24B
Context8K
Smallest9.8 GB
GE
Gemma 2 9b
Gemma · sheldonrobinson
Params9.24B
Context8K
Q4_K_M5.8 GB
AL
all MiniLM L12 v2 Q4 K M
All · sheldonrobinson
Params33M
Context512
Q4_K_M0.0 GB