GGUF models by Chungulus
34 GGUF models quantized by Chungulus on Hugging Face. Sizes run from 4.21B to 64.8B parameters. The longest trained context is 2048K tokens.
AG
Agents A1 Q4 K M imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Q4_K_M21.2 GB
AG
Agents A1 Q3 K M imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Smallest16.8 GB
AG
Agents A1 IQ3 M imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Smallest15.4 GB
AG
Agents A1 Q2 K imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Smallest12.9 GB
AG
Agents A1 Q5 K M imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Smallest24.7 GB
AG
Agents A1 Q6 K imatrix gguf fable5 calibrated
Agents · Chungulus
Params34.7B
Context256K
Smallest28.5 GB