GGUF models by richardyoung
35 GGUF models quantized by richardyoung on Hugging Face. Sizes run from 596M to 480B parameters. The longest trained context is 256K tokens.
QW
Qwen2.5 7B Instruct Heretic1X General
Qwen · richardyoung
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen2.5 Coder 14B Instruct Heretic1X Code
Qwen · richardyoung
Params14.8B
Context32K
Q4_K_M9.0 GB
QW
Qwen2.5 7B Instruct SOM Heretic
Qwen · richardyoung
Params7.62B
Context32K
Q4_K_M4.7 GB
SM
SmolVLM2 2.2B Instruct
Smolvlm · richardyoung
Params1.81B
Context8K
Q4_K_M1.1 GB
QW
Qwen2.5 Coder 7B Instruct Heretic1X Code
Qwen · richardyoung
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
Mistral Small 3.2 24B Instruct 2506 MPOA Heretic
Mistral · richardyoung
Params23.6B
Context128K
Q4_K_M14.3 GB
QW
Qwen2.5 7B Instruct ARA Heretic
Qwen · richardyoung
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Qwen3.8 27B SOM Heretic
Qwen · richardyoung
Params26.9B
Context256K
Q4_K_M16.5 GB
MI
Mistral Small Instruct 2409 DiM Heretic
Mistral · richardyoung
Params22.2B
Context32K
Q4_K_M13.3 GB
MI
Mistral Small Instruct 2409 MPOA Heretic
Mistral · richardyoung
Params22.2B
Context32K
Q4_K_M13.3 GB
MI
Mistral Small Instruct 2409 ARA Heretic
Mistral · richardyoung
Params22.2B
Context32K
Q4_K_M13.3 GB
QW
Qwen2.5 32B Instruct SOM Heretic
Qwen · richardyoung
Params32.8B
Context32K
Q4_K_M19.9 GB
UI
Uigen X 30b Moe
Uigen · richardyoung
Params30.5B
Context256K
Q4_K_M18.6 GB
OL
Olmo 3.1 32B Instruct SOM Heretic
OLMo · richardyoung
Params32.2B
Context64K
Q4_K_M19.5 GB
QW
Qwen3 Coder 480B A35B Instruct
Qwen · richardyoung
Params480B
Context256K
Smallest132.9 GB
KA
KAT Dev 72B Exp
Kat · richardyoung
Params72.7B
Context128K
Smallest25.5 GB
BF
Bfs Prover v2 32b
Bfs · richardyoung
Params32.8B
Context128K
Q4_K_M19.9 GB
LL
llama medx v32
Llama · richardyoung
Params3.21B
Context128K
Smallest2.0 GB
LL
openbiollm
Llama · richardyoung
Params8.03B
Context8K
Smallest8.5 GB