GGUF models by win10
97 GGUF models quantized by win10 on Hugging Face. Sizes run from 21M to 40.6B parameters. The longest trained context is 1024K tokens.
CH
ChatML Nemo Pro V2 Q8 0
Chatml · win10
Params12.2B
Context1000K
Smallest13.0 GB
QW
Qwerky QwQ 32B
Qwen · win10
Params34.9B
Context1024K
Smallest26.4 GB
ST
Steiner 32b Preview
Steiner · win10
Params32.8B
Context32K
Q4_K_M19.8 GB
PH
phi 3.5 sakura yuzu v2.5 Q8 0
Phi · win10
Params3.82B
Context128K
Smallest4.1 GB
AR
ArliAI RPMax v1.3 merge 13.3B Q8 0
Arliai · win10
Params13.3B
Context128K
Smallest14.1 GB
TE
Tempesthenno Ppo Ckpt40
Tempesthenno · win10
Params14.8B
Context128K
Smallest15.7 GB
QW
EVA Instruct QwQ 32B Preview Q4 K M
Qwen · win10
Params32.8B
Context128K
Q4_K_M19.9 GB
WE
Weirdslerp2 25B Q5 K M
Weirdslerp · win10
Params25.2B
Context128K
Smallest17.9 GB
TA
taide meta it 16b
Taide · win10
Params8.03B
Context8K
Smallest6.6 GB
CH
ChatML Nemo Pro
Chatml · win10
Params12.2B
Context1000K
Smallest13.0 GB
NE
NeMoria 21b Q8 0
Nemoria · win10
Params20.4B
Context1000K
Smallest21.7 GB
LL
llama3-13.45b-Instruct
Llama · win10
Params13.3B
Context8K
Smallest14.1 GB
ME
mergekit model stock injkqri Q8 0
Mergekit · win10
Params14.8B
Context986K
Smallest15.7 GB
MI
Mistral Nemo Instruct 2407
Mistral · win10
Params12.2B
Context128K
Q4_K_M7.5 GB
HI
high speed mixing 7B V2 Q8 0
High · win10
Params7.62B
Context32K
Smallest8.1 GB
GE
RYS Gemma 2 27b It
Gemma · win10
Params32.9B
Context8K
Smallest27.0 GB
PH
phi 3.5 sakura yuzu v2 Q8 0
Phi · win10
Params4.84B
Context4K
Smallest5.1 GB
LL
llama3-13.45b-Instruct
Llama · win10
Params13.3B
Context8K
Smallest9.4 GB
IN
Infinirc ArliAI RPMax v1.3 merge 13.3B Q8 0
Infinirc · win10
Params13.3B
Context128K
Smallest14.1 GB
NE
Openhands Nemotron 32B Karcher 300
Nemotron · win10
Params32.8B
Context32K
Q4_K_M19.9 GB
SP
SphinxMind 14B Q8 0
Sphinxmind · win10
Params14.8B
Context64K
Smallest15.7 GB
MI
Mistral Rp 24b Karcher
Mistral · win10
Params23.6B
Context32K
Smallest19.3 GB
PH
phi3.5 mini 24 09 30 Q8 0
Phi · win10
Params3.82B
Context128K
Smallest4.1 GB
IN
InternLM2_5 20B ArliAI RPMax v1.1
InternLM · win10
Params19.9B
Context32K
Smallest16.3 GB
PH
Phi 3.5 Mini Instruct
Phi · win10
Params3.82B
Context128K
Smallest4.1 GB
MI
Mistral Nemo Instruct 2407
Mistral · win10
Params22.6B
Context1000K
Smallest16.0 GB
CH
ChatML Nemo Pro Q8 0
Chatml · win10
Params12.2B
Context1000K
Smallest13.0 GB
ME
Mergekit Karcher Pifptpx
Mergekit · win10
Params8.03B
Context128K
Smallest8.5 GB
WI
wizardcoder-33b-v1.1-mirror
Wizardcoder · win10
Params33.3B
Context16K
Smallest12.4 GB
QW
QwQ 32B Preview
Qwen · win10
Params32.8B
Context32K
Q4_K_M19.9 GB