GGUF models by win10
97 GGUF models quantized by win10 on Hugging Face. Sizes run from 21M to 40.6B parameters. The longest trained context is 1024K tokens.
MI
Mistral RP 24b Karcher Pro
Mistral · win10
Params23.6B
Context128K
Q4_K_M14.3 GB
AY
Aya Expanse 32b
Aya · win10
Params32.3B
Context8K
Smallest23.1 GB
QW
Qwen2.5 Coder 7B
Qwen · win10
Params7.62B
Context32K
Smallest8.1 GB
QW
Qwerky QwQ 32B
Qwen · win10
Params34.9B
Context1024K
Q4_K_M22.8 GB
QW
Qwen2.5 7B
Qwen · win10
Params7.62B
Context128K
Smallest8.1 GB
DE
DeepSeek R1 Distill sthenno 14b 0121 union tokenizer Q8 0
DeepSeek · win10
Params14.8B
Context128K
Smallest15.7 GB
KA
Karcher Max Iter1000 32b
Karcher · win10
Params32.8B
Context32K
Q4_K_M19.8 GB
GR
Granite 3.0 8b Instruct
Granite · win10
Params8.17B
Context4K
Smallest8.7 GB
FA
falcon mamba 7b instruct Q8 0
Falcon · win10
Params7.27B
Context1024K
Smallest7.8 GB
LL
llama3-13.45b-Instruct
Llama · win10
Params13.3B
Context8K
Q4_K_M8.1 GB
BE
35b-beta-long
Beta · win10
Params35B
Context128K
Q4_K_M21.5 GB
BL
Blue Rose Coder 12.3B Instruct Q8 0
Blue · win10
Params12.3B
Context32K
Smallest13.1 GB
PH
Phi 3.5 24 10 06 Q8 0
Phi · win10
Params3.82B
Context128K
Smallest4.1 GB
PH
Phi 3.5 Sakura Yuzu v1.5
Phi · win10
Params7.22B
Context4K
Smallest7.7 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · win10
Params7.62B
Context32K
Smallest8.1 GB
AR
ArliAI RPMax v1.3 merge 8B Q8 0
Arliai · win10
Params8.03B
Context128K
Smallest8.5 GB
LL
Meta Llama 3 12B Instruct Q6 K
Llama · win10
Params11.5B
Context8K
Smallest9.5 GB
AY
Aya Expanse 32b
Aya · win10
Params32.3B
Context8K
Q4_K_M19.8 GB
QW
WhiteRabbitNeo 2.5 Qwen 2.5 Coder 12.3B Q8 0
Qwen · win10
Params12.3B
Context32K
Smallest13.1 GB
HI
high speed mixing 7B V1 Q8 0
High · win10
Params7.62B
Context128K
Smallest8.1 GB
ME
mergekit sce vjeombg Q8 0
Mergekit · win10
Params14.8B
Context128K
Smallest15.7 GB
BR
Breeze-13B-32k-Instruct-v1_0
Breeze · win10
Params12.7B
Context32K
Smallest13.5 GB
SH
shuttle 3 mini Q8 0
Shuttle · win10
Params14.8B
Context64K
Smallest15.7 GB
PH
phi 3.5 sakura yuzu v3.0 Q8 0
Phi · win10
Params3.82B
Context128K
Smallest4.1 GB
DE
DeepSeek V2 Lite XiaAi
DeepSeek · win10
Params15.7B
Context160K
Smallest16.7 GB
EV
EVA Meissa Mini Pro
Eva · win10
Params14.8B
Context64K
Smallest15.7 GB
ST
sthenno Test maybe is pro v2 Q8 0
Sthenno · win10
Params14.8B
Context128K
Smallest15.7 GB
NE
Openhands Nemotron 32B Karcher
Nemotron · win10
Params32.8B
Context32K
Q4_K_M19.9 GB
IN
ArliAI RPMax v1.3 Merge 8B
Infinirc · win10
Params8.03B
Context128K
Smallest8.5 GB
QW
Norns Qwen2.5 7B Q8 0
Qwen · win10
Params7.62B
Context32K
Smallest8.1 GB