GGUF models by ijohn07
106 GGUF models quantized by ijohn07 on Hugging Face. Sizes run from 773M to 34.7B parameters. The longest trained context is 1024K tokens.
GE
Gemma 2 2b
Gemma · ijohn07
Params2.61B
Context8K
Smallest2.8 GB
MA
MagpieLM 4B Chat v0.1 Q6 K
Magpielm · ijohn07
Params4.51B
Context128K
Smallest3.7 GB
CH
Chocolatine 14B Instruct DPO v1.2 Q8 0
Chocolatine · ijohn07
Params14B
Context4K
Smallest14.8 GB
DE
DeepSeek Coder V2 Lite Base
DeepSeek · ijohn07
Params15.7B
Context160K
Smallest16.7 GB
MI
Mistral Nemo Instruct 2407
Mistral · ijohn07
Params12.2B
Context128K
Smallest10.1 GB
QW
Qwen2.5-1.5B-Open-R1-Distill
Qwen · ijohn07
Params1.54B
Context32K
Smallest1.6 GB
OP
OpenCerebrum 2.0 7B Q8 0
Opencerebrum · ijohn07
Params7.24B
Context32K
Smallest7.7 GB
MI
Mistral Nemo Instruct 2407
Mistral · ijohn07
Params12.2B
Context1000K
Smallest13.0 GB
DA
H2O Danube2 1.8b Chat
Danube · ijohn07
Params1.83B
Context8K
Smallest1.9 GB
MI
Codestral 22B v0.1
Mistral · ijohn07
Params22.2B
Context32K
Smallest23.6 GB
YI
Yi 1.5 6B Chat
Yi · ijohn07
Params6.06B
Context4K
Smallest6.4 GB
MA
MagpieLM 4B SFT v0.1 Q8 0
Magpielm · ijohn07
Params4.51B
Context128K
Smallest4.8 GB
MI
Mistral Nemo Instruct 2407
Mistral · ijohn07
Params12.2B
Context1000K
Smallest4.8 GB
LL
Apollo 0.4 Llama 3.1 8B
Llama · ijohn07
Params8.03B
Context128K
Smallest8.5 GB
MI
Hercules 3.1 Mistral 7B Q8 0
Mistral · ijohn07
Params7.24B
Context32K
Smallest7.7 GB
AY
Aya 23 8B
Aya · ijohn07
Params8.03B
Context8K
Smallest3.4 GB