GGUF models by ijohn07
106 GGUF models quantized by ijohn07 on Hugging Face. Sizes run from 773M to 34.7B parameters. The longest trained context is 1024K tokens.
QW
Qwen2.5-1.5B-Open-R1-Distill
Qwen · ijohn07
Params1.54B
Context32K
Smallest1.3 GB
GE
gemma-3-4b-it-codeforces-SFT
Gemma · ijohn07
Params3.88B
Context128K
Q4_K_M2.5 GB
MI
Mistral NeMo Minitron 8B Base
Mistral · ijohn07
Params8.41B
Context8K
Smallest3.3 GB
GE
GELab Zero 4B Preview Sico Evolution
Gelab · ijohn07
Params4.02B
Context256K
Smallest4.3 GB
TM
Tmax 9b
Tmax · ijohn07
Params8.95B
Context256K
Smallest9.5 GB
SM
SmolLM3 3B
SmolLM · ijohn07
Params3.08B
Context64K
Smallest3.3 GB
DE
DeepSeek V2 Lite Chat Uncensored Unbiased Reasoner
DeepSeek · ijohn07
Params15.7B
Context4K
Smallest16.7 GB
IN
Internlm2_5 7b Chat
InternLM · ijohn07
Params7.74B
Context32K
Smallest3.0 GB
AG
Agents A1
Agents · ijohn07
Params34.7B
Context256K
Q4_K_M21.2 GB
MI
NeuralHyperion 2.0 Mistral 7B Q8 0
Mistral · ijohn07
Params7.24B
Context32K
Smallest7.7 GB
DA
H2O Danube3.1 4b Chat
Danube · ijohn07
Params3.96B
Context8K
Smallest3.3 GB
MI
Hyperion 2.1 Mistral 7B Q8 0
Mistral · ijohn07
Params7.24B
Context32K
Smallest7.7 GB
NE
neural chat mini v2.2 1.8B Q8 0
Neural · ijohn07
Params1.84B
Context32K
Smallest2.0 GB
MI
Mistral Small Instruct 2409
Mistral · ijohn07
Params22.2B
Context128K
Smallest8.3 GB
GE
Gemma 2 9b
Gemma · ijohn07
Params9.24B
Context8K
Q4_K_M5.8 GB
MI
Ministral 8B Instruct 2410 HF Q8 0
Mistral · ijohn07
Params8.02B
Context32K
Smallest8.5 GB
MA
MagpieLM 8B Chat v0.1 Q4 K M
Magpielm · ijohn07
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Ministral 8B Instruct 2410
Mistral · ijohn07
Params8.02B
Context32K
Smallest8.5 GB
MA
MagpieLM 4B SFT v0.1 Q6 K
Magpielm · ijohn07
Params4.51B
Context128K
Smallest3.7 GB
AG
Agents A1
Agents · ijohn07
Params34.7B
Context256K
Smallest12.9 GB
OL
OLMo 7B 0724 Instruct Hf
OLMo · ijohn07
Params6.89B
Context4K
Q4_K_M4.2 GB
QE
QED Nano
Qed · ijohn07
Params4.02B
Context256K
Smallest3.3 GB
MI
Hyperion 3.0 Mistral 7B DPO Q8 0
Mistral · ijohn07
Params7.24B
Context32K
Smallest7.7 GB
DE
DeepSeek V2 Lite Chat
DeepSeek · ijohn07
Params15.7B
Context160K
Smallest14.1 GB
GE
gemma-3-12b-it-codeforces-SFT
Gemma · ijohn07
Params11.8B
Context128K
Smallest12.5 GB
QW
Hercules 5.0 Qwen2 7B
Qwen · ijohn07
Params7.62B
Context128K
Smallest8.1 GB
HO
Holo 3.1 0.8B
Holo · ijohn07
Params1.01B
Context256K
Smallest1.1 GB
LL
Llama 3 Hercules 5.0 8B
Llama · ijohn07
Params8.03B
Context8K
Smallest8.5 GB
HO
Holo 3.1 4B
Holo · ijohn07
Params4.84B
Context256K
Smallest4.0 GB
AR
arco plus Q8 0
Arco · ijohn07
Params773M
Context8K
Smallest0.8 GB