GGUF models by second-state
291 GGUF models quantized by second-state on Hugging Face. Sizes run from 23M to 236B parameters. The longest trained context is 10240K tokens.
LL
Llama 3.2 3B Instruct
Llama · second-state
Params3.61B
Context128K
Q4_K_M2.2 GB
DE
DeepSeek R1 Distill Llama 8B
DeepSeek · second-state
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 3 12b It
Gemma · second-state
Params11.8B
Context128K
Q4_K_M7.3 GB
JI
jina embeddings v2 base en
Jina · second-state
Params137M
Context8K
Q4_K_M0.1 GB
QW
Qwen2.5 Math 1.5B Instruct
Qwen · second-state
Params1.54B
Context4K
Q4_K_M1.0 GB
GE
gemma-7b-it
Gemma · second-state
Params8.54B
Context8K
Q4_K_M5.1 GB
CO
Yi Coder 1.5B Chat
Coder · second-state
Params1.48B
Context128K
Q4_K_M1.0 GB
GE
gemma-2b-it
Gemma · second-state
Params2.51B
Context8K
Q4_K_M1.5 GB
QW
Qwen2.5 Coder 32B Instruct
Qwen · second-state
Params32.8B
Context32K
Q4_K_M19.9 GB
WI
WizardCoder Python 7B v1.0
Wizardcoder · second-state
Params6.74B
Context16K
Q4_K_M4.1 GB
DE
DeepSeek Coder V2 Instruct
DeepSeek · second-state
Params236B
Context160K
Q4_K_M142.5 GB
MI
Nous Hermes 2 Mixtral 8x7B DPO
Mistral · second-state
Params46.7B
Context32K
Q4_K_M28.4 GB
IN
InternLM2
InternLM · second-state
Params7.74B
Context32K
Q4_K_M4.7 GB
MI
Mistral-7B-Instruct-v0.3
Mistral · second-state
Params7.25B
Context32K
Q4_K_M4.4 GB
SM
SmolLM 135M Instruct
SmolLM · second-state
Params135M
Context2K
Q4_K_M0.1 GB
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · second-state
Params7.62B
Context128K
Q4_K_M4.7 GB
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen1.5-7B-Chat
Qwen · second-state
Params7.72B
Context32K
Q4_K_M4.8 GB
QW
Qwen3 0.6B
Qwen · second-state
Params752M
Context40K
Q4_K_M0.5 GB
YI
Dolphin 2.2 Yi 34B
Yi · second-state
Params34.4B
Context16K
Q4_K_M20.7 GB
QW
Qwen3 32B
Qwen · second-state
Params32.8B
Context40K
Q4_K_M19.8 GB
LL
Infinity Instruct 7M Gen Llama3_1 8B
Llama · second-state
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 Math 7B Instruct
Qwen · second-state
Params7.62B
Context4K
Q4_K_M4.7 GB
CO
Yi Coder 9B Chat
Coder · second-state
Params8.83B
Context128K
Q4_K_M5.3 GB
LL
Llama 4 Scout 17B 16E Instruct
Llama · second-state
Params108B
Context10240K
Q4_K_M65.4 GB
CO
C4Ai Command R Plus 08 2024
Command R · second-state
Params104B
Context128K
Q4_K_M62.8 GB
LL
Llama 3 Groq 70B Tool Use
Llama · second-state
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
MiniCPM Llama3 V 2 5
Llama · second-state
Params8.03B
Context8K
Q4_K_M4.9 GB
FU
functionary medium v3.1
Functionary · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
ME
Medgemma 27b Text It
Medgemma · second-state
Params27B
Context128K
Q4_K_M16.5 GB