GGUF models by second-state

291 GGUF models quantized by second-state on Hugging Face. Sizes run from 23M to 236B parameters. The longest trained context is 10240K tokens.

Most downloadedNewestLargestLongest contextSmallest
YI
Yi 34B Chat
Yi · second-state
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
QW
Qwen3 4B
Qwen · second-state
Params4.02B
Context40K
Q4_K_M2.5 GB
ReasoningTool use
YI
Yi-1.5-9B-Chat
Yi · second-state
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen2-72B-Instruct
Qwen · second-state
Params72.7B
Context32K
Q4_K_M47.4 GB
GGUF
MI
Codestral-22B-v0.1-hf
Mistral · second-state
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
SM
SmolLM2 1.7B Instruct
SmolLM · second-state
Params1.71B
Context8K
Q4_K_M1.1 GB
GGUF
YI
Yi-1.5-6B-Chat
Yi · second-state
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
OU
OuteTTS 0.2 500M
Outetts · second-state
Params499M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 2 7B Chat
Llama · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama 3.3 70B Instruct
Llama · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
GE
Gemma 3 270m It
Gemma · second-state
Params268M
Context32K
Q4_K_M0.3 GB
GGUF
QW
Qwen2-1.5B-Instruct
Qwen · second-state
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
MI
Mistral Small 3.1 24B Instruct 2503
Mistral · second-state
Params23.6B
Context128K
Q4_K_M14.3 GB
Vision
QW
Qwen2-7B-Instruct
Qwen · second-state
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
Llama-3-8B-Instruct-Gradient-1048k
Llama · second-state
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
LL
Reflection Llama 3.1 70B
Llama · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
NE
NeuralBeagle14 7B
Neuralbeagle · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
NE
Nemotron Mini 4B Instruct
Nemotron · second-state
Params4.19B
Context4K
Q4_K_M2.7 GB
Tool use
EX
EXAONE 3.5 32B Instruct
Exaone · second-state
Params32B
Context32K
Q4_K_M19.3 GB
Coding
OC
Octopus-v2
Octopus · second-state
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
FU
functionary small v3.2
Functionary · second-state
Params8.03B
Context128K
Q4_K_M4.9 GB
CodingTool use
IN
Internlm2_5 20b Chat
InternLM · second-state
Params19.9B
Context32K
Q4_K_M12.0 GB
Coding
LL
Llama 3 Taiwan 70B Instruct Rc3
Llama · second-state
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
MI
MiniCPM V 4
MiniCPM · second-state
Params3.61B
Context32K
Q4_K_M2.2 GB
CodingVision
PH
Phi 3.5 MoE Instruct
Phi · second-state
Params41.9B
Context128K
Q4_K_M25.3 GB
Coding
YI
Yi-1.5-34B-Chat
Yi · second-state
Params34.4B
Context4K
Q4_K_M20.7 GB
GGUF
EX
EXAONE Deep 32B
Exaone · second-state
Params32B
Context32K
Q4_K_M19.3 GB
Coding
YI
Yi 34Bx2 MoE 60B
Yi · second-state
Params60.8B
Context195K
Q4_K_M36.7 GB
GGUF
QW
Qwen3 14B
Qwen · second-state
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
EM
Embeddinggemma 300m
Embeddinggemma · second-state
Params303M
Context2K
Q4_K_M0.2 GB
Embedding

Keep browsing