GGUF models by second-state
291 GGUF models quantized by second-state on Hugging Face. Sizes run from 23M to 236B parameters. The longest trained context is 10240K tokens.
DE
DeepSeek R1 Distill Qwen 14B
DeepSeek · second-state
Params14.8B
Context128K
Q4_K_M9.0 GB
LL
Llava v1.5 7B
LLaVA · second-state
Params6.74B
Context4K
Q4_K_M4.1 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · second-state
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
Dolphin 2.7 mixtral 8x7b
Mistral · second-state
Params46.7B
Context32K
Q4_K_M26.4 GB
MI
Mistral 7B Instruct v0.2
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
OrionStar Yi 34B Chat Llama
Llama · second-state
Params34.4B
Context4K
Q4_K_M20.7 GB
QW
Qwen2.5 0.5B Instruct
Qwen · second-state
Params494M
Context32K
Q4_K_M0.4 GB
MI
Mixtral 8x7B Instruct v0.1
Mistral · second-state
Params46.7B
Context32K
Q4_K_M28.4 GB
SM
SmolVLM2 2.2B Instruct
Smolvlm · second-state
Params1.81B
Context8K
Q4_K_M1.1 GB
MI
dolphin 2.6 mistral 7B
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 3 1b It
Gemma · second-state
Params1B
Context32K
Q4_K_M0.8 GB
MI
Mistral Large Instruct 2407
Mistral · second-state
Params123B
Context128K
Q4_K_M73.2 GB
LL
CodeLlama 13B Instruct
Llama · second-state
Params13B
Context16K
Q4_K_M7.9 GB
MI
Mistral Nemo Instruct 2407
Mistral · second-state
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
Nous Hermes 2 Mixtral 8x7B SFT
Mistral · second-state
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
Meta Llama 3 70B Instruct
Llama · second-state
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
MiniCPM V 2 6
MiniCPM · second-state
Params7.61B
Context32K
Q4_K_M4.7 GB
JI
jina embeddings v2 base es
Jina · second-state
Params160M
Context8K
Q4_K_M0.1 GB
QW
Qwen1.5-110B-Chat
Qwen · second-state
Params111B
Context32K
Q4_K_M67.2 GB
ST
starcoder2-15b-instruct-v0.1
StarCoder · second-state
Params16B
Context16K
Q4_K_M9.9 GB
QW
Qwen1.5-14B-Chat
Qwen · second-state
Params14.2B
Context32K
Q4_K_M9.2 GB
QW
CodeQwen1.5-7B-Chat
Qwen · second-state
Params7.25B
Context64K
Q4_K_M4.7 GB
DE
DeepSeek-V2-Lite-Chat
DeepSeek · second-state
Params15.7B
Context160K
Q4_K_M10.4 GB
PH
Phi 3 medium 128k instruct
Phi · second-state
Params14B
Context128K
Q4_K_M8.6 GB
QW
Qwen1.5-0.5B-Chat
Qwen · second-state
Params620M
Context32K
Q4_K_M0.4 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · second-state
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma 3 4b It
Gemma · second-state
Params3.88B
Context128K
Q4_K_M2.5 GB
LL
Llama 3.1 8B Instruct
Llama · second-state
Params8B
Context128K
Q4_K_M4.9 GB
LL
Llama3.1 8B Chinese Chat
Llama · second-state
Params8B
Context128K
Q4_K_M4.9 GB
LL
FinGPT MT Llama 3 8B LoRA
Llama · second-state
Params8B
Context8K
Q4_K_M4.9 GB