GGUF models by second-state
291 GGUF models quantized by second-state on Hugging Face. Sizes run from 23M to 236B parameters. The longest trained context is 10240K tokens.
DE
DeepSeek R1 Distill Llama 70B
DeepSeek · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
MI
mathstral-7B-v0.1
Mistral · second-state
Params7.25B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 Nemotron 70B Instruct HF
Llama · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
GL
glm-4-9b-chat-1m
GLM · second-state
Params9.48B
Context1024K
Q4_K_M6.3 GB
LL
Meta-Llama-3-8B-Instruct
Llama · second-state
Params8.03B
Context8K
Q4_K_M4.9 GB
PA
Palmyra Fin 70B 32K
Palmyra · second-state
Params70.6B
Context32K
Q4_K_M42.5 GB
LL
Llama 3.2 Korean Bllossom 3B
Llama · second-state
Params3.21B
Context128K
Q4_K_M2.0 GB
QW
Qwen2 VL 72B Instruct
Qwen · second-state
Params72.7B
Context32K
Q4_K_M47.4 GB
ME
Megrez 3B Instruct
Megrez · second-state
Params2.92B
Context32K
Q4_K_M1.8 GB
ST
stable diffusion 3 medium
Stable · second-state
Params7.88B
Context—
Smallest4.6 GB
BI
Bielik 11B v2.3 Instruct
Bielik · second-state
Params11.2B
Context32K
Q4_K_M6.7 GB
LL
Llama 3.1 Nemotron 70B Reward HF
Llama · second-state
Params70.6B
Context128K
Q4_K_M42.5 GB
OU
OuteTTS 0.3 500M
Outetts · second-state
Params499M
Context32K
Q4_K_M0.4 GB
ZE
Zephyr 7B Beta
Zephyr · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
DeepSeek V2.5
DeepSeek · second-state
Params236B
Context160K
Q4_K_M142.5 GB
LL
llm-compiler-7b-ftd
Llm · second-state
Params6.74B
Context16K
Q4_K_M4.1 GB
EX
EXAONE 3.5 2.4B Instruct
Exaone · second-state
Params2.41B
Context32K
Q4_K_M1.5 GB
NE
Neural Chat 7B v3 1
Neural · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama-3-8B-Japanese-Instruct
Llama · second-state
Params8.03B
Context8K
Q4_K_M4.9 GB
MO
Moxin Instruct 7b
Moxin · second-state
Params8.11B
Context32K
Q4_K_M4.9 GB
GE
gemma-1.1-2b-it
Gemma · second-state
Params2.51B
Context8K
Q4_K_M1.6 GB
IN
Internlm3 8b Instruct
InternLM · second-state
Params8.8B
Context32K
Q4_K_M5.4 GB
XL
xLAM 8x22b R
Xlam · second-state
Params141B
Context64K
Q4_K_M85.6 GB
MI
Mistral Small 24B Instruct 2501
Mistral · second-state
Params23.6B
Context32K
Q4_K_M14.3 GB
MI
MiniCPM o 2 6
MiniCPM · second-state
Params7.61B
Context32K
Q4_K_M4.7 GB
FL
FLUX.1 Canny dev
Flux · second-state
Params11.9B
Context—
Smallest4.2 GB
MI
MistralLite 7B
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
JI
jina embeddings v2 small en
Jina · second-state
Params32M
Context8K
Q4_K_M0.0 GB
ST
stablelm-2-12b-chat
Stablelm · second-state
Params12.1B
Context4K
Q4_K_M7.4 GB
QW
QwQ 32B Preview
Qwen · second-state
Params32.8B
Context32K
Q4_K_M19.9 GB