General GGUF models
134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.
LL
Llama 3 8B ProLong 64k Base
Llama · RichardErkhov
Params8B
Context64K
Q4_K_M4.9 GB
YI
Yi-6B-200K
Yi · mradermacher
Params6.06B
Context195K
Q4_K_M3.7 GB
WI
WizardLM 70B V1.0
Wizardlm · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
FR
fredithefish_openzephyrchat-v0.2
Fredithefish · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LE
sanjiwatsuki_lelantos-7b
Lelantos · TheBloke
Params7.24B
Context8K
Q4_K_M4.4 GB
IN
Internlm2_5 7b Chat 1m
InternLM · duyntnet
Params7.74B
Context256K
Q4_K_M4.7 GB
IN
Index Translate 9B
Index · mradermacher
Params9.2B
Context256K
Q4_K_M5.8 GB
NE
NeuTTS Nano
Neutts · BricksDisplay
Params262M
Context2K
Q4_K_M0.2 GB
WI
WizardLM 30B Uncensored Guanaco SuperCOT 30b
Wizardlm · mradermacher
Params32.5B
Context2K
Q4_K_M19.6 GB
LL
Llama-3-Patronus-Lynx-8B-Instruct
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
CA
Carnice MoE 35B A3B
Carnice · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
SW
Swallow-70b-hf
Swallow · mradermacher
Params69.2B
Context4K
Q4_K_M41.5 GB
MI
Flirty Mistral
Mistral · mradermacher
Params7.3B
Context32K
Q4_K_M4.4 GB
QW
Qwen3 4B Instruct 2507 Heretic v4
Qwen · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
QW
ReSearch Qwen 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
lei HuggingFace Gemma 2 9B 4bit Chat Level Measurement Guide 07242024
Gemma · RichardErkhov
Params3.4B
Context—
Q4_K_M1.7 GB
KU
Kuchiki 1.1 L2 7B
Kuchiki · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
ER
Ernie-4.5-0.3B-Pt
Ernie · unsloth
Params361M
Context128K
Q4_K_M0.2 GB
NE
NeuralGanesha 7b
Neuralganesha · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
NO
Noromaid 13B v0.2
Noromaid · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
GE
Gemma 4 12B It QAT SOMPOA Heresy
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
LF
Lfm25 24b
Lfm · PollardWeights
Params24B
Context128K
Smallest5.8 GB
GE
Gemma 4 E4B It Erotica
Gemma · mradermacher
Params7.5B
Context128K
Q4_K_M5.3 GB
QW
Qwen3 0.6B
Qwen · QuantFactory
Params752M
Context40K
Q4_K_M0.5 GB
SC
ajibawa-2023_scarlett-7b
Scarlett · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
QW
Qwen1.5-1.8B-Chat-AWQ-fp16
Qwen · Qwen
Params1.84B
Context32K
Q4_K_M1.2 GB
LL
Llama 3.2 1B Instruct
Llama · QuantFactory
Params1.24B
Context128K
Q4_K_M0.8 GB
AK
Akins 3B
Akins · TheBloke
Params2.8B
Context4K
Q4_K_M1.7 GB
QW
sakura-1.5b-qwen2.5-v1.0-fp16
Qwen · SakuraLLM
Params1.78B
Context128K
Smallest3.6 GB
MI
Mistral Military 24B
Mistral · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB