General GGUF models
134659 GGUF models: general chat and instruction-following models. Sizes run from 0M to 20260731B parameters. The longest trained context is 4194304K tokens.
WE
WestLakev2 IreneRP Neural 7B slerp
Westlakev · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
GE
Gemma 4 12B Uncensored Opus4.7 Cot
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
QW
Qwen2.5 7B
Qwen · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
LL
Llama 3.3 70B Instruct Heretic
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GR
granite 3.1 2b instruct
Granite · QuantFactory
Params2.63B
Context128K
Q4_K_M1.6 GB
QR
Qra 13B chat i1
Qra · mradermacher
Params13B
Context4K
Q4_K_M7.9 GB
SA
vagosolutions_sauerkrautlm-solar-instruct
Sauerkrautlm · TheBloke
Params10.7B
Context8K
Q4_K_M6.5 GB
QW
typecastlm qwen3.5 3.8b
Qwen · mihailgribov
Params3.76B
Context256K
Smallest4.0 GB
RO
Rose 20B
Rose · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
MI
MiniCPM
MiniCPM · runfuture
Params2.72B
Context2K
Smallest1.7 GB
FI
Fimbulvetr 11B v2 iMat
Fimbulvetr · InferenceIllusionist
Params10.7B
Context4K
Q4_K_M6.5 GB
LI
Liberated Miqu 70B i1
Liberated · mradermacher
Params69B
Context32K
Q4_K_M41.7 GB
CH
Chocolatine 14B Instruct DPO v1.1 i1
Chocolatine · mradermacher
Params14B
Context4K
Q4_K_M8.6 GB
TI
TinySolar-248m-4k-py
Tinysolar · RichardErkhov
Params248M
Context32K
Q4_K_M0.2 GB
NE
NVIDIA-Nemotron-3.5-Lightning-30B-A3B
Nemotron · lmstudio-community
Params31.6B
Context1024K
Q4_K_M24.5 GB
LL
nvidia.Llama 3 1 Nemotron Ultra 253B v1
Llama · DevQuasar-13
Params253B
Context128K
Q4_K_M150.9 GB
YI
Yi-1.5-9B
Yi · mradermacher
Params8.83B
Context4K
Q4_K_M5.3 GB
KA
KangalKhan RawRuby 7B
Kangalkhan · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
MA
Mamba2 130m Hf
Mamba · rpatel622
Params168M
Context1024K
Smallest0.2 GB
VA
Valkyrie 49B v1
Valkyrie · bartowski
Params49.9B
Context128K
Q4_K_M30.2 GB
MI
Mistral Small 22B ArliAI RPMax v1.1
Mistral · bartowski
Params22.2B
Context32K
Q4_K_M13.3 GB
CY
BaronLLM Offensive Security abliterated
Cybersecurity · huihui-ai
Params8B
Context—
Smallest6.6 GB
QW
Qwen3.6 27B redteam v5
Qwen · RedTeamLab
Params27B
Context256K
Q4_K_M2.0 GB
CO
oobabooga_codebooga-34b-v0.1
Codebooga · TheBloke
Params33.7B
Context16K
Q4_K_M20.2 GB
MY
MythoMax L2 13b
Mythomax · mradermacher
Params13B
Context4K
Q4_K_M8.1 GB
LL
Llama3-18B-lingyang-v1
Llama · mradermacher
Params18.1B
Context1024K
Q4_K_M10.9 GB
LL
Llama 3.1 Nemotron Nano 4B v1.1
Llama · bartowski
Params4.51B
Context128K
Q4_K_M2.8 GB
GE
Gemma 2 2b It
Gemma · BafS
Params2.61B
Context8K
Q4_K_M1.7 GB
WA
Wayra-Perplexity-Estimator-55M
Wayra · latam-gpt
Params55M
Context2K
Q4_K_M0.0 GB
OP
OpenResearcher 30B A3B
Openresearcher · mradermacher
Params31.6B
Context1024K
Q4_K_M24.5 GB