GGUF models by mradermacher

71763 GGUF models quantized by mradermacher on Hugging Face. Sizes run from 2M to 673B parameters. The longest trained context is 10742K tokens.

Most downloadedNewestLargestLongest contextSmallest
AL
ALIA 40b Fc 2606
Alia · mradermacher
Params40.4B
Context160K
Q4_K_M24.6 GB
ReasoningTool use
QW
Qwen2.5 32B Instruct Abliterated v2
Qwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
PY
Pythia 410m
Pythia · mradermacher
Params405M
Context2K
Q4_K_M0.3 GB
GGUF
QW
Qwen3.5 35B A3B Kazakh
Qwen · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
Vision
AI
airoboros 34b 3.3
Airoboros · mradermacher
Params34.4B
Context195K
Q4_K_M21.2 GB
RoleplayTool use
GR
GRM 3.2 Sky Abliterated
Grm · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool use
QW
Mixture Summarizer Qwen3.5 2B
Qwen · mradermacher
Params1.88B
Context256K
Q4_K_M1.3 GB
ReasoningTool use
SA
Samantha v5 wizardlm2
Samantha · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 VL 8B Instruct C_Abliterated v3
Qwen · mradermacher
Params8.19B
Context256K
Q4_K_M5.0 GB
CodingVision
QW
Qwen2.5 Coder 32B Abliterated
Qwen · mradermacher
Params32.8B
Context32K
Q4_K_M19.9 GB
CodingTool use
BO
Borealis 1b
Borealis · mradermacher
Params1B
Context32K
Q4_K_M0.8 GB
GGUF
NE
news title generator
News · mradermacher
Params248M
Context512
Q4_K_M0.2 GB
GGUF
CH
CarbonGPT
Chat · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QW
ThinkLite Critic 7b
Qwen · mradermacher
Params7.62B
Context125K
Q4_K_M4.7 GB
ReasoningVision
QW
NQLSG Qwen2.5 14B MegaFusion v5 Roleplay
Qwen · mradermacher
Params14.8B
Context32K
Q4_K_M9.0 GB
Roleplay
SL
Slimaki 24B v1
Slimaki · mradermacher
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
MI
Nous-Hermes-2-Mixtral-8x7B-DPO
Mistral · mradermacher
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
SE
Seed OSS 36B Base
Seed · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
GGUF
MI
Devstral Small 2 24B Instruct Abliterated
Mistral · mradermacher
Params23.6B
Context384K
Q4_K_M14.3 GB
Coding
LL
llama-3-tulu-2-dpo-8b
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
PO
Polar 32B
Polar · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
GGUF
LL
liberalis-cogitator-llama-3.1-8b-dpo
Llama · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Roleplay
KR
Kreatix K1
Kreatix · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
NE
neural chat 7b v3 3
Neural · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OL
Olmo 3 7B Instruct SFT
OLMo · mradermacher
Params7.3B
Context64K
Q4_K_M4.5 GB
Tool use
SM
SmolLM 135M GrammarSFT
SmolLM · mradermacher
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
MI
Nous-Hermes-2-Mixtral-8x7B-SFT
Mistral · mradermacher
Params46.7B
Context32K
Q4_K_M28.4 GB
Tool use
LL
llama 2 7b chat sexed version3
Llama · mradermacher
Params6.74B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Qwen3 14B Base
Qwen · mradermacher
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF
AI
airoboros l2 70b 2.2
Airoboros · mradermacher
Params69B
Context4K
Q4_K_M41.7 GB
GGUF

Keep browsing