GGUF models by mradermacher

71763 GGUF models quantized by mradermacher on Hugging Face. Sizes run from 2M to 673B parameters. The longest trained context is 10742K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Hermes-2-Pro-Mistral-7B-infinity
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Qwimi 4B
Qwimi · mradermacher
Params4B
Context256K
Q4_K_M2.5 GB
Tool use
VI
MN Violet Lotus 12B
Violet · mradermacher
Params12.2B
Context128K
Q4_K_M7.5 GB
Roleplay
GE
Gemma 4 MoonGem 31B
Gemma · mradermacher
Params30.7B
Context256K
Q4_K_M18.7 GB
Roleplay
HE
HERETICSEEK 7B Ditill
Hereticseek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
TH
TherapyLlama 8B v1 i1
Therapyllama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
NU
Nu 17b
mradermacher
Params16.8B
Context32K
Q4_K_M10.3 GB
Roleplay
MI
MiniAI Quata2 4b
Miniai · mradermacher
Params4B
Context40K
Q4_K_M2.5 GB
Reasoning
DE
DeepSeek R1 Distill Qwen 7B
DeepSeek · mradermacher
Params7.62B
Context128K
Q4_K_M4.7 GB
Reasoning
GL
Q3.6 27B GLM 5.1 DA
GLM · mradermacher
Params27B
Context256K
Q4_K_M16.5 GB
Reasoning
QW
JoyFox Qwen3.6 35B A3B RP Aggressive
Qwen · mradermacher
Params35.5B
Context256K
Q4_K_M21.7 GB
Roleplay
AR
ArmenianGPT 1.0 3B
Armeniangpt · mradermacher
Params3.43B
Context256K
Q4_K_M2.1 GB
Reasoning
LL
Llama3-ChatQA-1.5-70B
Llama · mradermacher
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
FI
Fireplace 34b
Fireplace · mradermacher
Params34.4B
Context195K
Q4_K_M22.1 GB
CodingTool use
LL
Llama 3.3 70B Instruct ablated i1
Llama · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
GR
Granite 8b Code Instruct 4k
Granite · mradermacher
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use
QW
Qwen2.5 Wolverine CODER 11B V2
Qwen · mradermacher
Params10.9B
Context32K
Q4_K_M6.6 GB
Coding
GE
Gemma4 Gutenberg Literotica 12B
Gemma · mradermacher
Params11.9B
Context256K
Q4_K_M7.4 GB
GGUF
ME
L3.1 70b MeowMix i1
Meowmix · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
CO
C4AI orpo model
Command R · mradermacher
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
ST
Storm-7B
Storm · mradermacher
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
PH
Phind 70B
Phind · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Coding
YI
Yi-34B-200K-AEZAKMI-v2
Yi · mradermacher
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
LL
llama-3-8b-samantha
Llama · mradermacher
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GR
Granite 20b Code Instruct 8k
Granite · mradermacher
Params20B
Context8K
Q4_K_M12.8 GB
CodingTool use
SO
SOLAR 10.7B Instruct v1.0
Solar · mradermacher
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
HA
HarmonicHarlequin v5 20B
Harmonicharlequin · mradermacher
Params33.3B
Context4K
Q4_K_M20.1 GB
GGUF
MI
Infinity Instruct 7M Gen Mistral 7B
Mistral · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek-V2-Lite
DeepSeek · mradermacher
Params15.7B
Context160K
Q4_K_M10.4 GB
GGUF
LL
Code-Llama-2-13B-instruct-text2sql
Llama · mradermacher
Params13B
Context16K
Q4_K_M7.9 GB
Coding

Keep browsing