Heretic GGUF models

136 GGUF models in the Heretic family, quantized by 10 uploaders. Sizes run from 268M to 299B parameters. The longest trained context is 1024K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
HE
Hy MT2 1.8B Heretic
Heretic · mradermacher
Params1.79B
Context256K
Q4_K_M1.1 GB
GGUF
HE
Heretic_Aquif 3.5 Nano 1B
Heretic · mradermacher
Params1.72B
Context40K
Q4_K_M1.1 GB
CodingReasoning
HE
heretic_HelpingAI 3B Coder
Heretic · mradermacher
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
HE
G9v3 3B Heretic Abliterated
Heretic · Vortecks
Params3B
Context128K
Q4_K_M1.8 GB
Tool use
HE
Heretic AceInstruct 1.5B
Heretic · mradermacher
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
HE
Heretic FastApply 1.5B v1.0
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
HE
E4B Heretic Enriched
Heretic · mradermacher
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
HE
Heretic Openhands Lm 1.5b v0.1
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
HE
Heretic Rp Vision 31b
Heretic · mradermacher
Params30.7B
Context256K
Q4_K_M18.7 GB
ReasoningRoleplayTool use
HE
heretic_Q25 1.5B VeoLu
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
HE
Heretic InfiR 1B Instruct
Heretic · mradermacher
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
HE
Heretic_Magnum v2 4b
Heretic · mradermacher
Params4.51B
Context128K
Q4_K_M2.8 GB
GGUF
HE
heretic_Athena 2 0.5B
Heretic · mradermacher
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
HE
Heretic Goppa LogiLlama
Heretic · mradermacher
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
HE
G9v3 3B Heretic Abliterated
Heretic · mradermacher
Params2.99B
Context128K
Q4_K_M1.8 GB
ReasoningTool use
HE
Heretic Arcee Lite
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
HE
heretic_RimDialogue 3B v1
Heretic · mradermacher
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
HE
Heretic Scalpel E2B
Heretic · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
GGUF
HE
Heretic Kodify Nano
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
HE
Heretic AQUA 1B
Heretic · mradermacher
Params1B
Context32K
Q4_K_M0.8 GB
GGUF
HE
Hy3 heretic i1
Heretic · mradermacher
Params295B
Context—
Smallest0.6 GB
GGUF
HE
Heretic Smol Dirty Alice
Heretic · mradermacher
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
HE
Heretic YuLan Mini Instruct
Heretic · mradermacher
Params2.42B
Context28K
Q4_K_M1.8 GB
GGUF
HE
Heretic Fox 1 1.6B Instruct v0.1
Heretic · mradermacher
Params1.67B
Context8K
Q4_K_M1.1 GB
GGUF
HE
heretic_AwA 1.5B
Heretic · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
HE
Heretic Scalpel E2B
Heretic · mradermacher
Params4.65B
Context128K
Q4_K_M3.4 GB
Vision
HE
Heretic HyperCLOVAX SEED Text Instruct 0.5B
Heretic · mradermacher
Params566M
Context8K
Q4_K_M0.4 GB
GGUF
HE
Heretic OpenCoder 1.5B Instruct
Heretic · mradermacher
Params1.91B
Context4K
Q4_K_M1.4 GB
Coding
HE
Heretic 5Q3 Nexus Coder MAX
Heretic · mradermacher
Params34.7B
Context256K
Q4_K_M21.2 GB
CodingReasoningTool use
HE
Heretic_Aquif 3.5 Nano 1B
Heretic · mradermacher
Params1.72B
Context40K
Q4_K_M1.1 GB
CodingReasoningTool use

About the Heretic GGUF models

Heretic is represented on ggufmodels.com by 136 GGUF models, quantized by 10 uploaders on Hugging Face. The most downloaded are Hy MT2 1.8B Heretic, Heretic_Aquif 3.5 Nano 1B and heretic_HelpingAI 3B Coder. Parameter counts run from 268M to 299B, and the longest trained context window is 1024K tokens.

A Q4_K_M file of a Heretic model is between 0.3 to 21.2 GB, which is the quant most people start with. Most files come from mradermacher (121), ineosnexus (4) and OpenIntelligenceNet (3). The quants you will meet most often are Q6_K (127), Q4_K_M (127), Q5_K_M (122) and IQ4_XS (122), where the number is how many models offer that quant.

Frequently asked questions

Which Heretic GGUF should I download?

The most downloaded Heretic GGUF here is Hy MT2 1.8B Heretic. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Heretic model need?

Q4_K_M files of Heretic models on this site range from 0.3 to 21.2 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Heretic GGUF files come from?

The most common quantizers here are mradermacher, ineosnexus and OpenIntelligenceNet. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing