Hermes GGUF models

149 GGUF models in the Hermes family, quantized by 49 uploaders. Sizes run from 19M to 406B parameters. The longest trained context is 512K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
HE
Hermes 4 14B
Hermes · bartowski
Params14.8B
Context40K
Q4_K_M9.0 GB
Tool use
HE
Hermes-4-70B
Hermes · unsloth
Params70.6B
Context128K
Q4_K_M42.5 GB
ReasoningRoleplayTool use
HE
NousResearch_Hermes 4 70B
Hermes · lmstudio-community
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
HE
Hermes 4 70B
Hermes · bartowski
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
HE
Hermes 4.3 36B
Hermes · bartowski
Params36.2B
Context512K
Q4_K_M21.8 GB
ReasoningRoleplayTool use
HE
Hermes 4.3 36B Heretic
Hermes · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
ReasoningRoleplayTool use
HE
Hermes 4 14B
Hermes · gabriellarson
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningRoleplayTool use
HE
Hermes 2 DPO WestLake 7B v2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.5 GB
Tool use
HE
NousResearch.Hermes 4 405B
Hermes · DevQuasar-13
Params405B
Context128K
Q4_K_M243.1 GB
Tool use
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningRoleplayTool use
HE
Hermes 4 14B OBLITERATED V2
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
HE
conceptofmind_hermes-llongma-2-13b-8k
Hermes · TheBloke
Params13B
Context8K
Q4_K_M7.9 GB
Tool use
HE
Hermes 4 14B
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningRoleplayTool use
HE
Hermes 4 405B
Hermes · bartowski
Params405B
Context128K
Q4_K_M243.1 GB
Tool use
HE
Hermes 4 14B OBLITERATED
Hermes · mradermacher
Params14.8B
Context40K
Q4_K_M9.0 GB
Tool use
HE
Hermes ClimateStorm Sauerkraut abliterated i1
Hermes · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
HE
Hermes 4.3 36B ABLITERATED Heretic
Hermes · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
Tool use
HE
Hermes Storm Climate Sauerkraut abliterated i1
Hermes · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
HE
Hermes-2-Theta-L3-Euryale-Ties-0.8-70B
Hermes · mradermacher
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
HE
Hermes Storm lorablated
Hermes · mradermacher
Params8B
Context128K
Q4_K_M4.9 GB
Tool use
HE
Hermes-4-405B
Hermes · unsloth
Params405B
Context128K
Q4_K_M243.1 GB
ReasoningRoleplayTool use
HE
Hermes-2-Theta-L3-Euryale-Ties-0.8-70B
Hermes · mradermacher
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
HE
Hermes Storm Climate lorablated
Hermes · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
HE
Hermes 4 70B Heretic
Hermes · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
ReasoningRoleplayTool use
HE
Hermes 4.3 36B
Hermes · mradermacher
Params36.2B
Context512K
Q4_K_M21.8 GB
ReasoningRoleplayTool use
HE
Hermes ClimateStorm Sauerkraut abliterated
Hermes · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
HE
Hermes 4 70B
Hermes · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
ReasoningRoleplayTool use
HE
Hermes A1 20B
Hermes · mradermacher
Params20.9B
Context128K
Q4_K_M15.8 GB
Tool use
HE
Hermes Instruct 7B v0.2
Hermes · mradermacher
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
HE
NousResearch.Hermes 4.3 36B
Hermes · DevQuasar
Params36.2B
Context512K
Q4_K_M21.8 GB
ReasoningTool use

About the Hermes GGUF models

Hermes is represented on ggufmodels.com by 149 GGUF models, quantized by 49 uploaders on Hugging Face. The most downloaded are Hermes 4 14B, Hermes-4-70B and NousResearch_Hermes 4 70B. Parameter counts run from 19M to 406B, and the longest trained context window is 512K tokens.

A Q4_K_M file of a Hermes model is between 0.8 to 243.1 GB, which is the quant most people start with. Most files come from mradermacher (74), harkov000 (5) and danrichard6tyt (5). The quants you will meet most often are Q4_K_M (115), Q5_K_M (102), Q2_K (92) and Q3_K_M (90), where the number is how many models offer that quant.

Frequently asked questions

Which Hermes GGUF should I download?

The most downloaded Hermes GGUF here is Hermes 4 14B. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Hermes model need?

Q4_K_M files of Hermes models on this site range from 0.8 to 243.1 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Hermes GGUF files come from?

The most common quantizers here are mradermacher, harkov000 and danrichard6tyt. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing