Gguf GGUF models

335 GGUF models in the Gguf family, quantized by 226 uploaders. Sizes run from 1M to 299B parameters. The longest trained context is 1000K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
GG
gguf pythia 1.4b sft full
Gguf · Felladrin
Params1.41B
Context2K
Q4_K_M0.9 GB
GGUF
GG
gguf components
Gguf · JoyFusionAI
Params596M
Context32K
Smallest0.6 GB
CodingReasoningTool use
GG
gguf openhermes tinyllama sft qlora
Gguf · Felladrin
Params1.1B
Context2K
Q4_K_M0.7 GB
Tool use
GG
gguf jina reranker v1 tiny en
Gguf · Felladrin
Params33M
Context8K
Q4_K_M0.0 GB
CodingEmbedding
GG
my bu 30b a3b gguf q4km
Gguf · Maoweicao
Params30.5B
Context256K
Q4_K_M18.6 GB
Tool use
GG
gguf scanner bypass poc
Gguf · vellaveto
Params—
Context—
Smallest0.0 GB
GGUF
GG
Hy MT2 1.8B
Gguf · aoiandroid
Params1.79B
Context256K
Q4_K_M1.1 GB
GGUF
GG
gguf memory amplification poc
Gguf · reservoir-cmyk
Params2M
Context—
Smallest0.0 GB
GGUF
GG
gguf array bomb poc
Gguf · xiaoyaoes
Params—
Context—
Smallest0.0 GB
GGUF
GG
gguf ssti
Gguf · Retr0REG
Params620M
Context32K
Smallest0.3 GB
Coding
GG
Gguf_Src_Gravity 2
Gguf · squ11z1
Params3.09B
Context128K
Q4_K_M1.9 GB
Tool use
GG
gguf sharded pythia 1.4b sft full
Gguf · Felladrin
Params1.41B
Context2K
Q4_K_M0.9 GB
GGUF
GG
k6 ft v2 gguf acik
Gguf · lunarly0
Params7.52B
Context128K
Smallest5.8 GB
Reasoning
GG
gguf model
Gguf · asheldorwin
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GG
gguf models for archon ai
Gguf · ANISH-j
Params1.78B
Context32K
Q4_K_M0.8 GB
ReasoningTool use
GG
gguf model emoji phi unsloth
Gguf · deep-learning-analytics
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
GG
GGUF v4
Gguf · speach1sdef178
Params20.4B
Context—
Smallest12.1 GB
GGUF
GG
gguf misc
Gguf · Cebtenzzre
Params23M
Context512
Smallest0.0 GB
GGUF
GG
gguf pythia 3b deduped sft
Gguf · Felladrin
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
GG
gguf recursion bomb poc
Gguf · wowlegend
Params—
Context—
Smallest0.0 GB
GGUF
GG
gguf ggml model mirror
Gguf · Kj0rdan
Params3.21B
Context40K
Q4_K_M1.1 GB
Tool use
GG
gguf dos crash poc
Gguf · cyberwawa
Params—
Context—
Smallest0.0 GB
GGUF
GG
GGUF models
Gguf · satyadevineni
Params20.9B
Context128K
Smallest2.2 GB
Tool use
GG
Gguf_Multi_Modal_Model_G3_20260617
Gguf · HAHAJIN
Params3.88B
Context128K
Smallest7.8 GB
GGUF
GG
gguf sharded pythia 3b deduped sft
Gguf · Felladrin
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
GG
gguf poc samples
Gguf · yibai777
Params—
Context—
Smallest0.0 GB
GGUF
GG
my bu 30b a3b gguf q5km
Gguf · Maoweicao
Params30.5B
Context256K
Smallest21.7 GB
Tool useVision
GG
Hy3 mix12 IQ2XXS Q4KM
Gguf · hyperhuzaifa
Params299B
Context256K
Smallest99.7 GB
ReasoningTool use
GG
gguf format fine tunned model
Gguf · Sbazar
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GG
gguf model
Gguf · ID2223JR
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use

About the Gguf GGUF models

Gguf is represented on ggufmodels.com by 335 GGUF models, quantized by 226 uploaders on Hugging Face. The most downloaded are gguf pythia 1.4b sft full, gguf components and gguf openhermes tinyllama sft qlora. Parameter counts run from 1M to 299B, and the longest trained context window is 1000K tokens.

A Q4_K_M file of a Gguf model is between 0.0 to 18.6 GB, which is the quant most people start with. Most files come from Felladrin (14), NikolaiRaitschew (6) and Hironabe333 (6). The quants you will meet most often are Q4_K_M (44), Q8_0 (38), F16 (17) and Q5_K_M (16), where the number is how many models offer that quant.

Frequently asked questions

Which Gguf GGUF should I download?

The most downloaded Gguf GGUF here is gguf pythia 1.4b sft full. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Gguf model need?

Q4_K_M files of Gguf models on this site range from 0.0 to 18.6 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Gguf GGUF files come from?

The most common quantizers here are Felladrin, NikolaiRaitschew and Hironabe333. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing