Tinyllama GGUF models

1128 GGUF models in the Tinyllama family, quantized by 406 uploaders. Sizes run from 1M to 8.03B parameters. The longest trained context is 128K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
TI
tinyllama_tinyllama-1.1b-chat-v1.0
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
py007_tinyllama-1.1b-chat-v0.3
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · TinyLlama
Params1.1B
Context2K
Smallest0.6 GB
Coding
TI
Tinyllama-Cinder-1.3B-Reason-Test
Tinyllama · RichardErkhov
Params1.28B
Context2K
Q4_K_M0.8 GB
GGUF
TI
tinyllama split test
Tinyllama · ngxson
Params24M
Context—
Smallest0.0 GB
GGUF
TI
tinyllama_tinyllama-1.1b-intermediate-step-1431k-3t
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama 1.1B Chat v1.0 imatrix
Tinyllama · duyntnet
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
py007_tinyllama-1.1b-intermediate-step-715k-1.5t
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
py007_tinyllama-1.1b-python-v0.1
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
py007_tinyllama-1.1b-intermediate-step-480k-1t
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama-1.1B-DavidAttenborough
Tinyllama · mradermacher
Params1.1B
Context4K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · second-state
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
abdgrt_tinyllama-2-1b-miniguanaco
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tinyllama 15M
Tinyllama · QuantFactory
Params15M
Context256
Q4_K_M0.0 GB
GGUF
TI
jeff31415_tinyllama-1.1b-1t-openorca
Tinyllama · TheBloke
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama Indo Chat v.02
Tinyllama · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tinyllama 1.1b Abliterated
Tinyllama · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
tinyllama history
Tinyllama · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
minecraft
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · andrijdavid
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama 1.1B Chat v0.6
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama 1.1B Chat v0.5
Tinyllama · afrideva
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama Friendly Psychotherapist
Tinyllama · mradermacher
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama_v1.1
Tinyllama · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B 32k Instruct
Tinyllama · mradermacher
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B python v0.1
Tinyllama · TinyLlama
Params1.1B
Context2K
Smallest0.6 GB
Coding
TI
TinyLlama v1.1
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
TinyLlama 1.1B Chat v0.2
Tinyllama · kirp
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding

About the Tinyllama GGUF models

Tinyllama is represented on ggufmodels.com by 1,128 GGUF models, quantized by 406 uploaders on Hugging Face. The most downloaded are tinyllama_tinyllama-1.1b-chat-v1.0, py007_tinyllama-1.1b-chat-v0.3 and TinyLlama 1.1B Chat v0.6. Parameter counts run from 1M to 8.03B, and the longest trained context window is 128K tokens.

A Q4_K_M file of a Tinyllama model is between 0.0 to 4.9 GB, which is the quant most people start with. Most files come from mradermacher (189), RichardErkhov (116) and featherless-ai-quants (64). The quants you will meet most often are Q4_K_M (739), Q8_0 (565), Q2_K (508) and Q5_K_M (474), where the number is how many models offer that quant.

Frequently asked questions

Which Tinyllama GGUF should I download?

The most downloaded Tinyllama GGUF here is tinyllama_tinyllama-1.1b-chat-v1.0. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Tinyllama model need?

Q4_K_M files of Tinyllama models on this site range from 0.0 to 4.9 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Tinyllama GGUF files come from?

The most common quantizers here are mradermacher, RichardErkhov and featherless-ai-quants. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing