Tiny GGUF models

150 GGUF models in the Tiny family, quantized by 51 uploaders. Sizes run from 3M to 3.88B parameters. The longest trained context is 488K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
TI
Tiny Aya Global
Tiny · CohereLabs
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny LLM
Tiny · mradermacher
Params13M
Context1K
Q4_K_M0.0 GB
GGUF
TI
Tiny Pirate 1.1b v0.2 i1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tiny Cowboy 1.1b v0.1 i1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tiny Vicuna 1B
Tiny · afrideva
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
tiny1m
Tiny · shibatch
Params—
Context256
Q4_K_M0.0 GB
GGUF
TI
Tiny Aya Water
Tiny · CohereLabs
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny OR1 Rust
Tiny · mradermacher
Params1.72B
Context40K
Q4_K_M1.1 GB
CodingReasoning
TI
tiny sd
Tiny · turingevo
Params530M
Context—
Smallest0.8 GB
GGUF
TI
Tiny Vicuna 1b
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
CohereLabs.tiny Aya Water
Tiny · DevQuasar
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Aya Earth
Tiny · CohereLabs
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Aya Fire
Tiny · CohereLabs
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny-Pirate-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tiny Viking 1.1b v0.1 i1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
CohereLabs.tiny Aya Earth
Tiny · DevQuasar
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Pirate 1.1b v0.2
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
tiny random gpt2 i1
Tiny · mradermacher
Params7M
Context1K
Q4_K_M0.0 GB
GGUF
TI
Tiny Pirate 1.1b v0.1 i1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
Tiny DarkLlama SPIN Implementation i1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
Roleplay
TI
Tiny DarkLlama Chat
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
Roleplay
TI
CohereLabs.tiny Aya Global
Tiny · DevQuasar
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Purr 350M Merged
Tiny · mradermacher
Params354M
Context125K
Q4_K_M0.2 GB
Tool use
TI
Tiny-Cowboy-1.1b-v0.1
Tiny · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
CohereLabs.tiny Aya Fire
Tiny · DevQuasar
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
CohereLabs.tiny Aya Base
Tiny · DevQuasar
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Cowboy 1.1b v0.1
Tiny · mradermacher
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
TI
tiny search reader 60M
Tiny · ahkamboh
Params62M
Context512
Q4_K_M0.0 GB
GGUF
TI
Tiny Aya Global
Tiny · professorf
Params3.35B
Context488K
Q4_K_M2.1 GB
GGUF
TI
Tiny Charlotte
Tiny · mradermacher
Params51M
Context128
Q4_K_M0.0 GB
GGUF

About the Tiny GGUF models

Tiny is represented on ggufmodels.com by 150 GGUF models, quantized by 51 uploaders on Hugging Face. The most downloaded are Tiny Aya Global, Tiny LLM and Tiny Pirate 1.1b v0.2 i1. Parameter counts run from 3M to 3.88B, and the longest trained context window is 488K tokens.

A Q4_K_M file of a Tiny model is between 0.0 to 2.5 GB, which is the quant most people start with. Most files come from mradermacher (57), tensorblock (9) and RichardErkhov (8). The quants you will meet most often are Q4_K_M (97), Q2_K (88), Q5_K_M (82) and Q8_0 (79), where the number is how many models offer that quant.

Frequently asked questions

Which Tiny GGUF should I download?

The most downloaded Tiny GGUF here is Tiny Aya Global. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Tiny model need?

Q4_K_M files of Tiny models on this site range from 0.0 to 2.5 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Tiny GGUF files come from?

The most common quantizers here are mradermacher, tensorblock and RichardErkhov. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing