Base GGUF models

127 GGUF models in the Base family, quantized by 43 uploaders. Sizes run from 37M to 105B parameters. The longest trained context is 1024K tokens.

TinySmallMediumLarge
Most downloadedNewestLargestLongest contextSmallest
BA
internistai base 7b v0.2
Base · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BA
T5 Base
Base · chatpig
Params223M
Context512
Q4_K_M0.1 GB
Coding
BA
L2 7b Base Guanaco Uncensored i1
Base · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BA
T5 Base
Base · callgg
Params110M
Context512
Smallest0.1 GB
Coding
BA
MedWest 7B
Base · Aryanne
Params7.24B
Context32K
Smallest4.1 GB
GGUF
BA
EagleVLA
Base · diantoudefengshan
Params2.8B
Context128K
Smallest5.7 GB
Vision
BA
BASE 1.5B
Base · mradermacher
Params1.54B
Context32K
Q4_K_M1.0 GB
ReasoningTool use
BA
e5-base-v2
Base · ChristianAzinn
Params109M
Context512
Q4_K_M0.1 GB
Embedding
BA
S1 Base 1.5 32B 128K
Base · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
ReasoningTool use
BA
L BASE EXPERMIENTAL THINKER V1A i1
Base · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Reasoning
BA
S1 Base 8B
Base · mradermacher
Params8.19B
Context40K
Q4_K_M5.0 GB
GGUF
BA
e5 base v2 i1
Base · mradermacher
Params109M
Context512
Q4_K_M0.1 GB
Embedding
BA
Base FuseO1 Sky Flash Rombo
Base · mradermacher
Params32.8B
Context128K
Q4_K_M19.9 GB
Tool use
BA
Base Stories 8b
Base · mradermacher
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BA
UL3.3 X75 BASE 70B i1
Base · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
BA
L2 7b Base Guanaco Uncensored
Base · mradermacher
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BA
T5 Base
Base · callgg
Params223M
Context512
Smallest0.1 GB
GGUF
BA
e5 base v2
Base · mradermacher
Params109M
Context512
Q4_K_M0.1 GB
Embedding
BA
m3e base
Base · mradermacher
Params102M
Context512
Q4_K_M0.1 GB
Embedding
BA
S1 Base 1.5 32B 128K
Base · mradermacher
Params32.8B
Context40K
Q4_K_M19.8 GB
ReasoningTool use
BA
t5 base grammar corrector
Base · mradermacher
Params248M
Context512
Q4_K_M0.2 GB
GGUF
BA
UL3.3 X BASE 70B i1
Base · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
BA
Base 7b v0.2
Base · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BA
Base FuseO1 Sky Flash Rombo i1
Base · mradermacher
Params32.8B
Context128K
Q4_K_M19.9 GB
Tool use
BA
e5 base
Base · mradermacher
Params109M
Context512
Q4_K_M0.1 GB
Embedding
BA
UL3.3 X75 BASE 70B
Base · mradermacher
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
BA
T5 Base Japanese Question Generation
Base · mradermacher
Params223M
Context512
Q4_K_M0.1 GB
GGUF
BA
e5 base i1
Base · mradermacher
Params109M
Context512
Q4_K_M0.1 GB
Embedding
BA
MN 2407 Base ChatML Zeroed Interleaved 17.7B i1
Base · mradermacher
Params17.7B
Context128K
Q4_K_M10.7 GB
GGUF
BA
T5 Base Japanese Adapt
Base · mradermacher
Params248M
Context512
Q4_K_M0.2 GB
GGUF

About the Base GGUF models

Base is represented on ggufmodels.com by 127 GGUF models, quantized by 43 uploaders on Hugging Face. The most downloaded are internistai base 7b v0.2, T5 Base and L2 7b Base Guanaco Uncensored i1. Parameter counts run from 37M to 105B, and the longest trained context window is 1024K tokens.

A Q4_K_M file of a Base model is between 0.1 to 42.5 GB, which is the quant most people start with. Most files come from mradermacher (58), xlight05 (6) and tensorblock (6). The quants you will meet most often are Q4_K_M (78), Q2_K (69), Q5_K_M (63) and Q4_K_S (62), where the number is how many models offer that quant.

Frequently asked questions

Which Base GGUF should I download?

The most downloaded Base GGUF here is internistai base 7b v0.2. Pick the size that fits your GPU first, then choose a quant: Q4_K_M is the usual starting point, and Q5_K_M or Q6_K give more quality if you have memory to spare. The VRAM pages below list what fits each GPU.

How much VRAM does a Base model need?

Q4_K_M files of Base models on this site range from 0.1 to 42.5 GB. Add the KV cache for your context length and about 0.5 GB for the runtime, which usually means 1.5 to 3 GB on top of the file at an 8K context.

Where do Base GGUF files come from?

The most common quantizers here are mradermacher, xlight05 and tensorblock. They convert the original model and publish quantized files on Hugging Face. ggufmodels.com does not host files; every download button opens Hugging Face.

Keep browsing