Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
GE
Gemma 2 27b It Abliterated
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
MI
Mixtral 8x7B Instruct v0.1
Mistral · second-state
Params46.7B
Context32K
Q4_K_M28.4 GB
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
Llama 3 8B TKK Elite V1.0
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
ST
stable diffusion 3.5 large turbo
Stable · city96
Params8.15B
Context—
Smallest4.8 GB
WH
whisper base
Whisper · oxide-lab
Params73M
Context—
Smallest0.0 GB
WA
PORNHUBAI
Wan · Boombaaa
Params5.25B
Context—
Q4_K_M3.9 GB
LL
Llama 3.2 Taiwan 3B Instruct
Llama · lianghsun
Params3.61B
Context128K
Q4_K_M2.2 GB
ME
q-bert_merged-agi-7b
Merged · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Yarn-Llama-2-7b-128k
Llama · duyntnet
Params6.74B
Context128K
Q4_K_M4.1 GB
GR
Granite 20b Code Base 8k
Granite · RichardErkhov
Params20B
Context8K
Q4_K_M12.8 GB
LL
Carl Llama 2 13B
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LL
neuralmagic Llama 2 7b evolcodealpaca
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
PH
Phi 3 medium 128k instruct
Phi · mmnga
Params14B
Context128K
Q4_K_M8.6 GB
GE
gemma 2 9b it
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GE
Gemma 3 270m
Gemma · gguf-org
Params268M
Context32K
Q4_K_M0.3 GB
CA
Calme-7B-Instruct-v0.9
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NE
intel_neural-chat-7b-v3-3
Neural · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PS
Psyfighter2 13B Vore
Psyfighter · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
ME
malhajar_meditron-7b-chat
Meditron · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
IN
internlm3 8b instruct
InternLM · internlm
Params8.8B
Context32K
Q4_K_M5.4 GB
IA
Iambe 20B DARE
Iambe · TheBloke
Params20B
Context4K
Q4_K_M12.0 GB
LL
CodeLlama 70B Instruct
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
GE
Gemma 4 12B It
Gemma · giladgd
Params11.9B
Context256K
Q4_K_M7.4 GB
GA
udkai_garrulus
Garrulus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
PA
mihaiii_pallas-0.5-laser-0.6
Pallas · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
OP
OpenELM-1_1B-Instruct
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
QW
Qwen2.5 Coder 7B
Qwen · duyntnet
Params7.62B
Context32K
Q4_K_M4.7 GB
QW
Hermes-4-14B
Qwen · Mungert
Params14.8B
Context40K
Q4_K_M9.5 GB
LL
Hermes 2 Pro Llama 3 8B
Llama · AlekseiPravdin
Params7.81B
Context8K
Q4_K_M4.8 GB