Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
LM
LMCocktail 10.7B v1
Lmcocktail · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
CL
Claire 7B 0.1
Claire · TheBloke
Params7.22B
Context2K
Q4_K_M5.0 GB
VI
Vigogne 2 13B Instruct
Vigogne · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MY
MythoMix L2 13B
Mythomix · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
RE
Redmond Puffin 13B
Redmond · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
LE
leolm_leo-hessianai-13b-chat-bilingual
Leo · TheBloke
Params13B
Context8K
Q4_K_M7.9 GB
CA
CarbonVillain en 10.7B v4
Carbonvillain · TheBloke
Params10.7B
Context4K
Q4_K_M6.5 GB
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
SY
maywell_synatra-7b-v0.3-base
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
IM
Z Image Turbo
Image · tsqn
Params6.15B
Context—
Q4_K_M5.0 GB
EU
Euryale L2 70B
Euryale · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
SY
migtissera_synthia-7b-v1.3
Synthia · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 70B OASST 1 200
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
PH
Phi 3 Mini 4k Instruct
Phi · second-state
Params3.82B
Context4K
Q4_K_M2.4 GB
RW
rwitz_go-bruins
Rwitz · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
KA
karakuri-lm-8x7b-instruct-v0.1
Karakuri · mmnga
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
QW
Qwen1.5-7B-Chat-AWQ-fp16
Qwen · Qwen
Params7.72B
Context32K
Q4_K_M4.8 GB
LL
Nous Hermes Llama 2 7B
Llama · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
MI
maywell_pivot-10.7b-mistral-v0.2-rp
Mistral · TheBloke
Params10.7B
Context32K
Q4_K_M6.5 GB
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
lemonilia_ashhlimarp-mistral-7b
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
llama 2 13B chat limarp v2 merged
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
ST
stablelm zephyr 3b
Stablelm · TheBloke
Params2.8B
Context4K
Q4_K_M1.7 GB
LL
meta-llama-7b
Llama · TheBloke
Params6.74B
Context2K
Q4_K_M4.1 GB
LL
Chinese Llama 2 13B
Llama · TheBloke
Params13.3B
Context4K
Q4_K_M8.0 GB
HO
Holo 3.1 9B
Holo · prithivMLmods
Params8.95B
Context256K
Q4_K_M5.6 GB