Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
MI
cognitivecomputations_dolphin-2.6-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
MA
tiger-lab_mammoth-coder-13b
Mammoth · TheBloke
Params13B
Context16K
Q4_K_M7.9 GB
OR
Orca 2 13B SFT v5
Orca · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
OP
dreamgen_opus-v0-7b
Opus · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 3.1 Minitron 4B Width Base
Llama · ThomasBaruzier
Params4.51B
Context128K
Q4_K_M2.8 GB
MI
MiniCPM-OCR 1.0 DSpark
MiniCPM · openbmb
Params297M
Context256K
Q4_K_M0.2 GB
PH
Phi 3 medium 128k instruct imatrix
Phi · duyntnet
Params14B
Context128K
Q4_K_M8.6 GB
LT
LTX2.3 10Eros
Ltx · Abiray
Params21B
Context—
Q4_K_M14.3 GB
FA
Falcon H1 1.5B Instruct
Falcon · tiiuae
Params1.55B
Context128K
Q4_K_M0.9 GB
LL
dolphin 2.9 llama3 70b GGUF smashed
Llama · PrunaAI
Params70.6B
Context8K
Q4_K_M42.5 GB
MI
K2S3 Mistral 7bx2 48layers_v1.2
Mistral · RichardErkhov
Params10.9B
Context32K
Q4_K_M6.6 GB
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
PA
neuralnovel_panda-7b-v0.1
Panda · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
KA
karakuri lm 8x7b chat v0.1
Karakuri · mmnga
Params46.7B
Context32K
Smallest9.8 GB
HE
conceptofmind_hermes-llongma-2-13b-8k
Hermes · TheBloke
Params13B
Context8K
Q4_K_M7.9 GB
YI
SG Raccoon Yi 55B
Yi · TheBloke
Params55.6B
Context4K
Q4_K_M33.3 GB
ME
m42-health_med42-70b
Med · TheBloke
Params69B
Context2K
Q4_K_M41.4 GB
LL
nousresearch_yarn-llama-2-70b-32k
Llama · TheBloke
Params69B
Context32K
Q4_K_M41.4 GB
AV
average-dolphin-8x7B
Average · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
PA
mihaiii_pallas-0.5-frankenmerge
Pallas · TheBloke
Params36B
Context195K
Q4_K_M21.6 GB
NE
NVIDIA Nemotron Nano 9B v2 Japanese
Nemotron · mmnga-o
Params8.89B
Context1024K
Q4_K_M6.5 GB
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
Mistral 7B Instruct v0.2
Mistral · second-state
Params7.24B
Context32K
Q4_K_M4.4 GB
TO
tokyotech llm Swallow 13b instruct hf
Tokyotech · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
CH
austism_chronos-wizardlm-uc-scot-st-13b
Chronos · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
LL
Llm Jp 3.1 1.8b
Llm · mmnga
Params1.87B
Context4K
Q4_K_M1.2 GB
NO
doctor-shotgun_nous-capybara-limarpv3-34b
Nous · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
LA
law chat
Law · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
PH
Phi 3 mini 4k instruct imatrix
Phi · duyntnet
Params3.82B
Context4K
Q4_K_M2.4 GB
AU
AutoCoder S 6.7B
Autocoder · QuantFactory
Params6.74B
Context16K
Q4_K_M4.1 GB