Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
LL
nvidia Llama3 ChatQA 1.5 8B GGUF smashed
Llama · PrunaAI
Params8.03B
Context8K
Q4_K_M4.9 GB
SM
Biggie SmoLlm 0.4B
SmolLM · QuantFactory
Params401M
Context2K
Q4_K_M0.3 GB
LL
LLaMA2 13B Psyfighter2
Llama · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
AI
airoboros l2 7B gpt4 m2.0
Airoboros · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
LLaMAntino 2 7b Hf ITA
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
jisukim8873 mistral 7B alpaca case 1 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · QuantFactory
Params2.6B
Context8K
Q4_K_M1.7 GB
QW
Qwen1.5 MoE A2.7B
Qwen · QuantFactory
Params14.3B
Context8K
Q4_K_M9.5 GB
MI
vagosolutions_sauerkrautlm-7b-v1-mistral
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
EM
emma 5
Emma · MrMeOrYou
Params614M
Context2K
Q4_K_M0.4 GB
ME
mihaiii_metis-0.3
Metis · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral Nemo Instruct 2407
Mistral · second-state
Params12.2B
Context1000K
Q4_K_M7.5 GB
OR
OrpoLlama 3 8B
Orpollama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
MI
emrgnt-cmplxty_mistral-7b-sciphi-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
NO
Nous Hermes 2 SUS Chat 34B Slerp
Nous · TheBloke
Params34.4B
Context4K
Q4_K_M20.7 GB
ST
Starling-LM-7B-alpha-ExPO
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
ME
adaptllm_medicine-llm-13b
Medicine · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
FI
finance chat
Finance · TheBloke
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
Llama 3 Open Ko 8B
Llama · LiteLLMs
Params8B
Context8K
Q4_K_M4.9 GB
HI
HiDream I1 Dev
Hidream · city96
Params17.1B
Context—
Q4_K_M11.5 GB
MI
sao10k_sensualize-mixtral-bf16
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
WI
WizardLM 2 8x22B imatrix
Wizardlm · hermes42
Params141B
Context64K
Q4_K_M85.6 GB
MO
model 007 70B
Model · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
MA
mathstral-7B-v0.1
Mathstral · mmnga
Params7.25B
Context32K
Q4_K_M4.4 GB
YI
dragon yi 6B v0
Yi · TheBloke
Params6.06B
Context4K
Q4_K_M3.7 GB
XW
Xwin MLewd 13B v0.2
Xwin · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
AU
Augmental 13B
Augmental · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
MI
Mistral Evolved 11b v0.1
Mistral · bartowski
Params11.2B
Context32K
Q4_K_M6.7 GB
DI
distilgpt2
Distilgpt · QuantFactory
Params121M
Context1K
Q4_K_M0.1 GB
TR
trollek LittleInstructionMaker 4B v0.1
Trollek · RichardErkhov
Params2.3B
Context8K
Q4_K_M2.4 GB