Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
LL
Llama 3.2 1B
Llama · QuantFactory
Params1.24B
Context128K
Q4_K_M0.8 GB
CH
Chunky-Lemon-Cookie-11B
Chunky · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
DE
DeepSeek-Coder-V2-Lite-Instruct
DeepSeek · gaianet
Params15.7B
Context160K
Q4_K_M10.4 GB
MI
Ministral 3b instruct
Mistral · QuantFactory
Params3.32B
Context128K
Q4_K_M2.0 GB
QW
Qwen2.5 Coder 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
DA
arco reflection v4
Danube · QuantFactory
Params514M
Context8K
Q4_K_M0.3 GB
YI
mlinmg_sg-raccoon-yi-55b-200k
Yi · TheBloke
Params55.6B
Context195K
Q4_K_M33.3 GB
TU
allenai_tulu-2-7b
Tulu · TheBloke
Params6.74B
Context8K
Q4_K_M4.1 GB
QW
Josiefied Qwen2.5 7B Instruct Abliterated
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
SZ
szymonrucinski Curie 7B v1
Szymonrucinski · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
NI
Ninja v1 128k
Ninja · mmnga
Params7.24B
Context128K
Q4_K_M4.4 GB
QC
qCammel 13
Qcammel · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
RE
reader lm 1.5b
Reader · QuantFactory
Params1.78B
Context250K
Q4_K_M1.1 GB
EV
Evolutionstory
Evolutionstory · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
vietnamese llama2 7B 40GB
Llama · TheBloke
Params6.86B
Context4K
Q4_K_M4.2 GB
WE
senseable_westlake-7b-v2
Westlake · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
FL
FLOR 6.3B Instructed
Flor · xaviviro
Params6.45B
Context4K
Q4_K_M4.1 GB
TI
Tinystories Lay8 Hs512 Hd8 33M
Tinystories · RichardErkhov
Params165M
Context512
Q4_K_M0.1 GB
LL
Llama-3-8B-grammar-correction
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LU
L3 bluuwhale SAO MIX 8B V1 fp32 merge calc
Lunaris · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
PH
TroyDoesAI Phi 3 Context Obedient RAG GGUF smashed
Phi · PrunaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
Mistral 7B Instruct RDPO
Mistral · QuantFactory
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
ORLM LLaMA 3 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
LL
Llama 3.2 3B Instruct
Llama · second-state
Params3.21B
Context128K
Q4_K_M2.0 GB
TI
minecraft
Tinyllama · QuantFactory
Params1.1B
Context2K
Q4_K_M0.7 GB
WI
wizardlm_wizardmath-7b-v1.1
Wizardmath · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
flozi00_llama-2-13b-german-assistant-v2
Llama · TheBloke
Params13B
Context2K
Q4_K_M7.9 GB
QW
Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
orangetin_openhermes-mixtral-8x7b
Mistral · TheBloke
Params46.7B
Context32K
Q4_K_M26.4 GB
DE
Deepseek LLM 7B Chat
DeepSeek · second-state
Params6.91B
Context4K
Q4_K_M4.2 GB