Q5_0 GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 21732 models have a Q5_0 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
YI
brucethemoose_capybara-tess-yi-34b-200k-dare-ties
Yi · TheBloke
Params34.4B
Context195K
Q4_K_M20.7 GB
LL
Meta Llama 3 8B Instruct GGUF Imatrix smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
MI
Tamil-Mistral-7B-v0.1
Mistral · RichardErkhov
Params7.39B
Context32K
Q4_K_M4.5 GB
NE
Nemotron 3.5 Embed 8B BF16
Nemotron · W4D
Params7.95B
Context256K
Q4_K_M4.9 GB
LL
Llama 2 70B Ensemble v5
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
LL
upstage_llama-30b-instruct-2048
Llama · TheBloke
Params32.5B
Context2K
Q4_K_M19.6 GB
OP
OpenELM-3B-Instruct
Openelm · SanctumAI
Params3.04B
Context2K
Q4_K_M1.9 GB
MO
MOSS TTS Realtime
Moss · BricksDisplay
Params2.08B
Context40K
Q4_K_M1.1 GB
MO
pclanglais_monadgpt
Monadgpt · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen2.5 1.5B Instruct
Qwen · ThomasBaruzier
Params1.54B
Context32K
Q4_K_M1.0 GB
QW
Qwen2.5 Coder 14B Instruct AWQ
Qwen · EasierAI
Params14.8B
Context128K
Q4_K_M9.0 GB
AY
Aya Expanse 8b
Aya · duyntnet
Params8B
Context8K
Q4_K_M5.1 GB
DP
openaccess-ai-collective_dpopenhermes-7b
Dpopenhermes · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
GL
Glm Edge 4b Chat
GLM · zai-org
Params4.33B
Context3K
Q4_K_M2.6 GB
AI
Airoboros L2 13B 2.2
Airoboros · TheBloke
Params13B
Context4K
Q4_K_M7.9 GB
SY
maywell_synatra-7b-v0.3-rp
Synatra · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 DeltaCoder 9B
Qwen · danielcherubini
Params8.95B
Context256K
Q4_K_M5.6 GB
LL
chargoddard llama 2 26b trenchcoat stack
Llama · RichardErkhov
Params25.7B
Context—
Q4_K_M5.5 GB
SW
Swallow-70b-NVE-instruct-hf
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
LL
Llama 2 70B Orca 200k
Llama · TheBloke
Params69B
Context4K
Q4_K_M41.4 GB
TO
tokyotech llm Swallow 7b NVE instruct hf
Tokyotech · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
LL
codellama_codellama-70b-hf
Llama · TheBloke
Params69B
Context16K
Q4_K_M41.4 GB
DE
deepseek llm 67b chat
DeepSeek · TheBloke
Params67.4B
Context4K
Q4_K_M40.4 GB
GR
Granite 8b Code Instruct
Granite · duyntnet
Params8.05B
Context4K
Q4_K_M4.9 GB
CO
Cosmos Predict2 14B Text2Image
Cosmos · city96
Params14.3B
Context—
Q4_K_M9.5 GB
LL
llava llama 3 8b v1 1 imat
Llama · city96
Params8B
Context8K
Q4_K_M4.9 GB
AI
aidc-ai-business_marcoroni-7b-v3
Aidc · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
emrgnt-cmplxty_mistral-7b-phibrarian-32k
Mistral · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
AN
sao10k_ana-v1-m7
Ana · TheBloke
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
Deepseek Coder 6.7B Instruct
DeepSeek · second-state
Params6.74B
Context16K
Q4_K_M4.1 GB