Q5_1 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 17701 models have a Q5_1 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama3 Med42 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
QW
Qwen2.5 0.5b RBase
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
AH
Ahma 7B
Ahma · QuantFactory
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
LL
nvidia Llama3 ChatQA 1.5 8B GGUF smashed
Llama · PrunaAI
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SM
Biggie SmoLlm 0.4B
SmolLM · QuantFactory
Params401M
Context2K
Q4_K_M0.3 GB
GGUF
LL
LLaMAntino 2 7b Hf ITA
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
jisukim8873 mistral 7B alpaca case 1 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2B ArliAI RPMax v1.1
Gemma · QuantFactory
Params2.6B
Context8K
Q4_K_M1.7 GB
GGUF
QW
Qwen1.5 MoE A2.7B
Qwen · QuantFactory
Params14.3B
Context8K
Q4_K_M9.5 GB
GGUF
EM
emma 5
Emma · MrMeOrYou
Params614M
Context2K
Q4_K_M0.4 GB
GGUF
OR
OrpoLlama 3 8B
Orpollama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
ST
Starling-LM-7B-alpha-ExPO
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 Open Ko 8B
Llama · LiteLLMs
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
HI
HiDream I1 Dev
Hidream · city96
Params17.1B
Context—
Q4_K_M11.5 GB
GGUF
WI
WizardLM 2 8x22B imatrix
Wizardlm · hermes42
Params141B
Context64K
Q4_K_M85.6 GB
Tool use
DI
distilgpt2
Distilgpt · QuantFactory
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
TR
trollek LittleInstructionMaker 4B v0.1
Trollek · RichardErkhov
Params2.3B
Context8K
Q4_K_M2.4 GB
GGUF
BI
Bielik 7B Instruct v0.1
Bielik · QuantFactory
Params7.24B
Context4K
Q4_K_M4.4 GB
GGUF
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 0.5B Instruct
Qwen · QuantFactory
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Hermes-2-Theta-Llama-3-8B
Llama · thesven
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
FA
Falcon H1 Tiny Tool Calling 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
Tool use
AY
Aya Expanse 32b Ungated
Aya · RichardErkhov
Params32.3B
Context8K
Q4_K_M19.8 GB
GGUF
BL
bloom 560m
Bloom · QuantFactory
Params816M
Context1K
Q4_K_M0.6 GB
Coding
QW
bu-30b-a3b-preview
Qwen · Mungert
Params30.5B
Context256K
Q4_K_M18.9 GB
Vision
MI
starmpcc Asclepius Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 14B Gutenberg Instruct Slerpeno
Qwen · QuantFactory
Params14.8B
Context32K
Q4_K_M9.0 GB
GGUF
TO
Tongyi-DeepResearch-30B-A3B
Tongyi · Mungert
Params30.5B
Context128K
Q4_K_M19.0 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · duyntnet
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
LL
codellama CodeLlama 13b Python hf
Llama · RichardErkhov
Params13B
Context—
Q4_K_M0.0 GB
Coding

Other quants

Keep browsing