Q5_1 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 17701 models have a Q5_1 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
dolphin-2.9.2-mixtral-8x22b
Mistral · crusoeai
Params141B
Context64K
Q4_K_M85.6 GB
GGUF
GL
GLM-ASR-Nano-2512
GLM · Mungert
Params1.59B
Context8K
Q4_K_M1.1 GB
GGUF
RW
rwkv 6 finch 14b
Rwkv · latestissue
Params14.1B
Context1024K
Smallest5.4 GB
GGUF
NE
Nemotron 3.5 Embed 1B BF16
Nemotron · W4D
Params1.14B
Context256K
Q4_K_M0.7 GB
EmbeddingVision
SW
Swallow MS 7b Instruct v0.1
Swallow · RichardErkhov
Params7.33B
Context4K
Q4_K_M4.4 GB
GGUF
BR
Breeze 7B 32k Instruct V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
GE
gemma 2 27b it abliterated
Gemma · QuantFactory
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
QW
Qwen2.5 32B
Qwen · mradermacher
Params32.8B
Context128K
Q4_K_M19.9 GB
GGUF
AT
N ATLaS
Atlas · QuantFactory
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta Llama 3 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
MI
Chinese Mixtral 8x7B
Mistral · RichardErkhov
Params46.9B
Context32K
Q4_K_M28.6 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
GR
Granite 3b Code Instruct 128k
Granite · RichardErkhov
Params3.48B
Context125K
Q4_K_M2.1 GB
Coding
FA
Falcon H1 7B Instruct
Falcon · tiiuae
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
IM
Z Image Turbo
Image · tsqn
Params6.15B
Context—
Q4_K_M5.0 GB
GGUF
LL
Llama-3-8B-Instruct-262k
Llama · crusoeai
Params8B
Context256K
Q4_K_M4.9 GB
GGUF
SN
SniffyOtter 7B
Sniffyotter · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
saraprice llama2 7B backdoor DEPLOYMENT
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
QV
QVikhr 3 4B Instruction
Qvikhr · Vikhrmodels
Params4B
Context40K
Q4_K_M2.5 GB
GGUF
OP
OpenCodeInterpreter-DS-1.3B
Opencodeinterpreter · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
RA
RakutenAI-7B-instruct
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
QW
Qwen2.5 32B Instruct
Qwen · ThomasBaruzier
Params32.8B
Context32K
Q4_K_M19.9 GB
GGUF
QW
DMind-3-mini
Qwen · Mungert
Params4B
Context256K
Q4_K_M2.5 GB
Reasoning
QW
Qwen2.5 7B Instruct
Qwen · mradermacher
Params7.62B
Context32K
Q4_K_M4.7 GB
GGUF
LL
dolphin 2.9 llama3 8b 256k GGUF smashed
Llama · PrunaAI
Params8B
Context512K
Q4_K_M4.9 GB
GGUF
ST
stable code instruct 3b
Stable · QuantFactory
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
LT
LTX2.3 10Eros 1.4
Ltx · vantagewithai
Params21B
Context—
Q4_K_M14.3 GB
GGUF
SW
Swallow 70b Instruct v0.1
Swallow · RichardErkhov
Params69B
Context4K
Q4_K_M41.5 GB
GGUF
TU
Tulu2 7b_writing Prompts
Tulu · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Other quants

Keep browsing