Q5_1 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 17701 models have a Q5_1 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
InternLM2
InternLM · QuantFactory
Params7.74B
Context8K
Q4_K_M4.7 GB
GGUF
QW
QwQ 32B
Qwen · duyntnet
Params32.8B
Context40K
Q4_K_M19.9 GB
ReasoningTool use
LL
AMD Llama 135m code
Llama · QuantFactory
Params134M
Context2K
Q4_K_M0.1 GB
Coding
FL
Hyper FLUX.1 dev
Flux · martintomov
Params11.9B
Context—
Q4_K_M6.9 GB
GGUF
LL
colossal llama 2 7b base
Llama · shaowenchen
Params7.04B
Context4K
Smallest3.0 GB
GGUF
QW
Qwen2.5 1.5B
Qwen · QuantFactory
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
LL
Llama-3-13B-Instruct-v0.1
Llama · QuantFactory
Params13.3B
Context8K
Q4_K_M8.1 GB
GGUF
IS
Isaak Carter J.O.S.I.E.3 Beta9 7B slerp
Isaak · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BU
Buzz 8b Large v0.5
Buzz · QuantFactory
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
MI
mistral nemo wissenschaft 12B
Mistral · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama 3 Smaug 8B
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
DU
DuckDB-NSQL-7B-v0.1
Duckdb · duyntnet
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 7B
Qwen · QuantFactory
Params7.6B
Context128K
Q4_K_M4.7 GB
GGUF
GR
granite 8b code instruct 4k
Granite · QuantFactory
Params8.05B
Context4K
Q4_K_M4.9 GB
CodingTool use
GR
Granite 3.3 2b Base
Granite · mrutkows
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GR
granite 3b code base 2k
Granite · QuantFactory
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
SM
SmolLM 360M
SmolLM · QuantFactory
Params362M
Context2K
Q4_K_M0.3 GB
Tool use
LL
Llama 3 8B Lexi Uncensored
Llama · QuantFactory
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
BI
Bielik 11B v2.2 Instruct
Bielik · QuantFactory
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
QW
xpo-qwen2
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
GE
magnum v3 9b customgemma2
Gemma · QuantFactory
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
Minerva 3B base RAG
Minerva · QuantFactory
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
LL
Llama-3-8b-64k-PoSE
Llama · LiteLLMs
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
PE
PersianMind v1.0
Persianmind · QuantFactory
Params6.8B
Context2K
Q4_K_M4.1 GB
GGUF
MI
mathstral 7B v0.1
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
tifa benchmark llama2 tifa question generation
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
grimjim Llama 3 Instruct demi merge 8B
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
MV
mvpmaster NeuralDareDMistralPro 7b slerp
Mvpmaster · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AMD Llama 135m
Llama · QuantFactory
Params134M
Context2K
Q4_K_M0.1 GB
GGUF
LL
aifeifei798 DarkIdol Llama 3.1 8B Instruct 1.0 Uncensored
Llama · RichardErkhov
Params8B
Context—
Q4_K_M0.0 GB
GGUF

Other quants

Keep browsing