Q5_1 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 17701 models have a Q5_1 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
LI
Lite-Oute-1-300M-Instruct
Lite · OuteAI
Params300M
Context4K
Q4_K_M0.2 GB
GGUF
BI
Bielik 11B v2.1 Instruct
Bielik · RichardErkhov
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
ME
mergekit community mergekit task arithmetic jaowyfi
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-0.5B-SFT-5e-5-3ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
QW
Neumind Math 7B Instruct
Qwen · QuantFactory
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
LL
harborwater open llama 3b claude 30k
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
VI
vicuna-160m
Vicuna · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
CO
Typst Coder 9B
Coder · QuantFactory
Params8.83B
Context128K
Q4_K_M5.3 GB
Coding
LL
CDS CodeLlama 7b 20241113
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
philschmid code llama 3 1 8b text to sql
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Coding
LL
Llama 3.1 Tulu 3.1 8B
Llama · brittlewis12
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Replete LLM Qwen2 7b
Qwen · QuantFactory
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
PH
Phi 3.5 Mini investigator_16B
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
SQ
t2sql
Sql · gguf-org
Params4.02B
Context40K
Q4_K_M0.4 GB
Tool use
QW
NextCoder-14B
Qwen · Mungert
Params14.8B
Context32K
Q4_K_M9.1 GB
CodingTool use
QW
NuExtract 1.5 tiny
Qwen · QuantFactory
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
QW
jbaron34 Qwen2.5 0.5b bebop reranker
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Embedding
DE
deepseek-coder-1.3b-chat
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
AC
AceMath-1.5B-Instruct
Acemath · Mungert
Params1.78B
Context4K
Q4_K_M1.2 GB
GGUF
MI
mii community zefiro 7b base ITA
Mii · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KA
Kai 3B Instruct
Kai · SimplySara
Params3.08B
Context64K
Q4_K_M1.9 GB
ReasoningTool use
DE
DeepSeek Coder V2 Lite Base
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
MI
Flammen
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
ticlazau granite 8b instruct rpgle
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
GGUF
MI
Midm-2.0-Mini-Instruct
Midm · cookieshake
Params2.31B
Context32K
Q4_K_M1.4 GB
Tool use
LL
ELYZA-japanese-CodeLlama-7b-instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
FL
FLOR 6.3B
Flor · xaviviro
Params6.45B
Context4K
Smallest2.7 GB
GGUF
GR
granite-20b-code-instruct-8k
Granite · Mungert
Params20.1B
Context8K
Q4_K_M12.6 GB
CodingTool use
SA
vagosolutions_sauerkrautlm-3b-v1
Sauerkrautlm · TheBloke
Params3.43B
Context2K
Smallest2.0 GB
GGUF
NO
notHumpback-M1
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF

Other quants

Keep browsing