Q5_1 GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 17701 models have a Q5_1 file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.

Most downloadedNewestLargestLongest contextSmallest
CN
cnmoro TinyLlama ContextQuestionPair Classifier Reranker
Cnmoro · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Embedding
MI
Mistral 7B Instruct Uz
Mistral · QuantFactory
Params7.25B
Context32K
Q4_K_M4.4 GB
Tool use
PR
FW ProX 1.7B
Prox · QuantFactory
Params1.74B
Context2K
Q4_K_M1.1 GB
GGUF
MI
HuggingFaceH4 mistral 7b anthropic
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen3 Swallow 8B RL v0.2
Qwen · tatsuyaaaaaaa
Params8.19B
Context40K
Q4_K_M5.0 GB
ReasoningTool use
DE
ahmedheakl asm2asm deepseek 1.3b 500k x86 O0 arm gnueabi gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
MI
Candle Mistral 7B OpenOrca
Mistral · DanielClough
Params7.24B
Context—
Smallest2.4 GB
GGUF
GI
Gille StrangeMerges 51 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
AlphaMist7B Slr v1
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
FunDialogues llamav2 LoRaco 7b merged
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral bsz2048 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
TinyAgent 7B
Tinyagent · QuantFactory
Params8.03B
Context10K
Q4_K_M4.9 GB
GGUF
LL
Llama3.1 BestMix Chem Einstein 8B
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Foundation-Sec-8B
Llama · Mungert
Params8.03B
Context128K
Q4_K_M5.1 GB
GGUF
GE
gemma-7b-it
Gemma · asedmammad
Params8.54B
Context8K
Q4_K_M5.1 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use
LL
Llama-3-8B-Lexi-Uncensored
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OP
OpenCoder 8B Base
Opencoder · QuantFactory
Params7.77B
Context8K
Q4_K_M4.7 GB
Coding
SW
Swallow MX 8x7b NVE v0.1
Swallow · RichardErkhov
Params46.7B
Context4K
Q4_K_M28.4 GB
GGUF
EX
EXAONE 3.5 7.8B Instruct
Exaone · QuantFactory
Params7.82B
Context32K
Q4_K_M4.8 GB
GGUF
GE
Gemma 3 Glitter 12B
Gemma · duyntnet
Params11.8B
Context128K
Q4_K_M7.3 GB
GGUF
SW
Swallow 7b Hf
Swallow · RichardErkhov
Params6.83B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Llama3.1 8B Enigma
Llama · QuantFactory
Params8.03B
Context128K
Q4_K_M4.9 GB
CodingTool use
LY
Lyra4 Gutenberg 12B
Lyra · QuantFactory
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
LL
Llama 3 8B Instruct Chinese
Llama · Rookie
Params8.03B
Context8K
Smallest6.1 GB
GGUF
MI
dolphin-2.9.1-mixtral-1x22b
Mistral · crusoeai
Params22.2B
Context64K
Q4_K_M13.3 GB
GGUF
VI
vicuna 13b v1.5 16k
Vicuna · shaowenchen
Params13B
Context16K
Smallest5.4 GB
GGUF
LL
Llama-3.1-70B-Instruct
Llama · Mungert
Params70.6B
Context128K
Q4_K_M42.8 GB
Tool use
LU
Lucy Edit 1.1 Dev
Lucy · vantagewithai
Params5B
Context—
Q4_K_M3.4 GB
GGUF
QW
Qwen3 Reranker 0.6B
Qwen · QuantFactory
Params751M
Context40K
Q4_K_M0.5 GB
EmbeddingReasoningTool use

Other quants

Keep browsing