Q5_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15099 models have a Q5_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
OP
openai community gpt2
Openai · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
BI
Bielik 11B v2.1 Instruct
Bielik · RichardErkhov
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
ME
mergekit community mergekit task arithmetic jaowyfi
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-0.5B-SFT-5e-5-3ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
LL
harborwater open llama 3b claude 30k
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
VI
vicuna-160m
Vicuna · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
LL
CDS CodeLlama 7b 20241113
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
philschmid code llama 3 1 8b text to sql
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Coding
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · realgen4
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
PH
Phi 3.5 Mini investigator_16B
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
OL
OLMo 7B 0724 Instruct Hf
OLMo · aifoundry-org
Params6.89B
Context2K
Smallest1.6 GB
GGUF
QW
jbaron34 Qwen2.5 0.5b bebop reranker
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Embedding
SA
saiga2 7b
Saiga · IlyaGusev
Params6.74B
Context4K
Smallest2.8 GB
GGUF
DE
deepseek-coder-1.3b-chat
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
MI
mii community zefiro 7b base ITA
Mii · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LF
lfm2 audio 1.5b jp
Lfm · cstr
Params79M
Context—
Smallest0.2 GB
GGUF
DE
DeepSeek Coder V2 Lite Base
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
MI
Flammen
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
ticlazau granite 8b instruct rpgle
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
GGUF
LL
llama 3 chinese 8b instruct
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
GGUF
LL
ELYZA-japanese-CodeLlama-7b-instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NO
notHumpback-M1
Nothumpback · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
CN
cnmoro TinyLlama ContextQuestionPair Classifier Reranker
Cnmoro · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Embedding
MI
HuggingFaceH4 mistral 7b anthropic
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
ahmedheakl asm2asm deepseek 1.3b 500k x86 O0 arm gnueabi gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
GI
Gille StrangeMerges 51 7B dare ties
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AL
AlphaMist7B Slr v1
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
FunDialogues llamav2 LoRaco 7b merged
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
mlfoundations dev hp ablations mistral bsz2048 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing