Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LF
LFM2.5-24B-A2B
Lfm · Arki05
Params24B
Context128K
Smallest0.0 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
WikiHow Mistral Instruct 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 3b Code Base 2k
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
DE
DeepSeek Coder V2 Lite Instruct
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
GE
micks99 gemma 2b instruct ft Data Analytics Digital Marketing Project Management QAv2
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen3.5 4B
Qwen · titan0115
Params4.21B
Context256K
Q4_K_M2.7 GB
ReasoningTool use
MI
nbeerbower bophades v2 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HA
Hare-1.1B-base
Hare · RichardErkhov
Params1.12B
Context32K
Q4_K_M0.7 GB
GGUF
CH
chinese alpaca 2 7b 64k
Chinese · hfl
Params6.93B
Context64K
Smallest2.6 GB
GGUF
OP
OpenHermes 2.5 Code 290k 13B
Openhermes · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
CodingTool use
BE
benhaotang nemo math science philosophy 12B
Benhaotang · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
DR
Dragonwar 7b Alpha
Dragonwar · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
openai community gpt2
Openai · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
BI
Bielik 11B v2.1 Instruct
Bielik · RichardErkhov
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
ME
mergekit community mergekit task arithmetic jaowyfi
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-0.5B-SFT-5e-5-3ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
LL
harborwater open llama 3b claude 30k
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
VI
vicuna-160m
Vicuna · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
LL
CDS CodeLlama 7b 20241113
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
philschmid code llama 3 1 8b text to sql
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Coding
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · realgen4
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
PH
Phi 3.5 Mini investigator_16B
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
OL
OLMo 7B 0724 Instruct Hf
OLMo · aifoundry-org
Params6.89B
Context2K
Smallest1.6 GB
GGUF
QW
jbaron34 Qwen2.5 0.5b bebop reranker
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Embedding
SA
saiga2 7b
Saiga · IlyaGusev
Params6.74B
Context4K
Smallest2.8 GB
GGUF
DE
deepseek-coder-1.3b-chat
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
MI
mii community zefiro 7b base ITA
Mii · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepSeek Coder V2 Lite Base
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding

Other quants

Keep browsing