Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral 7B Merge 14 v0.1
Mistral · LiteLLMs
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DM
dmis lab meerkat 7b v1.0
Dmis · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 7B Instruct Abliterated v3
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · nurdich
Params1.86B
Context256K
Smallest0.9 GB
GGUF
LL
hf 100 Llama 3.1 Spellbound StoryWriter 70b instruct 0.3 16bit
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
MI
Voxtral-Mini-4B-Realtime-2602
Mistral · andrijdavid
Params4.43B
Context—
Q4_K_M2.9 GB
GGUF
BE
beomi Yi Ko 34B Chat Preview
Beomi · RichardErkhov
Params34.6B
Context2K
Q4_K_M20.8 GB
GGUF
SA
saishf Top Western Maid 7B
Saishf · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 9b It
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
QW
Qwen3.5 9B
Qwen · eaddario
Params8.95B
Context256K
Q4_K_M5.6 GB
ReasoningTool useVision
QW
Qwen3.5 2B Base
Qwen · titan0115
Params1.88B
Context256K
Q4_K_M1.3 GB
ReasoningTool use
DE
Deepseekmath 7b Sft MATH v2
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
QW
Soob3123_Amoral Qwen3 14B
Qwen · Alcoft
Params14.8B
Context40K
Q4_K_M9.0 GB
ReasoningTool use
AS
X-ASR-zh-en streaming zipformer2
Asr · Luigi
Params153M
Context—
Smallest0.1 GB
GGUF
GO
Gonzo Code 7B
Gonzo · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
QW
Qwen3.8-27B
Qwen · Aydge
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
FL
FLUX.1 Fill dev
Flux · gaianet
Params84M
Context—
Smallest0.3 GB
GGUF
GR
Granite 20b Code Instruct 8k
Granite · RichardErkhov
Params20.1B
Context8K
Q4_K_M12.8 GB
Coding
WA
WAN2.2 14B Rapid AllInOne v12.2
Wan · Novice25
Params17.3B
Context—
Smallest6.4 GB
GGUF
QW
Qwen3.8-27B
Qwen · Didkk
Params27.3B
Context256K
Q4_K_M16.6 GB
Vision
LL
llama-3-chinese-8b-instruct-v3
Llama · hfl
Params8.03B
Context8K
Smallest3.2 GB
GGUF
MI
Ft Mistral 7b Instruct v0.2 Sorry Bench 202406
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
lemon mint gemma ko 7b it v0.41
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
SI
Sixtyoneeighty 7b
Sixtyoneeighty · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DR
Dracarys2 72B Instruct
Dracarys · RichardErkhov
Params72.7B
Context32K
Q4_K_M47.4 GB
Tool use
LL
Configurable-Llama-3-8B-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo3epoch v2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
VI
Vistral 24B Instruct
Vistral · Vikhrmodels
Params23.6B
Context128K
Q4_K_M14.3 GB
GGUF
OP
OpenELM-270M
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
FL
flux.1 lite 8B
Flux · Green-Sky
Params8.16B
Context—
Smallest2.9 GB
GGUF

Other quants

Keep browsing