Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
gemma 4 31b it assistant mtp
Gemma · NotMe404
Params470M
Context256K
Q4_K_M0.4 GB
GGUF
MM
mmq test
Mmq · Beinsezii
Params14.8B
Context128K
Smallest0.4 GB
ReasoningTool use
DE
Huihui DeepSeek V4 Flash 0731 abliterated
DeepSeek · CoreWolf
Params284B
Context1024K
Smallest10.9 GB
GGUF
LL
Reflection Llama 3.1 70B IMat
Llama · legraphista
Params70.6B
Context8K
Smallest15.3 GB
GGUF
MI
mistral-7b-v0.1-layla-v4-chatml
Mistral · l3utterfly
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
YI
Yi 1.5 6B
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · Unrestricted
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool useVision
GE
gemma 4 E4B it
Gemma · eaddario
Params7.52B
Context128K
Smallest0.0 GB
GGUF
VA
VAGOsolutions SauerkrautLM 7b HerO
Vagosolutions · RichardErkhov
Params3.8B
Context—
Q4_K_M2.2 GB
GGUF
LL
microsoft LLaMA 2 7b GTL Delta
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · ATOMIKMN
Params1.86B
Context—
Smallest0.9 GB
GGUF
MI
Mixtral 8x7B Instruct v0.1
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
TY
Typhoon 7b Instruct 02 19 2024
Typhoon · RichardErkhov
Params7.27B
Context32K
Q4_K_M4.4 GB
GGUF
MI
MistarlingMaid 2x7B Base
Mistarlingmaid · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
CodingTool use
BR
Breeze 7B 32k Base V1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
Hermes 3 Llama 3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
Tool use
DE
DeepSeek V4 Flash
DeepSeek · ddh0
Params284B
Context1024K
Smallest3.8 GB
GGUF
MI
nbeerbower Gutensuppe mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
KA
Karakuri Lm 7b Apm v0.1
Karakuri · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
GL
glm-4-9b-chat-1m
GLM · legraphista
Params9.48B
Context1024K
Smallest4.0 GB
GGUF
MI
Mistral Large Instruct 2407
Mistral · legraphista
Params123B
Context32K
Smallest26.0 GB
GGUF
GE
gemma 4 26B A4B it
Gemma · eaddario
Params25.2B
Context256K
Smallest0.1 GB
GGUF
DE
deepseek-coder-1.3b-chat-and-function-calling
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
CodingTool use
ML
mlfoundations dev stackoverflow 5000tasks 1p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LA
Labradorite 13b
Labradorite · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
VI
Vikhrmodels/Vikhr-YandexGPT-5-Lite-8B-it
Vikhrmodels · Vikhrmodels
Params8.04B
Context32K
Q4_K_M4.9 GB
GGUF
QW
Replete LLM V2.5 Qwen 14b
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use

Other quants

Keep browsing