Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Huihui Qwen3 Coder Next Opus 4.6 Reasoning Distilled abliterated
Qwen · Momix-44
Params80B
Context256K
Q4_K_M48.5 GB
CodingReasoning
RE
reader lm 0.5b IMat
Reader · legraphista
Params494M
Context250K
Smallest0.3 GB
GGUF
LL
Llama Guard 3 1B
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
GGUF
QW
Qwen Image 2.1 Turbo Uncensored
Qwen · YueFong
Params8.2B
Context256K
Smallest3.3 GB
CodingVision
MI
juhwanlee llmdo Mistral 7B case 1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
cognitivecomputations dolphin 2.6 mistral 7b dpo laser
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
Falcon H1 0.5B Instruct
Falcon · tiiuae
Params521M
Context16K
Q4_K_M0.3 GB
GGUF
MI
flammenai flammen16 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama3 Med42 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
Llama 3.1 Storm 8B
Llama · legraphista
Params8B
Context128K
Smallest2.0 GB
Reasoning
LL
LLaMAntino 2 7b Hf ITA
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
jisukim8873 mistral 7B alpaca case 1 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · Chobrr
Params7.5B
Context128K
Q4_K_M5.3 GB
Vision
QW
Qwen_Qwen3 4B
Qwen · Alcoft
Params4.02B
Context40K
Q4_K_M2.5 GB
ReasoningTool use
ST
Starling-LM-7B-alpha-ExPO
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
TR
trollek LittleInstructionMaker 4B v0.1
Trollek · RichardErkhov
Params2.3B
Context8K
Q4_K_M2.4 GB
GGUF
MI
Mistral NeMo Minitron 8B Base
Mistral · legraphista
Params8.41B
Context8K
Smallest2.1 GB
GGUF
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · KingKassell
Params1.86B
Context—
Smallest0.9 GB
GGUF
LL
Hermes-2-Theta-Llama-3-8B
Llama · thesven
Params8B
Context8K
Q4_K_M4.9 GB
Tool use
GE
Gemma 2 2b Jpn It
Gemma · webbigdata
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
FA
Falcon H1 Tiny Tool Calling 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
Tool use
AY
Aya Expanse 32b Ungated
Aya · RichardErkhov
Params32.3B
Context8K
Q4_K_M19.8 GB
GGUF
MI
starmpcc Asclepius Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llava v1.6 34B
LLaVA · cjpais
Params34.4B
Context4K
Q4_K_M20.7 GB
Vision
LL
codellama CodeLlama 13b Python hf
Llama · RichardErkhov
Params13B
Context—
Q4_K_M0.0 GB
Coding
QW
Vikhr-Qwen-2.5-1.5B-Instruct
Qwen · Vikhrmodels
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF
MI
PVD 160k Mistral 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
K2S3 Mistral 7b v1.48
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
OP
OpenELM-450M
Openelm · RichardErkhov
Params457M
Context2K
Q4_K_M0.3 GB
GGUF

Other quants

Keep browsing