Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama 3.2 1B Instruct Ja Version2
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev oh v1.3 unnatural instructions x4
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QV
QVikhr 3 1.7B Instruction Noreasoning
Qvikhr · Vikhrmodels
Params1.72B
Context40K
Q4_K_M1.1 GB
ReasoningTool use
NE
NESPED GEN TinyLlama schemaLinking min
Nesped · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
aifeifei798 llama3 8B DarkIdol 2.2 Uncensored 1048K
Llama · RichardErkhov
Params8.03B
Context1024K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct STOCK_SOLAR Passthrough
Solar · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
GGUF
MI
jisukim8873 mistral 7B alpaca case 3 2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 1b7
Bloom · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
GGUF
GE
Gemma 2 2b
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DE
Final DEC Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
CH
chinese alpaca 2 7b 16k
Chinese · hfl
Params6.93B
Context16K
Smallest2.6 GB
GGUF
ML
mlfoundations dev oh v1 2 only unnatural instructions
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Codestral-22B-v0.1-hf
Mistral · thesven
Params22.2B
Context32K
Q4_K_M13.3 GB
Coding
MI
Hebrew Mistral 7B 200K
Mistral · RichardErkhov
Params7.5B
Context256K
Q4_K_M4.6 GB
GGUF
IN
Instant RP Noodles 12B v1.4
Instant · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
RoleplayTool use
TH
Themis
Themis · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
XA
XALMA 13B Pretrain
Xalma · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
YI
HenryJJ Instruct Yi 6B Dolly CodeAlpaca
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
SM
SmolLM2 135M Instruct
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Vision
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · datamart99
Params1.86B
Context—
Smallest0.9 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
grounded ai phi3 rag relevance judge merge
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
collaiborateorg Collaiborator MEDLLM Llama 3 8B v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OL
ai2 adapt dev olmo 7b peteish7 step928646
OLMo · RichardErkhov
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP GGUF q4kp
Qwen · lstari
Params1.86B
Context256K
Smallest0.9 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing