Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
codellama CodeLlama 34b Instruct hf
Llama · RichardErkhov
Params33.7B
Context16K
Q4_K_M20.2 GB
Coding
GE
Gemma 2 9b It 390K
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo1epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
kayfour T3Q llama3.1 8B it Ko safe
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
MH
MHGanainy gpt2 xl lora multi shared 512 top
Mhganainy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ME
Merdeka LLM merdeka llm lawyer 3b 128k instruct
Merdeka · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
OP
Opencsg Stable Code 3b v1
Opencsg · RichardErkhov
Params2.8B
Context16K
Q4_K_M1.7 GB
Coding
MA
martimfasantos tinyllama 1.1b mt sft full
Martimfasantos · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama 3.2 3B Wildguard Ko 2410
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
LL
ValiantLabs Llama2 13B ShiningValiant
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
PY
pythia-160m_ft_CookingRecipes
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
MI
Mirxa3.6 27B
Mirxa · mirxa2
Params26.9B
Context256K
Smallest10.0 GB
ReasoningTool useVision
GR
Grafted Hermetic Platypus B 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
MI
Chinese Mistral 7B v0.1
Mistral · RichardErkhov
Params7.5B
Context32K
Q4_K_M4.5 GB
GGUF
YA
YandexGPT 5 Lite 8B pretrain
Yandexgpt · yaroslav0530
Params8.04B
Context32K
Smallest0.6 GB
GGUF
XU
XuanYuan 6B
Xuanyuan · RichardErkhov
Params6.39B
Context8K
Q4_K_M3.9 GB
GGUF
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
LL
smol_llama-220M-open_instruct
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
GGUF
DE
DeepSeek R1 0528 Qwen3 8B
DeepSeek · quantized4all
Params8.19B
Context128K
Q4_K_M5.0 GB
ReasoningTool use
YI
Yi 34B 200K DARE Merge v5
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
LL
Suzume Llama 3 8B Multilingual
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · adsdaw
Params34.7B
Context256K
Q4_K_M21.2 GB
ReasoningTool useVision
SO
I SOLAR 10.7B Sft v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
HA
habanoz tinyllama oasst1 top1 instruct full lr1 5 v0.1
Habanoz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
MinaMila GermanCredit Ext Mitig CFA llama 3 InstBase ImpFeat 20ep withsex
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 1.5B Instruct MLX
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use

Other quants

Keep browsing