Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
UN
universeTBD astrollama
Universetbd · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
AM
AmberChat
Amberchat · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Llama 2 7b Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OC
occiglot occiglot 7b eu5 instruct
Occiglot · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
codellama-13b-oasst-sft-v10
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding
MI
mlfoundations dev hp ablations mistral epoch1 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Huihui Qwen3.8 27B Abliterated
Qwen · xleeelx
Params27.3B
Context256K
Smallest10.9 GB
ReasoningTool useVision
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
JA
japanese-stablelm-base-beta-7b
Japanese · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
mlfoundations dev hp ablations grid mistral bsz1024 lr5e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ruggsea Llama3 stanford encyclopedia philosophy QA
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
ricepaper vi gemma 2b specialized function calling
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
Tool use
TI
TinyLlama-1.1B-Chat-v1.0-function-calling
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Tool use
MI
MiniMA 2 3B
Minima · andrijdavid
Params3.02B
Context4K
Q4_K_M1.8 GB
GGUF
MA
malaysian-tinyllama-1.1b-16k-instructions-v4
Malaysian · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
SI
Siddalingamurthy gita text generation gpt2
Siddalingamurthy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
chlee10 T3Q LLM3 Llama3 sft1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · Whytime
Params1.86B
Context256K
Smallest0.9 GB
GGUF
GI
Gille StrangeMerges 34 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev hp ablations mistral lr2e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
QW
AiisNothing Qwen 2 0.5B finetuned on 23 september
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
ME
mergekit community mergekit ties rsmvina
Mergekit · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama DNA 1.0 8B Instruct
Llama · dnotitia
Params8.03B
Context128K
Smallest3.2 GB
GGUF
ST
Stablelm 2 12b
Stablelm · RichardErkhov
Params12.1B
Context4K
Q4_K_M7.4 GB
GGUF
QW
Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Qwen · Mirxa1
Params122B
Context256K
Q4_K_M74.2 GB
ReasoningTool useVision
LL
LLaMA-Pro-8B-Instruct
Llama · RichardErkhov
Params8.36B
Context4K
Q4_K_M5.1 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.4
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF

Other quants

Keep browsing