Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
grimjim Llama 3 Oasis v1 OAS 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloomz 3b
Bloomz · RichardErkhov
Params3.64B
Context2K
Q4_K_M2.4 GB
GGUF
QW
Qwen2 0.5B Instruct
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
GGUF
GW
Gweizheng Marcoro14 7B dare
Gweizheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GH
Ghost 8b Beta 1608
Ghost · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
LL
Llama 7b Hf
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GE
google-gemma-2b-it-dequantized
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.10 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BA
Balaaditya GPT2 Finetuned Emotion Classification
Balaaditya · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen3.8 27B Heretic Ara
Qwen · filvyb
Params26.9B
Context256K
Smallest10.7 GB
CodingReasoningTool use
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackoverflow 25000tasks 75p
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
FI
Fixed_Sana
Fixed · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
QW
dat lequoc fast apply 16bit v0.v15 Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Coding
ST
StableCode 3B
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
AU
automerger ShadowYamshadow 7B
Automerger · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
Falcon H1 Tiny R 90M
Falcon · tiiuae
Params108M
Context256K
Q4_K_M0.1 GB
Tool use
NE
NeuralExperiment 7b MagicCoder v7
Neuralexperiment · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
MI
unsloth Hermes 2 Pro Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
TI
TinyLlama 1.1B Tele
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
RU
ruchi012 saul 7B merged 3000
Ruchi · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LU
lumatic ai BongLlama 1.1B Chat alpha v0
Lumatic · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
Mistral-7B-Merge-14-v0.5
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Embedding
PH
Phi 3 Medium 4k Instruct
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.4 GB
GGUF
MI
NeuralHermes-2.5-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
LL
jan hq llama3 s instruct v0.3 checkpoint 6000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
JINJIN7987 llama3 3b neg mtba
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
GE
Gemma-4-E4B-Uncensored-HauhauCS-Aggressive
Gemma · ATOMIKMN
Params7.52B
Context128K
Q4_K_M5.3 GB
ReasoningTool useVision
LL
popatkishan1 Llama 3 8B Instruct AIH RAG
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ZU
a1zubair RedditData Fine Tuned GPT2
Zubair · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF

Other quants

Keep browsing