Q3_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
SC
scb10x typhoon v1.5 72b instruct
Scb · RichardErkhov
Params72.3B
Context32K
Q4_K_M44.1 GB
GGUF
QW
Qwen2.5 Math 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context4K
Q4_K_M1.0 GB
Tool use
LL
nawhgnuj KamalaHarris Llama 3.1 8B Chat
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LI
lidoreliya13 microlama lidor finetuned
Lidoreliya · RichardErkhov
Params305M
Context2K
Q4_K_M0.2 GB
GGUF
LL
Llama 3.2 1B Instruct MGSM8K Sp
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev webinstruct v1 beta stage 3 shareGPT format
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
meta-llama/Llama-3.1-8B-Instruct
Llama · ISTA-DASLab
Params8.03B
Context128K
Smallest3.2 GB
Tool use
LL
mazzaqq meta llama Meta Llama 3.1 8B Instruct DPO noSFT 1800 merged
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
mindy 7b v2
Mindy · andrijdavid
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
Sailor-0.5B-Chat
Sailor · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
GE
Google Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LL
chinese llama 2 7b 16k
Llama · shaowenchen
Params6.93B
Context16K
Smallest2.9 GB
GGUF
JI
jimzp gita text generation gpt2
Jimzp · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
TO
toastloaf autotrain gpt2 finetune crab 4batches
Toastloaf · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
akshit Gupta phi3 sharegpt fullnew merged FinetunedByAG
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
QW
Agnuxo Agente GPT Qwen 2.5 7B Spanish 16bit
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
LL
herisan llama 3 8b mental health counseling conversations
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
hydra project OpenHyperion 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
SantoshGollapudi gita text generation gpt2
Santoshgollapudi · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Mistral 7B Instruct v0.2 Turkish
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BH
ai4bharat hercule hi
Bharat · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
DE
Deepseek Coder 6.7B Kexer
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
mlfoundations dev hp ablations mistral scheduler linear warmup0.05
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Base SFT RDPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Mikhil jivus Llama 3.2 1B FineTuned
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3 Instruct 8B CPO
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
ME
MexIvanov MistRAG 7B ruen v1 merged
Mexivanov · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
MI
avemio digital mistral 7b Sft orpo 3000 steps merge
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ZE
Zenith 7B Dpo
Zenith · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Other quants

Keep browsing