Q4_K GGUF models

A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
terry69 qwen feedback dirty 0.5
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
MI
Mistral Nemo Instruct 2407
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama 3.2 Korean Bllossom 3B
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
DI
distilgpt2-tiny-conversational
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
DE
Deepseek Coder 7b Base v1.5
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
Coding
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BI
bitnet_b1_58-3B
Bitnet · RichardErkhov
Params3.32B
Context2K
Q4_K_M2.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
SC
scb10x typhoon v1.5 72b instruct
Scb · RichardErkhov
Params72.3B
Context32K
Q4_K_M44.1 GB
GGUF
QW
Qwen2.5 Math 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context4K
Q4_K_M1.0 GB
Tool use
FL
EclecticEuphoria Flux Unet Models
Flux · ND911
Params11.9B
Context—
Smallest6.9 GB
GGUF
LL
nawhgnuj KamalaHarris Llama 3.1 8B Chat
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LI
lidoreliya13 microlama lidor finetuned
Lidoreliya · RichardErkhov
Params305M
Context2K
Q4_K_M0.2 GB
GGUF
LL
Llama 3.2 1B Instruct MGSM8K Sp
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
ML
mlfoundations dev webinstruct v1 beta stage 3 shareGPT format
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
meta-llama/Llama-3.1-8B-Instruct
Llama · ISTA-DASLab
Params8.03B
Context128K
Smallest3.2 GB
Tool use
LL
mazzaqq meta llama Meta Llama 3.1 8B Instruct DPO noSFT 1800 merged
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
mindy 7b v2
Mindy · andrijdavid
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
Sailor-0.5B-Chat
Sailor · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
GE
Google Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LL
chinese llama 2 7b 16k
Llama · shaowenchen
Params6.93B
Context16K
Smallest2.9 GB
GGUF
JI
jimzp gita text generation gpt2
Jimzp · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
TO
toastloaf autotrain gpt2 finetune crab 4batches
Toastloaf · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
PH
akshit Gupta phi3 sharegpt fullnew merged FinetunedByAG
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
QW
Agnuxo Agente GPT Qwen 2.5 7B Spanish 16bit
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
Tool use
LL
herisan llama 3 8b mental health counseling conversations
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
MI
hydra project OpenHyperion 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
SantoshGollapudi gita text generation gpt2
Santoshgollapudi · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Mistral 7B Instruct v0.2 Turkish
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BH
ai4bharat hercule hi
Bharat · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use

Other quants

Keep browsing