Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen3.5 9B Base
Qwen · titan0115
Params8.95B
Context256K
Q4_K_M5.6 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Rboy18
Params34.7B
Context256K
Q4_K_M21.2 GB
PO
postboxat18 AR400GroqModel
Postboxat · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3 10b It Kor Extented Chang
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
WI
WizardLM-2-7B-abliterated
Wizardlm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SM
Smaug 34B v0.1
Smaug · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
QW
quocdat25 fast apply 16bit v0.v16 Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
Llama-2-7B-instruct-text2sql
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MA
Matter 0.1 Slim 7B C
Matter · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
TI
stories converted
Tinyllamas · shibatch
Params24M
Context128
Smallest0.0 GB
SE
semcoder semcoder 1030
Semcoder · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
OC
PP OCRv6 medium rec
Ocrv · cstr
Params19M
Context—
Smallest0.0 GB
VI
vicuna 33b v1.3
Vicuna · shaowenchen
Params32.5B
Context2K
Smallest13.5 GB
PH
IntelLabs sqft phi 3 mini 4k 30 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
LL
Unichat llama3 Chinese 8B
Llama · Monor
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
leap llm Meta Llama 3 8B Instruct sft alfworld iter2
Llama · RichardErkhov
Params8B
Context8K
Q4_K_M4.9 GB
MX
mxbai rerank xsmall v1
Mxbai · cstr
Params71M
Context512
Smallest0.1 GB
LL
Papy_2_Llama 3.1 8B Instruct_text
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MA
Matter 0.1 7B Boost
Matter · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Qwen3.5 2B
Qwen · ZuzeTt
Params1.88B
Context256K
Q4_K_M1.3 GB
TI
TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · andrijdavid
Params1.1B
Context2K
Q4_K_M0.7 GB
MA
magnum v1 32b
Magnum · anthracite-org
Params32.5B
Context32K
Smallest10.3 GB
PH
tuong nguyen prd Phi 3.5 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
SA
SambaLingo Arabic Base
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
BI
BioMistral BioMistral 7B TIES
Biomistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RO
roadz dv finetuned 211124
Roadz · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
liyimengwork llama 3.1 8B wenyanwen2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
LI
lightblue reranker 0.5 bincont filt
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
QW
Qwen3.5 0.8B
Qwen · titan0115
Params752M
Context256K
Q4_K_M0.5 GB