Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
Qwen3.8 27B Uncensored HauhauCS Aggressive MTP
Qwen · maxidistination
Params1.86B
Context256K
Smallest0.9 GB
MA
ManthanKulakarni TinyLlama 1.1B Text2SQL
Manthankulakarni · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
PH
ihughes15234 phi35 tictactoe dpo5epoch
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
BR
Breeze-7B-Instruct-v1_0
Breeze · RichardErkhov
Params7.49B
Context32K
Q4_K_M4.5 GB
IN
Internlm2_5 7b Chat Abliterated
InternLM · RichardErkhov
Params7.74B
Context32K
Q4_K_M4.7 GB
LL
Llama 3 LewdPlay 8B Evo
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
DE
Polaris 7B Preview
DeepSeek · quantized4all
Params7.62B
Context128K
Q4_K_M4.7 GB
ML
mlfoundations dev stackoverflow 5000tasks .25p
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
thkim0305 Meta Llama3 8B Circuit breaker
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
SO
SOLAR-10.7B-Instruct-SOLARC-M-10.7B-slerp
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
BR
BroAlanTaps GPT2 large 4 62000steps
Broalantaps · RichardErkhov
Params839M
Context1K
Q4_K_M0.5 GB
MO
Mondhirch Natural Language to Cypher 2.7b
Mondhirch · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
SA
SakanaAI EvoLLM JP v1 7B
Sakanaai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
RA
RaushanTurganbay GPT2 sft and dpo tuned
Raushanturganbay · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
DA
Dans DiscountModels Dans Instruct Mix 8b ChatML V0.0.1
Dans · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
CH
BM25Chunk
Chunk · RichardErkhov
Params1.35B
Context8K
Q4_K_M0.9 GB
MI
Mistral 7B Base SFT SimPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
phildunphy14 llama 3 1 non quant 8b 55k
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
OR
orpheus 3b 0.1 ft
Orpheus · dahara1
Params3.78B
Context128K
Smallest2.4 GB
LL
arnavgrg codellama 13b instruct nf4 fp16 upscaled
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
QW
Qwen3 58B Embiggened
Qwen · QuixiAI
Params58B
Context40K
Q4_K_M38.2 GB
PH
grounded ai phi3.5 rag relevance judge merge
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Layerskip Llama3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
JP
jpalgo jp 3k shlow qft v1
Jpalgo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Vikhr-Llama-3.2-1B-Instruct-abliterated
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LO
LordNoah Alpaca spin gpt2 e1 se0
Lordnoah · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
LL
AIML GEEK promptv5 finetuned CodeLlama7b Instruct
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
BL
Bloom 1b1
Bloom · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
DP
OGNO 7B
Dpo · LiteLLMs
Params7.24B
Context32K
Q4_K_M4.4 GB