Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
PH
rhaymison phi 3 portuguese tom cat 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
LL
Tiny Random Llama 3
Llama · RichardErkhov
Params4M
Context4K
Q4_K_M0.0 GB
MI
harshitv804 MetaMath Mistral 2x7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GE
Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced
Gemma · Ishowbackup
Params25.2B
Context256K
Q4_K_M16.8 GB
TI
tinyfrank 1.4B
Tinyfrank · andrijdavid
Params1.36B
Context2K
Q4_K_M0.8 GB
LL
ihughes15234 llama 3 1 8bi tictactoe dpo5epoch v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
IN
InfinityKumon 2x7B
Infinitykumon · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
BL
Bloomz 7b1 Mt Sft Chat
Bloomz · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
OP
OpenELM 1_1B DPO Full 1 5
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
SA
SauerkrautLM SOLAR Instruct
Sauerkrautlm · RichardErkhov
Params10.7B
Context8K
Q4_K_M6.5 GB
SY
SyntheticMoist-11B-v2
Syntheticmoist · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Qwen · Craftwork333
Params34.7B
Context256K
Q4_K_M21.2 GB
IN
InternLM2
InternLM · RichardErkhov
Params7.74B
Context8K
Q4_K_M4.7 GB
LL
DevsDoCode LLama 3 8b Uncensored
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 3 Youko 8b En Ja MT LoRA
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
VA
Value4AI ValueLlama 3 8B
Value · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 3 Chinese 8b Instruct v2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr1e 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
muzammil eds Meta Llama 3.1 8B Instruct English to French v3
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
dolphin-2.1-mistral-7b-snr-laser
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
nkpz llama2 22b chronos alpaca experiment1
Llama · RichardErkhov
Params21.8B
Context4K
Q4_K_M13.2 GB
SM
Smollm2 135m Bootleg Instruct04
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
PU
pushpendrasingh21 gpt2 medical chatbot1
Pushpendrasingh · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GE
Kitsunebi v1 Gemma2 8k 9B
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
TH
thanhtunguet gpt2 address standardizer prompted
Thanhtunguet · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
SP
SparseLLM ReluLLaMA 13B
Sparsellm · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
OP
OpenLLM Ro RoMistral 7b Instruct
Openllm · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
PH
sujithatz finbot phi 3.5 mini instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
BR
BroAlanTaps GPT2 large 128 12000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB