Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
JA
japanese-stablelm-instruct-ja_vocab-beta-7b
Japanese · RichardErkhov
Params6.88B
Context4K
Q4_K_M4.2 GB
ME
Meltemi 7B Instruct v1
Meltemi · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
LL
Devarui379 VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
SA
SambaLingo Arabic Chat 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
EL
EleutherAI_pythia 6.9b Deduped__Sft__Tldr
Eleutherai · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.4 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
torchtune_1B_full_finetuned_llama3.2_millfield_241219_meta_header_word_3epoch
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
BE
beowolx CodeNinja 1.0 OpenChat 7B
Beowolx · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
LL
Lllma 3.2 1B
Lllma · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LO
LordNoah Alpaca refine gpt2 e1 se0
Lordnoah · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
PH
parsak phi 2 code instruct
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
LL
Nov2_Oasst_Aft_Llama_Lr_3E 5
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
AE
aetherwiing MN 12B Starsong v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
MI
gqd mistral merge 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
nbeerbower Lyra Gutenberg mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
YI
dolphin-2.9.1-yi-1.5-9b
Yi · RichardErkhov
Params8.83B
Context12K
Q4_K_M5.3 GB
GE
SangMoone merge gemma2 2b it difficulty hard
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
jan hq Llama3.1 s instruct 2024 09 10 v2 cp 1000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
EU
Eurus 7b Kto
Eurus · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 2 7b Chat Hf Gpt 3.5 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
CU
curiousily tiny crypto sentiment analysis
Curiousily · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
kajuyerim Llama 3.1 8B Instruct Financial Sentiment Analysis
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GI
Gille StrangeMerges 53 7B model stock
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MY
mychen76 openmixtral 6x7b v2
Mychen · RichardErkhov
Params35.4B
Context32K
Q4_K_M21.4 GB
LL
CodeLlama 7B Instruct Fp16
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
dalau627 text to sql finetuned mistral 7b
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
HE
Hello Bible Gabriel 8x7B Instruct v0.1
Hello · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB