Q4_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 16028 models have a Q4_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
QW
terry69 qwen 0.5 feedback dirty
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
PR
Preethi 1995 NEW MERGED MODEL
Preethi · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.37B
Context128K
Q4_K_M0.9 GB
PO
PowerLM 3b
Powerlm · RichardErkhov
Params3.51B
Context4K
Q4_K_M2.1 GB
YI
NobodyExistsOnTheInternet Yi 34B GiftedConvo merged
Yi · RichardErkhov
Params34.4B
Context4K
Q4_K_M20.7 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.10 minlr1e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev hp ablations mistral scheduler constant
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
rahuldshetty tiny starcoder instruct
StarCoder · RichardErkhov
Params164M
Context8K
Q4_K_M0.1 GB
NO
nomic embed text v2 moe
Nomic · cstr
Params475M
Context—
Smallest0.4 GB
MI
Nondzu Mistral 7B codealpaca lora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
DopeorNope Mistralopithecus v0.1 10.8B
Mistral · RichardErkhov
Params10.8B
Context32K
Q4_K_M6.5 GB
RE
Rendoo_Luman0002_05_45
Rendoo · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
archangel_sft_llama7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
ME
mergekit community config smart ablit
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2-7B-Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
LL
Orenguteng Llama 3 8B LexiFun Uncensored V1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
Llama 13b
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
MI
NousResearch Nous Hermes 2 Mixtral 8x7B DPO
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
nota ai st llama 1 5.5b ppl
Llama · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
BR
BroAlanTaps GPT2 large 128 10000steps
Broalantaps · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
KO
Tinyllama 2B
Kobbletinyv · Aculi
Params2.07B
Context2K
Smallest0.8 GB
RE
realPCH kosolra wiki QA
Realpch · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
LL
llama3-8b-math-sft-mix-2-0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
jpalgo llama 3.1 sl12k overwrite v1
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
Qwen2.5 Coder 7B
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
YI
Yi 1.5 9B Chat
Yi · RichardErkhov
Params8.83B
Context8K
Q4_K_M5.3 GB
ST
StarCoder
StarCoder · RichardErkhov
Params1.14B
Context8K
Q4_K_M0.8 GB
MA
mamba-2.8b-hf
Mamba · jpodivin
Params2.77B
Context1024K
Smallest1.4 GB
PH
Phi 3.5 Mini IFRS Accountant
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB