Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
CA
Calme-7B-Instruct-v0.1.1
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
failspy Llama 3 8B Instruct abliterated
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
VI
it-5.3-fp16-32k
Vikhrmodels · Vikhrmodels
Params7.63B
Context32K
Q4_K_M4.6 GB
CA
calme-2.3-legalkit-8b
Calme · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MI
Mistral Megamerge Dare 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AL
Alphacode AI AlphaMist7B slr v4 slow
Alphacode · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Instruct IPO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SA
SanjiWatsuki neural chat 7b v3 3 wizardmath dare me
Sanjiwatsuki · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
StopCarbon 10.7B v5
Stopcarbon · andrijdavid
Params10.7B
Context4K
Q4_K_M6.5 GB
PY
pythia1b-sft-tldr
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
DA
DAVinCI Yi Ko 6B v0.8
Davinci · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
AR
arcee ai raspberry 3B
Arcee · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
LL
RohitSuresh15 llama3.2 1b medical v1
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
SR
sravankumarmsb gita text generation gpt2
Sravankumarmsb · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
OP
openbuddy-yi1.5-9b-v21.1-32k
Openbuddy · RichardErkhov
Params8.85B
Context32K
Q4_K_M5.3 GB
JA
Japanese Starling ChatV 7B
Japanese · RichardErkhov
Params7.24B
Context4K
Q4_K_M4.4 GB
QW
maars505 trained qwen2 dpo model5 deepspeed
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
MI
KoboldAI Mistral 7B Erebus v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
SO
SOLAR 10.7B Merge Dpo_V1
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
ST
Opencsg Starcoder2 3b v0.1
StarCoder · RichardErkhov
Params3.18B
Context16K
Q4_K_M2.0 GB
AR
Arkana08 NIHAPPY L3.1 8B v0.09
Arkana · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
IM
z image big love v3
Image · gimly13
Params6.15B
Context40K
Q4_K_M2.5 GB
RO
Rohan5076 medquad text generation gpt2
Rohan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
OP
openchat openchat 3.5 1210
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
ST
Stablelm 2 1_6b
Stablelm · RichardErkhov
Params1.64B
Context4K
Q4_K_M1.0 GB
QW
Qwen2.5-0.5B-SFT-1e-5-5ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
GI
Gille StrangeMerges 7 7B slerp
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Vezora Mistral 14b Merge Base
Mistral · RichardErkhov
Params14.2B
Context32K
Q4_K_M8.6 GB
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 200 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llama3 ChatQA 1.5 8B
Llama · Monor
Params8.03B
Context8K
Q4_K_M4.9 GB