Q3_K GGUF models
A GGUF quantization format. Lower numbers mean smaller files and more quality loss. 15171 models have a Q3_K file. Sizes run from 1M to 5632B parameters. The longest trained context is 102400K tokens.
CO
CodeV CL 7B
Codev · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
PH
Phi 3 Medium 4k Instruct Abliterated v3
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
PE
PersianLLaMA 13B
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
HE
Heejindo rationale model e3 save5000 rp f1
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
klcsp llama3.1 8b gpt4o 100k coding fft
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
AB
Abhinavanand21 gita text generation gpt2
Abhinavanand · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
mlfoundations dev hp ablations mistral scheduler inverse sqrt
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
EmTpro01 codellama 7b finetuned
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
FA
ssmits Falcon2 5.5B Polish
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.5B
Context128K
Q4_K_M1.0 GB
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 100 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MA
Math Chunk Refining Lm
Math · RichardErkhov
Params354M
Context2K
Q4_K_M0.2 GB
ML
mlfoundations dev oh v1.3 camel biology x.5
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
CDS CodeLlama 7b Instruct Hf 20240926
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GE
gemma-2b-data-std
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
LL
komt-llama2-7b-v1
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
YI
HenryJJ Instruct Yi 6B Dolly15K
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
MI
shenzhi wang Mistral 7B v0.3 Chinese Chat
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
MI
Speechless Mistral 7B v0.2 Mixed 1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
meta llama LlamaGuard 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
PH
tuong nguyen prd Phi 3.1 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
BC
Bce Reranker Base_V1
Bce · gpustack
Params278M
Context512
Q4_K_M0.2 GB
RO
rocket-3B
Rocket · RichardErkhov
Params2.8B
Context4K
Q4_K_M1.7 GB
DC
Dclm Perplexity Correlations Spearmanr No Samp 160m
Dclm · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
LL
ArliAI Llama 3 8B Instruct DPO v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GR
Granite 3.0 2b Instruct
Granite · RichardErkhov
Params2.63B
Context4K
Q4_K_M1.6 GB
GE
ecdev gemma 2b instruct ft medical qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GP
gpt2-medium-vaguely-human-dialogue
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
IS
ishmanish gpt2 autotrain text HrPolicy aug v2
Ishmanish · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB