GGUF models by PrunaAI

95 GGUF models quantized by PrunaAI on Hugging Face. Sizes run from 10M to 141B parameters. The longest trained context is 1024K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
Phi 3 mini 4k instruct GGUF Imatrix smashed
Phi · PrunaAI
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
PH
Phi 3 mini 128k instruct GGUF Imatrix smashed
Phi · PrunaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
ZE
zephyr orpo 141b A35b v0.1 GGUF smashed
Zephyr · PrunaAI
Params141B
Context64K
Q4_K_M85.6 GB
GGUF
GP
gpt2 GGUF smashed
Gpt · PrunaAI
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
Meta Llama 3 8B Instruct GGUF Imatrix smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
dolphin 2.9 llama3 8b 256k GGUF smashed
Llama · PrunaAI
Params8B
Context512K
Q4_K_M4.9 GB
GGUF
LL
dolphin 2.9 llama3 8b GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 8B Instruct Gradient 1048k GGUF smashed
Llama · PrunaAI
Params8B
Context1024K
Q4_K_M4.9 GB
GGUF
MI
Mistral 7B Instruct v0.2 GGUF smashed
Mistral · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GP
openai community gpt2 GGUF smashed
Gpt · PrunaAI
Params124M
Context1K
Q4_K_M0.1 GB
GGUF
WI
WizardLM 2 7B GGUF smashed
Wizardlm · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
dolphin 2.9 llama3 70b GGUF smashed
Llama · PrunaAI
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
OpenBioLLM Llama3 70B GGUF smashed
Llama · PrunaAI
Params70.6B
Context8K
Q4_K_M42.5 GB
GGUF
LL
llava llama 3 8b v1 1 GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
Vision
LL
nvidia Llama3 ChatQA 1.5 8B GGUF smashed
Llama · PrunaAI
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LY
lynn 7b alpha GGUF smashed
Lynn · PrunaAI
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
llama3 42b v0 GGUF smashed
Llama · PrunaAI
Params43.2B
Context8K
Smallest16.3 GB
GGUF
PH
TroyDoesAI Phi 3 Context Obedient RAG GGUF smashed
Phi · PrunaAI
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
CO
codegemma 2b GGUF smashed
Codegemma · PrunaAI
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
CO
codegemma 7b it GGUF smashed
Codegemma · PrunaAI
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
LL
Meta Llama Guard 2 8B GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
gemma 1.1 2b it GGUF smashed
Gemma · PrunaAI
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Llama 3 8b 64k PoSE GGUF smashed
Llama · PrunaAI
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Unichat llama3 Chinese 8B GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
cognitivecomputations dolphin 2.9.1 llama 3 8b GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
GE
gemma 2b GGUF smashed
Gemma · PrunaAI
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
Meta Llama 3 8B Instruct GGUF smashed
Llama · PrunaAI
Params8B
Context8K
Q4_K_M4.9 GB
GGUF
LL
Llama 3 13B Instruct v0.1 GGUF smashed
Llama · PrunaAI
Params13.3B
Context8K
Q4_K_M8.1 GB
GGUF
MI
Mistral 22B v0.2 GGUF smashed
Mistral · PrunaAI
Params22.2B
Context64K
Q4_K_M13.3 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · PrunaAI
Params3.61B
Context128K
Q4_K_M2.2 GB
Tool use

About PrunaAI on Hugging Face

PrunaAI has published 95 GGUF models that ggufmodels.com tracks, across families such as Llama, Phi and Qwen. The most downloaded are Phi 3 mini 4k instruct GGUF Imatrix smashed, Phi 3 mini 128k instruct GGUF Imatrix smashed and zephyr orpo 141b A35b v0.1 GGUF smashed.

The quants PrunaAI publishes most often are Q5_K_M (95), Q4_K_S (95), Q3_K_S (95) and Q3_K_M (95). Their Q4_K_M files range from 0.0 to 85.6 GB. Parameter counts run from 10M to 141B, and the longest trained context window is 1024K tokens.

Frequently asked questions

What GGUF models has PrunaAI published?

95 models are listed here, mainly from the Llama, Phi and Qwen families. The most downloaded is Phi 3 mini 4k instruct GGUF Imatrix smashed.

Which quants does PrunaAI publish?

The most common are Q5_K_M (95 models), Q4_K_S (95 models), Q3_K_S (95 models) and Q3_K_M (95 models).

Keep browsing