GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Dumi Llama3.2 3b Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
FA
ssmits Falcon2 5.5B Dutch
Falcon · RichardErkhov
Params5.47B
Context2K
Q4_K_M3.4 GB
GGUF
LL
lianghsun Llama 3.2 Taiwan 3B
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
MU
mudler Asinello Minerva 3B v0.1
Mudler · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
NU
Nutanix checkpoint 1800 merged
Nutanix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
TI
tinyllama-1.1B-intermediate-step-715k-1.5T-dpo-lora-merged
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
PH
duy nq phi 3 mini 4k merge v4
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
yjwon mp mistral7b sft1 b512 lr5e 6
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
BFS Search mistral multi rel I2B2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama-2-7b-instruct
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
GP
Gpt2 Large
Gpt · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
LL
Llama-2-13b-chat-longlora-32k-sft
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 7
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CH
ChatAYT Lora Assamble Marcoroni
Chatayt · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MA
Magpie Align MagpieLM 8B SFT v0.1
Magpie · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
KO
KoSOLAR-10.7B-v0.2
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
LLama 3.1 8B HyperNova
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
EL
Eleuther Pythia70M Hh Dpo
Eleuther · RichardErkhov
Params70M
Context2K
Q4_K_M0.0 GB
GGUF
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
GR
Iq Code Evmind v1 Granite 8b Instruct
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
Coding
LL
mlfoundations dev llama2 oh teknium scaling down random 0.5
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ED
edbeeching EleutherAI pythia 2.8b
Edbeeching · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
GP
Gpt2 Cia World Factbook
Gpt · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
ZE
zephyr-beta-Nebula-v2-7B
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
DeepFeatTextEmbeddingLarge
Deepfeattextembeddinglarge · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
Embedding
ZE
zephyr-7b-dpo-full-judge-before
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B Instruct Zh De Block
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
QW
psneto qwen2 arithmetic 2T FT
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF

Keep browsing