GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
gqd mistral merge 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
nbeerbower Lyra Gutenberg mistral nemo 12B
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
YI
dolphin-2.9.1-yi-1.5-9b
Yi · RichardErkhov
Params8.83B
Context12K
Q4_K_M5.3 GB
GGUF
GE
SangMoone merge gemma2 2b it difficulty hard
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
jan hq Llama3.1 s instruct 2024 09 10 v2 cp 1000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
EU
Eurus 7b Kto
Eurus · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 2 7b Chat Hf Gpt 3.5 80k Base_Lora
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CU
curiousily tiny crypto sentiment analysis
Curiousily · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
kajuyerim Llama 3.1 8B Instruct Financial Sentiment Analysis
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GI
Gille StrangeMerges 53 7B model stock
Gille · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MY
mychen76 openmixtral 6x7b v2
Mychen · RichardErkhov
Params35.4B
Context32K
Q4_K_M21.4 GB
GGUF
LL
CodeLlama 7B Instruct Fp16
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
dalau627 text to sql finetuned mistral 7b
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
Tool use
HE
Hello Bible Gabriel 8x7B Instruct v0.1
Hello · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
PH
IntelLabs sqft phi 3 mini 4k 40 base
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
SA
SashaSheykina gpt2 finetuned cXg nl to code
Sashasheykina · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
Coding
LL
nnethercott llava v1.5 7b hf vicuna
LLaVA · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
Vision
GE
nbeerbower gemma2 gutenberg 27B
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LL
Llama 3 OffsetBias 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Mini 4k Grammar Correction
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
ML
mlfoundations dev oh v1 2 only evol instruct
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
unclemusclez Unsloth Qwen2.5 Coder 1.5B Devinator v1
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
OG
Ogno Monarch Jaskier Merge 7b
Ogno · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.1 Storm 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
JU
juvi21 Hermes 2 Theta L3 Euryale Ties 0.8 70B
Juvi · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
Tool use
PY
Pythia 1b Sft Hh
Pythia · RichardErkhov
Params1.01B
Context2K
Q4_K_M0.7 GB
GGUF
LL
transitionGap PROPERTYMUTATION IN Llama3.1 8B smallset
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
NL
NLPinas yi bagel 2x34b
Nlpinas · RichardErkhov
Params34.4B
Context195K
Q4_K_M12.9 GB
GGUF
ZE
zephyr-7b-full-ut-sft
Zephyr · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing