GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Mistral 7B Instruct v0.2 Function Calling
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
QW
elinas Qwen2 11.3B
Qwen · RichardErkhov
Params11.3B
Context128K
Q4_K_M6.9 GB
GGUF
QW
cofeg Finetuned Xunzi Qwen2 1.5B for ancient text generation
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
EX
Excalibur 7b
Excalibur · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
DE
AymanTarig deepseek coder 1.3b FC v2
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
DO
DongfuJiang prm version2 subsample hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
kimleang123 qwen 7b v2 khmer qa 1000 v10 merge model attn flash 2
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
KO
JY623 KoSOLAR 10.7B merge v3.0
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
ruslandev llama 3 8b gpt 4o ru1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SO
SOLAR 10.7B Instruct v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
SFR Iterative DPO LLaMA 3 8B R
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GL
Glm 4 9b Chat
GLM · RichardErkhov
Params9.4B
Context128K
Q4_K_M6.3 GB
Tool use
LL
yujiepan llama 3 tiny random
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
GGUF
SM
akhilfau fine tuned smolLM2 physics 10
SmolLM · RichardErkhov
Params362M
Context8K
Q4_K_M0.3 GB
GGUF
IN
internlm2-limarp-chat-20b
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
GE
Gemma 2 2b Jpn It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
GP
gpt2-small-italian-embeddings
Gpt · RichardErkhov
Params132M
Context1K
Q4_K_M0.1 GB
Embedding
CP
cpt_st-vicuna-v1.3-1.5b-ppl
Cpt · RichardErkhov
Params1.48B
Context2K
Q4_K_M0.9 GB
GGUF
MI
Nondzu Mistral 7B code 16k qlora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Coding
LL
tyson0420 stack codellama 7b inst
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
NO
nota ai st vicuna v1.3 10.5b taylor
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
GGUF
SH
Shwetaupadhyay7878 gita text generation gpt2
Shwetaupadhyay · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
Open_Hermes_Orca_Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
HA
harry_potter_chatbot
Harry · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
CS
csg-wukong-1B-chat-v0.1
Csg · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
PH
disi unibo nlp phi3.5 SFT medqa triples nographeval cot
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 math heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF

Keep browsing