GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
MI
Mistral 7B Instruct v0.2 Function Calling
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
QW
elinas Qwen2 11.3B
Qwen · RichardErkhov
Params11.3B
Context128K
Q4_K_M6.9 GB
QW
cofeg Finetuned Xunzi Qwen2 1.5B for ancient text generation
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
EX
Excalibur 7b
Excalibur · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Llama 7b
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
DE
AymanTarig deepseek coder 1.3b FC v2
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
DO
DongfuJiang prm version2 subsample hf
Dongfujiang · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
QW
kimleang123 qwen 7b v2 khmer qa 1000 v10 merge model attn flash 2
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
KO
JY623 KoSOLAR 10.7B merge v3.0
Kosolar · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
LL
ruslandev llama 3 8b gpt 4o ru1.0
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
SO
SOLAR 10.7B Instruct v1.0
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
LL
SFR Iterative DPO LLaMA 3 8B R
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GL
Glm 4 9b Chat
GLM · RichardErkhov
Params9.4B
Context128K
Q4_K_M6.3 GB
LL
yujiepan llama 3 tiny random
Llama · RichardErkhov
Params14M
Context—
Q4_K_M0.0 GB
SM
akhilfau fine tuned smolLM2 physics 10
SmolLM · RichardErkhov
Params362M
Context8K
Q4_K_M0.3 GB
IN
internlm2-limarp-chat-20b
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GE
Gemma 2 2b Jpn It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GE
Gemma 2 27b It
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GP
gpt2-small-italian-embeddings
Gpt · RichardErkhov
Params132M
Context1K
Q4_K_M0.1 GB
CP
cpt_st-vicuna-v1.3-1.5b-ppl
Cpt · RichardErkhov
Params1.48B
Context2K
Q4_K_M0.9 GB
MI
Nondzu Mistral 7B code 16k qlora
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
tyson0420 stack codellama 7b inst
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
NO
nota ai st vicuna v1.3 10.5b taylor
Nota · RichardErkhov
Params10.5B
Context2K
Q4_K_M6.3 GB
SH
Shwetaupadhyay7878 gita text generation gpt2
Shwetaupadhyay · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
MI
Open_Hermes_Orca_Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
HA
harry_potter_chatbot
Harry · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
CS
csg-wukong-1B-chat-v0.1
Csg · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
PH
disi unibo nlp phi3.5 SFT medqa triples nographeval cot
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
PH
IntelLabs sqft sparsepeft phi 3 mini 4k 50 math heu
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB