GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Phi 3.5 Mini Instruct
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
PH
Phi 2 Layla v1 Chatml
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
YE
yejingfu MN 12B Starcannon v2
Yejingfu · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
YI
Faro Yi 34B
Yi · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
mia llm autotrain bloom 11b sft slm
Mia · RichardErkhov
Params1.45B
Context2K
Q4_K_M1.0 GB
GGUF
LL
jan hq Ichigo llama3.1 s instruct v0.4 step 8000
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
JA
Jais Adapted 7b Chat
Jais · RichardErkhov
Params7B
Context4K
Q4_K_M4.3 GB
GGUF
MI
Mistral-7B-Instruct-v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
SM
jbaron34 SmolLM2 135M Bebop Reranker
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Embedding
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
IN
interview eval zephyr 7b math train
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OR
Yi Ko 6B Orca Alpaca Gpt4 Math Lora
Orca · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
LL
RuterNorway Llama 2 13b chat norwegian
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
Mistral 7b v0.1 Layla v4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
ahmedheakl qwen2.5 0.5b llamafac 500k
Llama · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
GP
gpt2-medium-650k-stable-diffusion-prompt-generator
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
LL
georgesung llama2 7b chat uncensored
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
ST
Stablecode Completion Alpha 3b 4k
Stablecode · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
QW
Josiefied Qwen2.5 7B Instruct Abliterated
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
GE
rsrishi10 gemma text2sql Instruct Finetune test
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
GE
generalbakudan gemma 2b instruct ft medical qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
brunosdorneles llama3.1 8B ips realtime multilanguage
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
EV
Evillain StarDust 20B v0.2
Evillain · RichardErkhov
Params20B
Context4K
Q4_K_M4.4 GB
GGUF
GR
Granite 8b Qiskit
Granite · RichardErkhov
Params8.05B
Context4K
Q4_K_M4.9 GB
GGUF
GE
Gemma 2 9b It DPO
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
MI
mia llm autotrain bloom 17b sft slm
Mia · RichardErkhov
Params2.24B
Context4K
Q4_K_M1.5 GB
GGUF
CO
CodeV CL 7B
Codev · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
PH
Phi 3 Medium 4k Instruct Abliterated v3
Phi · RichardErkhov
Params14B
Context4K
Q4_K_M8.6 GB
GGUF
PE
PersianLLaMA 13B
Persianllama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
GGUF
HE
Heejindo rationale model e3 save5000 rp f1
Heejindo · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Roleplay

Keep browsing