GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Lambent qwen2.5 reinstruct alternate lumen 14B
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
AL
AlphaMist7B Slr v2
Alphamist · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 8b Gemini Point21 60 Wo Cot
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Syouf llamaWithoutGuff
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Llama 2 7b Pruned70 Retrained
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Prosparse Llama 2 7b
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
CO
CompassJudger 1 32B Instruct
Compassjudger · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
GE
Google Gemma 2 27b
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
HA
haes95 cdlm 7 ko nl2sql v1.0
Haes · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
LL
smol_llama-220M-openhermes
Llama · RichardErkhov
Params218M
Context2K
Q4_K_M0.1 GB
Tool use
GE
Ruqiya gemma2 2b fine tuned arabic dataset
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.7
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
Meta Llama 3.1 70B Instruct
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
DI
diffusionfamily diffullama
Diffusionfamily · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
NX
Nxcode-CQ-7B-orpo
Nxcode · RichardErkhov
Params7.25B
Context64K
Q4_K_M4.7 GB
GGUF
JA
japanese-stablelm-instruct-ja_vocab-beta-7b
Japanese · RichardErkhov
Params6.88B
Context4K
Q4_K_M4.2 GB
GGUF
ME
Meltemi 7B Instruct v1
Meltemi · RichardErkhov
Params7.48B
Context32K
Q4_K_M4.5 GB
GGUF
LL
Devarui379 VersatiLlama Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
SA
SambaLingo Arabic Chat 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
GGUF
EL
EleutherAI_pythia 6.9b Deduped__Sft__Tldr
Eleutherai · RichardErkhov
Params6.86B
Context2K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
LL
torchtune_1B_full_finetuned_llama3.2_millfield_241219_meta_header_word_3epoch
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
BE
beowolx CodeNinja 1.0 OpenChat 7B
Beowolx · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
Lllma 3.2 1B
Lllma · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
PH
Phi 3 Mini 128k Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LO
LordNoah Alpaca refine gpt2 e1 se0
Lordnoah · RichardErkhov
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
PH
parsak phi 2 code instruct
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
Coding
LL
Nov2_Oasst_Aft_Llama_Lr_3E 5
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
Tool use
AE
aetherwiing MN 12B Starsong v1
Aetherwiing · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF

Keep browsing