GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
MI
Ae Calem Mistral 7b v0.1
Mistral · RichardErkhov
Params7.29B
Context32K
Q4_K_M4.4 GB
GGUF
GA
garage bAInd Stable Platypus2 13B
Garage · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
MI
Vezora Mistral 22B v0.2
Mistral · RichardErkhov
Params22.2B
Context64K
Q4_K_M13.3 GB
GGUF
GE
Gemma 2b Aps It
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
GE
zelk12 recoilme gemma 2 Ataraxy 9B v0.1 t0.25
Gemma · RichardErkhov
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
NE
nenad1002 quantum research bot v1.0
Nenad · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LO
LongAlpaca 70B
Longalpaca · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
HE
0 hero Matter 0.1 Slim 7B A
Hero · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GI
giux78 zefiro 7b beta ITA v0.1
Giux · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
WikiHow Mistral Instruct 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Granite 3b Code Base 2k
Granite · RichardErkhov
Params3.48B
Context2K
Q4_K_M2.1 GB
Coding
DE
DeepSeek Coder V2 Lite Instruct
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
Coding
GE
micks99 gemma 2b instruct ft Data Analytics Digital Marketing Project Management QAv2
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
nbeerbower bophades v2 mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
HA
Hare-1.1B-base
Hare · RichardErkhov
Params1.12B
Context32K
Q4_K_M0.7 GB
GGUF
OP
OpenHermes 2.5 Code 290k 13B
Openhermes · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
CodingTool use
BE
benhaotang nemo math science philosophy 12B
Benhaotang · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
DR
Dragonwar 7b Alpha
Dragonwar · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
openai community gpt2
Openai · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
BI
Bielik 11B v2.1 Instruct
Bielik · RichardErkhov
Params11.2B
Context32K
Q4_K_M6.7 GB
GGUF
ME
mergekit community mergekit task arithmetic jaowyfi
Mergekit · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5-0.5B-SFT-5e-5-3ep
Qwen · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
Tool use
LL
harborwater open llama 3b claude 30k
Llama · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
VI
vicuna-160m
Vicuna · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
LL
CDS CodeLlama 7b 20241113
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
philschmid code llama 3 1 8b text to sql
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Coding
PH
Phi 3.5 Mini investigator_16B
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
QW
jbaron34 Qwen2.5 0.5b bebop reranker
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Embedding

Keep browsing