GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
gte-Qwen2-1.5B-instruct
Qwen · RichardErkhov
Params1.78B
Context128K
Q4_K_M1.1 GB
GGUF
ST
StableBeluga 7B
Stablebeluga · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
SU
SurgeGlobal OpenBezoar HH RLHF SFT
Surgeglobal · RichardErkhov
Params3.43B
Context2K
Q4_K_M2.6 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
YI
Yi 1.5 6B Chat
Yi · RichardErkhov
Params6.06B
Context4K
Q4_K_M3.7 GB
GGUF
VI
Vicuna 7b v1.3
Vicuna · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
MI
Kant-Test-0.1-Mistral-7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
Starling LM 7B Beta
Starling · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding
MI
microsoft_WizardLM-2-7B
Microsoft · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 3B Instruct Abliterated
Llama · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
GGUF
MI
Mistral 7b Sandbagging Cb New
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CG
cgato TheSpice 7b FT ExperimentalOrca
Cgato · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3 8B ProLong 64k Base
Llama · RichardErkhov
Params8B
Context64K
Q4_K_M4.9 GB
GGUF
GE
lei HuggingFace Gemma 2 9B 4bit Chat Level Measurement Guide 07242024
Gemma · RichardErkhov
Params3.4B
Context—
Q4_K_M1.7 GB
GGUF
ZE
ZeusLabs Chronos Divergence 33B
Zeuslabs · RichardErkhov
Params32.5B
Context16K
Q4_K_M19.6 GB
GGUF
UP
upstage SOLAR 10.7B Instruct v1.0
Upstage · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Openthai Llama Pretrained 7B
Llama · RichardErkhov
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
IN
Inv Konstanta V4 Alpha 7B
Inv · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
OP
OpenELM-270M-Instruct
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
LL
Taigi Llama 2 Translator 7B
Llama · RichardErkhov
Params6.94B
Context4K
Q4_K_M4.2 GB
GGUF
RE
ReflectionCoder DS 33B
Reflectioncoder · RichardErkhov
Params33.3B
Context16K
Q4_K_M19.9 GB
Coding
MI
Mistral 7B Instruct v0.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
UC
UCLA AGI zephyr 7b sft full SPIN iter0
Ucla · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mrm8488 mistral 7b ft AgentInstruct
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
K2S3 Mistral 7bx2 48layers_v1.2
Mistral · RichardErkhov
Params10.9B
Context32K
Q4_K_M6.6 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
AV
average-dolphin-8x7B
Average · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
PH
microsoft Phi 3 mini 128k instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
TO
tokyotech llm Swallow 13b instruct hf
Tokyotech · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF

Keep browsing