GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Nous-Hermes-Llama2-13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
Tool use
MI
Mistral 7B Instruct v0.1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MELT-TinyLlama-1.1B-Chat-v1.0
Melt · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
OP
OpenELM 1_1B DPO Full Max Min Reward
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
QW
Rombos LLM V2.5 Qwen 32b
Qwen · RichardErkhov
Params32.8B
Context32K
Q4_K_M19.9 GB
Tool use
LL
ReBatch Llama 3 8B dutch
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
klcsp llama3 8b fft summarization 11 v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama2 oh teknium scaling down random 0.2
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
MI
Mistral 7B Instruct RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
PH
microsoft Phi 3 mini 4k instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
VI
Vizzy CA plates fine tuned gpt2 minimal test
Vizzy · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
GraySwanAI Mistral 7B Instruct RR
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OL
OLMoE 1B 7B 0924
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
GGUF
SY
Synatra_TbST02M_IN01
Synatra · RichardErkhov
Params10.7B
Context32K
Q4_K_M6.5 GB
GGUF
LL
Zhao Ching TWLLM Llama2 Extend VTLoss Enhanced
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
ME
MetaMath Cybertron
Metamath · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ME
MelangeB-70b
Melangeb · RichardErkhov
Params69B
Context4K
Q4_K_M41.4 GB
GGUF
GE
jstredacted gemma 2b instruct trivia qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
HU
ai human lab EEVE Korean Instruct 10.8B expo
Human · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Gemma 2b
Llama · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
LL
nsmagar llama 2 7b law korean
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
OP
Openchat_3.5 Gpt 4 80k
Openchat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
nbeerbower llama3.1 gutenberg 8B
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
TU
Tulu 2 Dpo 13b
Tulu · RichardErkhov
Params13B
Context8K
Q4_K_M7.9 GB
GGUF
QW
Qwen2.5 3B Instruct
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
SW
v000000 SwallowMaid 8B L3 SPPO abliterated
Swallowmaid · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CA
Calme-7B-Instruct-v0.1.1
Calme · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
failspy Llama 3 8B Instruct abliterated
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
CA
calme-2.3-legalkit-8b
Calme · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
Mistral Megamerge Dare 7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing