GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
GE
lemon mint gemma ko 7b it v0.41
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
SI
Sixtyoneeighty 7b
Sixtyoneeighty · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DR
Dracarys2 72B Instruct
Dracarys · RichardErkhov
Params72.7B
Context32K
Q4_K_M47.4 GB
Tool use
LL
Configurable-Llama-3-8B-v0.2
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
ihughes15234 llama 3 1 8bi tictactoe dpo3epoch v2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
OP
OpenELM-270M
Openelm · RichardErkhov
Params272M
Context2K
Q4_K_M0.2 GB
GGUF
LL
PatronusAI Llama 3 Patronus Lynx 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
FU
FuseChat 7B VaRM
Fusechat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
MR
MrRobotoAI Hathor v3.3
Mrrobotoai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
OP
OpenELM-1_1B
Openelm · RichardErkhov
Params1.08B
Context2K
Q4_K_M0.7 GB
GGUF
YI
Yi-1.5-9B
Yi · RichardErkhov
Params8.83B
Context4K
Q4_K_M5.3 GB
GGUF
LL
LongWriter Llama3.1 8b
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
RA
RakutenAI-7B-chat
Rakutenai · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
LZ
lzw1008 Emollama 7b
Lzw · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
NE
NeuralMona_MoE-4x7B
Neuralmona · RichardErkhov
Params24.2B
Context32K
Q4_K_M14.6 GB
GGUF
WA
Wanfq FuseLLM 7B
Wanfq · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
AU
AutoCoder_S_6.7B
Autocoder · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
FL
flammenai flammen15 gutenberg DPO v1 7B
Flammenai · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
RESMPDEV Qwen2 Wukong 5B Instruct Sliced
Qwen · RichardErkhov
Params5.75B
Context32K
Q4_K_M3.6 GB
GGUF
AG
AgentEvol 7B
Agentevol · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
PO
Poro 34B Chat
Poro · RichardErkhov
Params35.1B
Context7K
Q4_K_M22.4 GB
GGUF
GE
thusinh1969 gemma2 2b rerank checkpoint 8800
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
Embedding
LL
scb10x llama 3 typhoon v1.5x 8b instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
TheDrummer Big Tiger Gemma 27B v1
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
GGUF
LL
Laim Llama 3.1 MedPalm2 imitate 8B Instruct
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
AL
alpindale pygmalion instruct
Alpindale · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
CH
CheckGPT SOLAR 10.7B
Checkgpt · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
Llama 3 6B v0
Llama · RichardErkhov
Params6.29B
Context8K
Q4_K_M3.9 GB
GGUF
PH
Phi 3.5 Mini instruct_Uncensored
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LE
Leia Swallow 13b
Leia · RichardErkhov
Params13.1B
Context4K
Q4_K_M7.9 GB
GGUF

Keep browsing