GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
TI
TinyLlama 1.1B Chat v1.0
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
GE
gemma-2-2b-int-jp-lora3
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
DR
DreadPoor Satyr 7B Model Stock
Dreadpoor · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
ME
MetaMath Bagel 34b v0.2 C1500
Metamath · RichardErkhov
Params34.4B
Context195K
Q4_K_M20.7 GB
GGUF
MI
ChatHercules 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FR
freewheelin free solar instrunction v0.2
Freewheelin · RichardErkhov
Params10.9B
Context4K
Q4_K_M6.6 GB
GGUF
LL
Llama 2 7b chat_FFT_GSM8K
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
LL
T3Q-LLM1-Solar-10.8B-v1.0
Llm · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
JA
Jais Adapted 13b
Jais · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.1 GB
GGUF
MI
Komt Mistral 7b v1 Dpo Osy v1
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BR
braindao iq code evmind 0.5b instruct v0.2411.3 100
Braindao · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
CodingTool use
TA
taicheng zephyr 7b align scan 0.0 0.0 polynomial 2
Taicheng · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
SA
SambaLingo Hungarian Chat
Sambalingo · RichardErkhov
Params6.95B
Context4K
Q4_K_M4.2 GB
GGUF
DE
Srijith rkr deepseek base 1e 4 NO cot only failed samples 3 epoch
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
HK
hkss hk SOLAR 10.7B v1.2
Hkss · RichardErkhov
Params10.8B
Context4K
Q4_K_M6.5 GB
GGUF
LL
LLM360 Amber
Llm · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
LL
Replete AI Replete Coder Llama3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Coding
AI
AIFT Ko Orca Plat Yi Ko 6b v1.2 Dpo 3
Aift · RichardErkhov
Params6.18B
Context2K
Q4_K_M3.8 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
SA
SanjiWatsuki Loyal Macaroni Maid 7B
Sanjiwatsuki · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
LLaMA 3.1 8B IT GraphRAG Finetuned
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
nbeerbower phi3.5 gutenberg 4B
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama 3.2 3B Instruct
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
ML
mlfoundations dev oh v1.3 camel math x8
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
firqaaa indo gemma 2b alpaca
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SO
SOLAR 10.7B Instruct T3Q Ko Solar Dpo Ties
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
MA
matsant01 STEMerald 2b
Matsant · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
SD
Sdff Ltba LightChatAssistant TypeB 2x7B
Sdff · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF

Keep browsing