GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
SM
SmolLM2 135M Instruct
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Vision
GP
Gpt2 Medium
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
GE
Gemma 2 2b It
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
PH
grounded ai phi3 rag relevance judge merge
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
LL
collaiborateorg Collaiborator MEDLLM Llama 3 8B v1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
OL
ai2 adapt dev olmo 7b peteish7 step928646
OLMo · RichardErkhov
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
FA
FairMind Minerva 3B Instruct v1.0
Fairmind · RichardErkhov
Params2.89B
Context16K
Q4_K_M1.8 GB
GGUF
LL
Llama 3.2 1B Instruct
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
LL
Llama 3 8B Instruct SPPO Score Iter2_gp_8b Table 0.002
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
hirushanan Code LLaMA 7b optimize code
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Llama 3 10b Wiki 240709 F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
GGUF
QW
Qwen1.5-1.8B-Chat
Qwen · RichardErkhov
Params1.84B
Context32K
Q4_K_M1.2 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
LL
suzume-llama-3-8B-multilingual-orpo-borda-top25
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
PH
Phi 3 Mini 4k Instruct
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.3 GB
GGUF
LL
Youngwon Llama 3.2 1B info 4bit
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
LL
Dolphin 2.9 Llama3 8b 256k
Llama · RichardErkhov
Params8.03B
Context512K
Q4_K_M4.9 GB
GGUF
LL
haqishen Llama 3 8B Japanese Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
mlfoundations dev llama3 1 8b oh v3.1 wo camel ai chemistry
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
Prosparse Llama 2 13b
Llama · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
PH
Phi 2 Electrical Engineering
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.8 GB
GGUF
DE
Srijith rkr deepseek base 1e 3 NO cot all samples 3 epoch
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
GR
Granite 3.0 8b Instruct
Granite · RichardErkhov
Params8.17B
Context4K
Q4_K_M4.9 GB
Tool use
TI
TinyLlama 1.1B Intermediate Step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
Coding
GR
GritLM 8x7B
Gritlm · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
SA
SambaLingo Arabic Base 70B
Sambalingo · RichardErkhov
Params69.4B
Context4K
Q4_K_M41.7 GB
GGUF

Keep browsing