GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
jasonhwan phi3 redteamer
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
MiniLLM Gpt2 120M
Minillm · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
inswave AISquare Instruct llama2 koen 13b v0.9.5
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
PH
Phi 3.5 Mini Instruct
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
MI
AlexeyRyzhikov mistral7b text to json v5.20
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
PH
fractalego wafl phi3 mini 4k
Phi · RichardErkhov
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
MI
mistral-7b-ft-h4-no_robots_instructions
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GL
Glm 4 9b
GLM · RichardErkhov
Params9.4B
Context8K
Q4_K_M6.3 GB
GGUF
TA
Taiwan LLM 13B v2.0 Chat
Taiwan · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
GGUF
TO
toastloaf autotrain gpt2 finetune crab
Toastloaf · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
SM
SmolLM2-135M-FT-SCP-Wiki
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
ST
Starcoder2 3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
SM
SmolLM 1.7B Instruct
SmolLM · RichardErkhov
Params1.71B
Context2K
Q4_K_M1.1 GB
GGUF
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange proofassistants
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
GE
gemma-2b-translation-v0.150
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
Qwen2 7B Instruct Refine
Qwen · RichardErkhov
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
AI
aisha partha medical question and answer gpt2
Aisha · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
MI
OpenHermes 2.5 Mistral 7B
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
MA
Magnum Instruct DPO
Magnum · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
Tool use
GE
Gemma-Wukong-2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
QW
terry69 qwen feedback dirty 0.5
Qwen · RichardErkhov
Params464M
Context32K
Q4_K_M0.3 GB
GGUF
MI
Mistral Nemo Instruct 2407
Mistral · RichardErkhov
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
LL
Llama 3.2 Korean Bllossom 3B
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
DI
distilgpt2-tiny-conversational
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
GGUF
DE
Deepseek Coder 7b Base v1.5
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
Coding
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05 minlr1e 6 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
BI
bitnet_b1_58-3B
Bitnet · RichardErkhov
Params3.32B
Context2K
Q4_K_M2.5 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF

Keep browsing