GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
TI
TinyLlama-1.1b-karasu-merged
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
kavish218 finetuned llama 3 2 1B description multi domain 5
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
GP
gpt2-medium-stable-diffusion-prompt-generator
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
QW
Qwen2.5 0.5B
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
Tool use
PH
bunnycore Phi 3.5 mini TitanFusion 0.4
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
RE
Replete AI Replete Coder Instruct 8b Merged
Replete · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Coding
SO
Solar OrcaDPO Solar Instruct SLERP
Solar · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
AI Sweden Models Llama 3 8B instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
SG
sgraham merged archae metadata model
Sgraham · RichardErkhov
Params135M
Context2K
Q4_K_M0.1 GB
GGUF
SM
abhishek autotrain smollm2 135m finetune guanaco
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
GGUF
GP
gpt2-xl-conversational
Gpt · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
GGUF
GE
Gemma 2 2b Finance It v1
Gemma · RichardErkhov
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
SH
shanthi 323 fine tuned model dialogpt
Shanthi · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
LL
inswave AISquare Instruct llama2 koen 13b v0.9.13
Llama · RichardErkhov
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
LL
gradientai Llama 3 70B Instruct Gradient 524k
Llama · RichardErkhov
Params70.6B
Context512K
Q4_K_M42.5 GB
GGUF
SM
SmolPlatypus-1.5B-Sorted
Smolplatypus · RichardErkhov
Params1.54B
Context2K
Q4_K_M0.9 GB
GGUF
LL
Llama 3.1 8B Instruct Phishing Classification
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
BA
barc0 google cloude test 20k transduction gpt4omini lr1e 5 epoch2 seed50
Barc · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
LL
Llama 3.2 3b It Ecommerce ChatBot
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
LL
blueintell Llama 3.2 Korean Bllosom 3B Instruct cp finetuned
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
GE
Gemma 2b
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
CU
TechxGenus CursorCore DS 1.3B
Cursorcore · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
MI
nilq baby python mistral 1L tiny TinyStories ft
Mistral · RichardErkhov
Params35M
Context2K
Q4_K_M0.0 GB
GGUF
LL
ADT7 SN11 Llama3.1 8b v7
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MI
bullerwins Codestral 22B v0.1 hf
Mistral · RichardErkhov
Params22B
Context—
Q4_K_M0.0 GB
Coding
MI
dongyru Mistral 7B Claim Extractor
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Watashiha Llama 2 13B Ogiri Sft
Llama · RichardErkhov
Params13.1B
Context4K
Q4_K_M8.0 GB
GGUF
LL
Llama 2 7b Chat Hf
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
PH
Phi 3.5 Mini Instruct CPT D1_chosen Dpo Mix Shuffled5
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama-3-11B-Instruct-attenuated
Llama · RichardErkhov
Params11.5B
Context8K
Q4_K_M7.0 GB
GGUF

Keep browsing