GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
InstructLM-1.3B
Instructlm · RichardErkhov
Params1.35B
Context2K
Q4_K_M0.8 GB
GGUF
TE
TeeZee DarkForest 20B v1.1
Teezee · RichardErkhov
Params20B
Context4K
Q4_K_M0.7 GB
GGUF
DE
Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
Coding
QW
Phi 3.5 Mini Instruct
Qwen · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
Reasoning
MO
MobiLlama 1B Chat
Mobillama · RichardErkhov
Params1.26B
Context2K
Q4_K_M0.8 GB
GGUF
ML
mlfoundations dev oh v1.3 unnatural instructions x.125
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
MO
Mohan 08 gpt 2 guanaco finetuned
Mohan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
codellama CodeLlama 7b Instruct hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
MI
wanadzhar913 malaysian mistral llmasajudge v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
Mistral 7B Base SFT KTO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DE
KSU HW SEC Deepseek SFT RL Text only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
GGUF
MO
Monarch 7B
Monarch · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-8B-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingTool use
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
KI
Kiwi_Solar_Merge_Slerp_Test_V1
Kiwi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
GGUF
LL
nota ai st llama 1 5.5b taylor
Llama · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
GGUF
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
GGUF
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
GGUF
MI
K2S3 Mistral 7b v1.42
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
GGUF
PH
Phi 3.5 Mini Instruct LoRA 128
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
Hertz Mistral Hermes 2x7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
QW
Xennon BD Qwen uncensored v2
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
GGUF
MI
MiniMA 3B
Minima · RichardErkhov
Params3.02B
Context4K
Q4_K_M1.8 GB
GGUF
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 1960 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
NI
Nitral AI Nyan Stunna 7B
Nitral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GP
Gpt2_Medium_Tdata2_40_Epochs
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
GGUF
MI
EM German
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
ST
finetune_starcoder2_3b_tokens_added_6k_steps
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
Coding
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use

Keep browsing