GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
IN
InstructLM-1.3B
Instructlm · RichardErkhov
Params1.35B
Context2K
Q4_K_M0.8 GB
TE
TeeZee DarkForest 20B v1.1
Teezee · RichardErkhov
Params20B
Context4K
Q4_K_M0.7 GB
DE
Deepseek Coder 1.3b Instruct
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
QW
Phi 3.5 Mini Instruct
Qwen · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MO
MobiLlama 1B Chat
Mobillama · RichardErkhov
Params1.26B
Context2K
Q4_K_M0.8 GB
ML
mlfoundations dev oh v1.3 unnatural instructions x.125
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
MO
Mohan 08 gpt 2 guanaco finetuned
Mohan · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
LL
codellama CodeLlama 7b Instruct hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MI
wanadzhar913 malaysian mistral llmasajudge v3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral 7B Base SFT KTO
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DE
KSU HW SEC Deepseek SFT RL Text only
DeepSeek · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
MO
Monarch 7B
Monarch · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta-Llama-3-8B-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen2.5 Coder 1.5B
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
mlfoundations dev hp ablations mistral scheduler cosine warmup0.05
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
KI
Kiwi_Solar_Merge_Slerp_Test_V1
Kiwi · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
LL
nota ai st llama 1 5.5b taylor
Llama · RichardErkhov
Params5.52B
Context2K
Q4_K_M3.3 GB
MI
Mistral 7B v0.3
Mistral · RichardErkhov
Params7.25B
Context32K
Q4_K_M4.4 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
K2S3 Mistral 7b v1.42
Mistral · RichardErkhov
Params7.37B
Context32K
Q4_K_M4.5 GB
PH
Phi 3.5 Mini Instruct LoRA 128
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
MI
Hertz Mistral Hermes 2x7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
Xennon BD Qwen uncensored v2
Qwen · RichardErkhov
Params494M
Context128K
Q4_K_M0.4 GB
MI
MiniMA 3B
Minima · RichardErkhov
Params3.02B
Context4K
Q4_K_M1.8 GB
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 1960 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
NI
Nitral AI Nyan Stunna 7B
Nitral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GP
Gpt2_Medium_Tdata2_40_Epochs
Gpt · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
MI
EM German
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
ST
finetune_starcoder2_3b_tokens_added_6k_steps
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB