GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
IN
InternLM2
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
QW
Qwen2.5 7b Instruct Cpt v1.1
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
MI
dolphin-2.6-mistral-7b-dpo-orca
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
hydra project ChatHercules 2.5 Mistral 7B
Mistral · RichardErkhov
Params470M
Context—
Q4_K_M0.3 GB
MI
Mistral 7B Base SFT RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AM
ambrosfitz TinyLlama 1.1B Chat yawp
Ambrosfitz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
SM
SmolLM2 135M
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
KU
Kukedlc Neural Krishna Multiverse 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
QW
Qwen1.5-0.5B-Chat
Qwen · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 400 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
LL
Llama2_Biomed_Meditron
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
LO
Local Novel LLM project Ninja v1 NSFW 128k
Local · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
mlfoundations dev mistral alpaca sft sample 4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
AIML GEEK mini prompt finetuned CodeLlama7b
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
CO
Yi Coder 1.5B Chat Jlama
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
VI
vitumafeni tiny crypto sentiment analysis
Vitumafeni · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
LL
Llama 3.2 3b It ComfyUIPrompt
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
SA
saishf SOVL Mega Mash V2 L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
kykim0 Llama 2 7b ultrachat200k 2e
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M2.0 GB
CO
Codegemma 2b
Codegemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
MI
speechless-mistral-7b-dare-0.85
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
DI
Distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
TI
TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
MI
abhishek autotrain mixtral7x8b math
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
LL
Llama 3 10b Ko 240604 E2F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
ML
mlfoundations dev oh dcft v1.2 no curation gpt 4o mini wo metamath
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
CodeLlama-13B-Instruct-fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB