GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
IN
InternLM2
InternLM · RichardErkhov
Params19.9B
Context32K
Q4_K_M12.0 GB
GGUF
QW
Qwen2.5 7b Instruct Cpt v1.1
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
Tool use
MI
dolphin-2.6-mistral-7b-dpo-orca
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
hydra project ChatHercules 2.5 Mistral 7B
Mistral · RichardErkhov
Params470M
Context—
Q4_K_M0.3 GB
GGUF
MI
Mistral 7B Base SFT RRHF
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
AM
ambrosfitz TinyLlama 1.1B Chat yawp
Ambrosfitz · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
SM
SmolLM2 135M
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
Coding
KU
Kukedlc Neural Krishna Multiverse 7b
Kukedlc · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen1.5-0.5B-Chat
Qwen · RichardErkhov
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.1 8B Lexi Uncensored V2
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
M1LL1X microsoft phi 3 5 mini trained abliterated QA 400 all modules merged
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
LL
Llama2_Biomed_Meditron
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
LO
Local Novel LLM project Ninja v1 NSFW 128k
Local · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
MI
mlfoundations dev mistral alpaca sft sample 4
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
AIML GEEK mini prompt finetuned CodeLlama7b
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
CO
Yi Coder 1.5B Chat Jlama
Coder · RichardErkhov
Params1.48B
Context128K
Q4_K_M1.0 GB
Coding
VI
vitumafeni tiny crypto sentiment analysis
Vitumafeni · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
Llama 3.2 3b It ComfyUIPrompt
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
SA
saishf SOVL Mega Mash V2 L3 8B
Saishf · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
LL
kykim0 Llama 2 7b ultrachat200k 2e
Llama · RichardErkhov
Params6.74B
Context4K
Q4_K_M2.0 GB
GGUF
CO
Codegemma 2b
Codegemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
MI
speechless-mistral-7b-dare-0.85
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
DI
Distilgpt2
Distilgpt · RichardErkhov
Params121M
Context1K
Q4_K_M0.1 GB
Coding
TI
TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
MI
abhishek autotrain mixtral7x8b math
Mistral · RichardErkhov
Params46.7B
Context32K
Q4_K_M28.4 GB
GGUF
LL
Llama 3 10b Ko 240604 E2F
Llama · RichardErkhov
Params9.78B
Context8K
Q4_K_M6.0 GB
GGUF
ML
mlfoundations dev oh dcft v1.2 no curation gpt 4o mini wo metamath
Mlfoundations · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
LL
CodeLlama-13B-Instruct-fp16
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
Coding

Keep browsing