GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
DE
ahmedheakl asm2asm deepseek 1.3b 500k bad x86 O0 arm gnueabi gcc
DeepSeek · RichardErkhov
Params1.35B
Context16K
Q4_K_M0.9 GB
GGUF
LL
andykcheng tv remote llama3 chinese
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
GGUF
BL
Bloom 560m
Bloom · RichardErkhov
Params816M
Context1K
Q4_K_M0.6 GB
GGUF
LL
sumeet hande llama 3.1 srh fine tuned 2.0
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
MI
mistralopithecus-v3-dpo-7b
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Nomath_Code_Rm_Llama2_Sft_Ver2_Iter1_Dpo
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
Coding
MA
Mamba 1.4b Instruct Hf
Mamba · RichardErkhov
Params1.37B
Context1024K
Q4_K_M0.8 GB
GGUF
GE
Gemma 7b Custom Tokenizer Base
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
MI
mlfoundations dev hp ablations mistral epoch3
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
GR
Grafted Hermetic Platypus A 2x7B
Grafted · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
MV
mvpmaster Einstein 4D MoE 2x7b test
Mvpmaster · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
LL
Lyte Llama 3.2 3B Overthinker
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
TI
TinyLlama-1.1B-32k-Instruct
Tinyllama · RichardErkhov
Params1.1B
Context32K
Q4_K_M0.7 GB
GGUF
LL
Llama 3 Open Ko 8B Instruct Sample
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
GE
at2507 gemma 2b instruct ft medical qa
Gemma · RichardErkhov
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
OL
OLMo 7B Instruct Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
GGUF
TI
TinyLlama-1.1B
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
lliu01 llama 3.2 3B adminguide
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
LL
itsnebulalol Llama 3.2 Nemotron 3B
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
GGUF
BL
Bloomz 560m
Bloomz · RichardErkhov
Params816M
Context2K
Q4_K_M0.6 GB
GGUF
LL
Llama 3.2 1B Instruct SFT D_chosen Pref Mix5
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
Tool use
DE
Srijith rkr deepseek base 1e 4 NO cot only failed samples 100 ckpt
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
GGUF
DA
H2O Danube3 4b Chat
Danube · RichardErkhov
Params3.96B
Context8K
Q4_K_M2.4 GB
GGUF
MI
RAG KO Mixtral 7Bx2 v2.1
Mistral · RichardErkhov
Params12.9B
Context32K
Q4_K_M7.8 GB
GGUF
LG
lgaalves gpt2 open platypus
Lgaalves · RichardErkhov
Params163M
Context1K
Q4_K_M0.1 GB
GGUF
LL
MinaMila Ext Mitig CFA llama 3 Base ImpFeat 1ep withsex
Llama · RichardErkhov
Params8B
Context128K
Q4_K_M4.9 GB
GGUF
OL
OLMo-7B-hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
GGUF
LL
Minirecord llama13b dpo loss0 OTL
Llama · RichardErkhov
Params13.2B
Context4K
Q4_K_M8.0 GB
GGUF
PH
crisp im mirage phi3 instruct rank
Phi · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.4 GB
GGUF
MI
shleeeee mistral ko OpenOrca 2000
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF

Keep browsing