GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
AN
anthracite org magnum v2 32b
Anthracite · RichardErkhov
Params32.5B
Context32K
Q4_K_M19.7 GB
QW
MAmmoTH2 8B Plus
Qwen · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
OP
OpenLLM Ro RoLlama2 7b Instruct 2024 05 14
Openllm · RichardErkhov
Params6.74B
Context4K
Q4_K_M4.1 GB
MI
Mistral 7B v0.1 Half Naive A
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
QW
finetune-Qwen2-0.5B-codeDataset
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
LL
Meta-Llama-3-8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
LL
llama-3-experiment-v1-9B
Llama · RichardErkhov
Params8.9B
Context8K
Q4_K_M5.4 GB
DE
DeepSeek V2 Lite
DeepSeek · RichardErkhov
Params15.7B
Context160K
Q4_K_M10.4 GB
GE
1 800 SHARED TASKS GEMMA2 27B NLI 16bit
Gemma · RichardErkhov
Params27.2B
Context8K
Q4_K_M16.6 GB
DE
Deepseek Math 7b Rl
DeepSeek · RichardErkhov
Params6.91B
Context4K
Q4_K_M4.2 GB
MI
Mistral NeMo Minitron 8B Base
Mistral · RichardErkhov
Params8.41B
Context8K
Q4_K_M5.1 GB
LI
lightblue reranker 0.5 cont
Lightblue · RichardErkhov
Params630M
Context32K
Q4_K_M0.5 GB
CU
CultriX NeuralCeptrix 7B SLERP
Cultrix · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
StructLM 7B Mistral
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
LL
NousResearch Hermes 2 Pro Llama 3 70B
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
LL
Llama3-70B-Fireplace
Llama · RichardErkhov
Params70.6B
Context8K
Q4_K_M42.5 GB
AR
aragpt2-base
Aragpt · RichardErkhov
Params184M
Context1K
Q4_K_M0.1 GB
MI
Mistral Large Instruct 2407
Mistral · RichardErkhov
Params123B
Context128K
Q4_K_M73.2 GB
LL
dakshvar22 cmd gen travel assistant codellama 7b unsloth params
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
LL
mlfoundations dev llama3 1 8b mlfoundations dev stackexchange reverseengineering
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
dfurman Llama 3 8B Orpo v0.1
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
DH
Dhana8907 labsmergedModel0312
Dhana · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
tchen175 llama3.1 8b per sent
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Taigi Llama 2 Translator 13B
Llama · RichardErkhov
Params13.3B
Context4K
Q4_K_M8.0 GB
LA
lamm mit PRefLexOR ORPO DPO EXO 10242024
Lamm · RichardErkhov
Params3.61B
Context128K
Q4_K_M2.2 GB
LL
aifeifei798 llama3 8B DarkIdol 2.1 Uncensored 32K
Llama · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Llama-3-8B-Ultra-Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
OL
OLMo 7B Twin 2T Hf
OLMo · RichardErkhov
Params6.89B
Context2K
Q4_K_M4.2 GB
DA
Darkknight535 OpenCrystal 15B L3 v3
Darkknight · RichardErkhov
Params15B
Context8K
Q4_K_M9.1 GB
LL
Hermes 3 Llama 3.1 70B
Llama · RichardErkhov
Params70.6B
Context128K
Q4_K_M42.5 GB