GGUF models by RichardErkhov
14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
LL
Pair Preference Model LLaMA3 8B
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
GE
Gemma 7b
Gemma · RichardErkhov
Params8.54B
Context8K
Q4_K_M5.3 GB
PY
Pythia-160M-Deduped-Adventure
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
MI
OpenMath-Mistral-7B-v0.1-hf
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
AI
aialt MMedL3 MI
Aialt · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
MO
Monero WizardLM 30B Uncensored Guanaco SuperCOT 30b
Monero · RichardErkhov
Params32.5B
Context2K
Q4_K_M19.6 GB
FA
Falcon
Falcon · RichardErkhov
Params7.22B
Context2K
Q4_K_M5.0 GB
QW
Qwen2.5 Coder 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context32K
Q4_K_M4.7 GB
TR
trojblue gpt2 prompt upscaler v1
Trojblue · RichardErkhov
Params406M
Context1K
Q4_K_M0.3 GB
ST
starcoder2-3b
StarCoder · RichardErkhov
Params3.03B
Context16K
Q4_K_M1.9 GB
QW
online-dpo-qwen2-2
Qwen · RichardErkhov
Params494M
Context32K
Q4_K_M0.4 GB
FA
Falcon
Falcon · RichardErkhov
Params11.1B
Context2K
Q4_K_M6.8 GB
ET
ethzanalytics ai msgbot gpt2 XL dialogue
Ethzanalytics · RichardErkhov
Params1.64B
Context1K
Q4_K_M1.2 GB
LL
rasa cmd gen codellama 13b calm demo
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
CodeLlama 13b Instruct Hf
Llama · RichardErkhov
Params13B
Context16K
Q4_K_M7.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
MI
Bitnet-Mistral.0.2-330m-v0.2-grokfast-v2.9
Mistral · RichardErkhov
Params330M
Context4K
Q4_K_M0.2 GB
OL
OLMoE 1B 7B 0924 Instruct Base
OLMo · RichardErkhov
Params6.92B
Context4K
Q4_K_M4.2 GB
SM
SmolLM2-135M-Instruct-K12-5000
SmolLM · RichardErkhov
Params135M
Context8K
Q4_K_M0.1 GB
IN
interview eval zephyr 7b math case 8 fixed
Interview · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
MI
Mistral Instruct Mcqa Circuit Broken
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GR
Granite 3.0 3b A800M Base
Granite · RichardErkhov
Params3.37B
Context4K
Q4_K_M2.1 GB
RE
reflect_llm8B_om2-mstlrg-300k460k-llm33-130k-t12_SFTt1-DPOsPSDPt1_b2.0
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
LL
Ottis llama3.2 arcLoRaFT
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
LL
Meta Llama 3 8B Instruct
Llama · RichardErkhov
Params8.03B
Context8K
Q4_K_M4.9 GB
BL
Bloom 7b1
Bloom · RichardErkhov
Params8.1B
Context4K
Q4_K_M5.3 GB
LL
T3Q LLM Product T3Q LLM2 Solar 10.7B v1.0
Llm · RichardErkhov
Params10.7B
Context4K
Q4_K_M6.5 GB
QW
Qwen2.5 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB