GGUF models by RichardErkhov

14354 GGUF models quantized by RichardErkhov on Hugging Face. Sizes run from 1M to 316B parameters. The longest trained context is 102400K tokens.

Most downloadedNewestLargestLongest contextSmallest
FA
FaizanPervaz stablecode alpha 3b finetuned
Faizanpervaz · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.8 GB
GGUF
OP
OpenHermes-2.5-Nebula-v2-7B
Openhermes · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
Tool use
PA
Pastiche Crown Clown 7b Dare
Pastiche · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
codellama CodeLlama 7b Python hf
Llama · RichardErkhov
Params6.74B
Context16K
Q4_K_M4.1 GB
Coding
LL
Phi 3 Mini 128k Instruct LLaMAfied
Llama · RichardErkhov
Params3.82B
Context128K
Q4_K_M2.3 GB
GGUF
LL
abhishek2475 Llama 3.2 3B Instruct finetuned
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
ME
mergekit community Mathing 1.5B
Mergekit · RichardErkhov
Params1.59B
Context32K
Q4_K_M1.0 GB
GGUF
MI
Mistral_Pro_8B_v0.1
Mistral · RichardErkhov
Params8.99B
Context32K
Q4_K_M5.4 GB
GGUF
QW
abdulmannan 01 qwen 2.5 3b finetuned for sql generation
Qwen · RichardErkhov
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
FU
FuseChat 7B v2.0
Fusechat · RichardErkhov
Params7.24B
Context8K
Q4_K_M4.4 GB
GGUF
LL
LDCC Instruct Llama 2 Ko 13B v4.1.8
Llama · RichardErkhov
Params13B
Context2K
Q4_K_M7.9 GB
GGUF
LL
GNER LLaMA 7B
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · RichardErkhov
Params1.54B
Context32K
Q4_K_M1.0 GB
CodingReasoningTool use
QW
FINGU AI Qwen2.5 14B Instruct Fine Tuned v3
Qwen · RichardErkhov
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
MI
mlfoundations dev hp ablations mistral adambeta1 0.85 dcftv1.2
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
LL
Llama 3.2 1B
Llama · RichardErkhov
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
MI
mlfoundations dev hp ablations grid mistral bsz512 lr2e 6 scheduler cosine warmup0.15
Mistral · RichardErkhov
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
QW
Qwen2.5 Math 7B Instruct
Qwen · RichardErkhov
Params7.62B
Context4K
Q4_K_M4.7 GB
Tool use
QW
Qwen2.5 1b
Qwen · RichardErkhov
Params1.54B
Context128K
Q4_K_M1.0 GB
GGUF
LL
realPCH 240103 llama test 2
Llama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
_Spydaz_Web_AI_Llava
LLaVA · RichardErkhov
Params7.24B
Context512K
Q4_K_M4.4 GB
Vision
RE
reflect_llm8B_llm-mstlrg-om2-80k_SFTt2_psdp-t02_b.5
Reflect · RichardErkhov
Params8.03B
Context128K
Q4_K_M4.9 GB
Tool use
PH
yuhonghhh microsoft phi 2 fine tune alpaca chinese merged model
Phi · RichardErkhov
Params2.78B
Context2K
Q4_K_M1.7 GB
GGUF
TI
TinyLlama TinyLlama 1.1B intermediate step 1431k 3T
Tinyllama · RichardErkhov
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
LL
brianmatzelle llama3.2 3b instruct hasanpiker abliterated
Llama · RichardErkhov
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
NO
Nous Hermes Platypus2 13B QLoRA 0.80 Epoch
Nous · RichardErkhov
Params13B
Context4K
Q4_K_M7.9 GB
Tool use
PY
Pythia 160m Deduped
Pythia · RichardErkhov
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
LL
llama-7b-sft-float32
Llama · RichardErkhov
Params6.74B
Context2K
Q4_K_M4.1 GB
GGUF
VI
Vinallama 2.7b Chat
Vinallama · RichardErkhov
Params2.77B
Context4K
Q4_K_M1.7 GB
GGUF
PS
Psyonic Cetacean V1 20B Ultra Quality Float32
Psyonic · RichardErkhov
Params20B
Context4K
Q4_K_M12.0 GB
GGUF

Keep browsing