GGUF models by Felladrin
267 GGUF models quantized by Felladrin on Hugging Face. Sizes run from 23M to 229B parameters. The longest trained context is 1024K tokens.
GE
Gemma 2 2b It
Gemma · Felladrin
Params2.61B
Context8K
Smallest3.8 GB
BG
gguf Q8 0 bge reranker v2 m3
BGE · Felladrin
Params568M
Context8K
Smallest0.6 GB
MI
Minueza-3-95M-RP
Minueza · Felladrin
Params95M
Context8K
Smallest0.1 GB
LA
gguf LaMini Flan T5 248M
Lamini · Felladrin
Params248M
Context512
Q4_K_M0.2 GB
WI
gguf WizardVicuna pythia 410m deduped
Wizardvicuna · Felladrin
Params405M
Context2K
Q4_K_M0.4 GB
PH
Phi 3.5 Mini Instruct
Phi · Felladrin
Params3.82B
Context128K
Smallest2.8 GB
LL
gguf Qwen1.5 0.5B Chat llamafy
Llama · Felladrin
Params620M
Context4K
Q4_K_M0.4 GB
NA
NanoLM 0.3B Instruct v2
Nanolm · Felladrin
Params315M
Context128K
Smallest0.3 GB
TI
gguf Tinyllama 616M Cinder
Tinyllama · Felladrin
Params616M
Context2K
Q4_K_M0.4 GB
TI
gguf TinyLlama 1.1B 1T OpenOrca
Tinyllama · Felladrin
Params1.1B
Context2K
Smallest0.5 GB
SM
SmolLM 360M
SmolLM · Felladrin
Params362M
Context2K
Smallest0.6 GB
DO
gguf DopeyTinyLlama 1.1B v1
Dopeytinyllama · Felladrin
Params1.1B
Context2K
Q4_K_M0.7 GB
QW
gguf Qwen2 0.5B Instruct
Qwen · Felladrin
Params494M
Context32K
Q4_K_M0.4 GB
OL
gguf OLMoE 1B 7B 0924 Instruct
OLMo · Felladrin
Params6.92B
Context4K
Smallest2.7 GB
IN
InternLM2
InternLM · Felladrin
Params1.89B
Context32K
Q4_K_M1.2 GB
IP
gguf IPythia 410m
Ipythia · Felladrin
Params405M
Context2K
Q4_K_M0.3 GB
GE
Gemma 3 4b It
Gemma · Felladrin
Params3.88B
Context128K
Smallest2.4 GB
GE
gguf sharded gemma 2b orpo
Gemma · Felladrin
Params2.51B
Context8K
Q4_K_M1.6 GB
GP
gguf gpt2 chatbot
Gpt · Felladrin
Params1.64B
Context1K
Q4_K_M1.2 GB
DA
H2O Danube3 500m Chat
Danube · Felladrin
Params514M
Context8K
Smallest0.7 GB
SM
gguf SmolLM 360M Instruct
SmolLM · Felladrin
Params362M
Context2K
Smallest0.2 GB
PH
gguf sharded phi 2 orange v2
Phi · Felladrin
Params2.78B
Context2K
Smallest2.0 GB
YI
Yi 1.5 6B Chat
Yi · Felladrin
Params6.06B
Context4K
Q4_K_M3.7 GB
FL
gguf sharded flan t5 large
Flan · Felladrin
Params783M
Context512
Q4_K_M0.6 GB
MO
gguf sharded MobileLLaMA 1.4B Chat
Mobilellama · Felladrin
Params1.36B
Context2K
Q4_K_M0.8 GB
OL
gguf zephyr 1b olmo sft qlora
OLMo · Felladrin
Params1.18B
Context2K
Q4_K_M0.7 GB
GR
Granite 3.0 2b Instruct
Granite · Felladrin
Params2.63B
Context4K
Smallest1.9 GB
CO
gguf BF16 Comet 60B A12B
Comet · Felladrin
Params60.3B
Context1024K
Smallest120.7 GB
TI
gguf TinyMistral 248M SFT v4
Tinymistral · Felladrin
Params248M
Context32K
Q4_K_M0.2 GB
FL
gguf flan t5 large
Flan · Felladrin
Params783M
Context512
Q4_K_M0.6 GB