GGUF models by BoscoTheDog
58 GGUF models quantized by BoscoTheDog on Hugging Face. Sizes run from 121M to 9.24B parameters. The longest trained context is 128K tokens.
PO
poetrybot-gpt2-large
Poetrybot · BoscoTheDog
Params838M
Context1K
Q4_K_M0.5 GB
FI
fietje 3 mini 4k instruct Q4 K M gguf chunked
Fietje · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.4 GB
DA
danube3-4B
Danube · BoscoTheDog
Params3.96B
Context8K
Q4_K_M4.8 GB
AP
Apollo medical
Apollo · BoscoTheDog
Params2.51B
Context8K
Q4_K_M1.6 GB
AP
Apollo-6B
Apollo · BoscoTheDog
Params6.06B
Context4K
Smallest7.0 GB
LL
llama3 therapybeagle chunked
Llama · BoscoTheDog
Params8.03B
Context8K
Smallest3.2 GB
ON
oneirogen-1.5B
Oneirogen · BoscoTheDog
Params1.54B
Context128K
Smallest1.9 GB
PH
phi 3 mini 128k instruct ru
Phi · BoscoTheDog
Params3.82B
Context128K
Smallest1.7 GB
UK
Ukranian Q2 chunked
Ukranian · BoscoTheDog
Params7.24B
Context32K
Smallest2.7 GB
MI
ministral it 3b q5 k m chunked
Mistral · BoscoTheDog
Params3.32B
Context128K
Smallest2.3 GB
MI
open buddy mistral 7B 32k chunked
Mistral · BoscoTheDog
Params7.28B
Context32K
Smallest4.4 GB
PH
Phi 3.1 mini 4k instruct Q4 K M gguf chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.4 GB
GE
Gemma 2 2b It
Gemma · BoscoTheDog
Params2.61B
Context8K
Smallest1.6 GB
MI
bio mistral xxs
Mistral · BoscoTheDog
Params7.24B
Context32K
Smallest2.0 GB
CO
cooking
Cooking · BoscoTheDog
Params121M
Context1K
Smallest0.1 GB
FI
fietje 2b instruct Q5 K M chunked
Fietje · BoscoTheDog
Params2.78B
Context2K
Smallest2.0 GB
BI
bitnet_b1_58-3B
Bitnet · BoscoTheDog
Params3.32B
Context2K
Q4_K_M5.0 GB
GE
gemma mental health 2b gguf chunked
Gemma · BoscoTheDog
Params2.51B
Context8K
Smallest5.3 GB
QW
code qwen 1 5 7B chunked
Qwen · BoscoTheDog
Params7.25B
Context64K
Smallest3.5 GB
MI
inspire mistral 7B 32K chunked
Mistral · BoscoTheDog
Params7.24B
Context32K
Smallest3.2 GB
PH
Phi 3 mini 4k instruct q4 gguf chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Smallest2.4 GB
RE
RedPajama INCITE Chat 3B gguf chunked
Redpajama · BoscoTheDog
Params2.78B
Context2K
Smallest1.6 GB
PH
bling phi 3 RAG chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Smallest2.4 GB
TI
TinyLlama-1.1B-Chat-v1.0
Tinyllama · BoscoTheDog
Params1.1B
Context2K
Smallest0.6 GB
NE
neuralReyna mini 1.8b Q5 chunked
Neuralreyna · BoscoTheDog
Params1.84B
Context32K
Smallest1.4 GB
FI
fietje 2b chat Q5 K M chunked
Fietje · BoscoTheDog
Params2.78B
Context2K
Smallest2.0 GB
PH
phi3 mini 128k gguf chunked
Phi · BoscoTheDog
Params3.82B
Context128K
Smallest2.2 GB
QW
Qwen2 1.5B Instruct Q4 K M chunked
Qwen · BoscoTheDog
Params1.54B
Context32K
Q4_K_M1.0 GB