GGUF models by BoscoTheDog

58 GGUF models quantized by BoscoTheDog on Hugging Face. Sizes run from 121M to 9.24B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
PO
poetrybot-gpt2-large
Poetrybot · BoscoTheDog
Params838M
Context1K
Q4_K_M0.5 GB
GGUF
FI
fietje 3 mini 4k instruct Q4 K M gguf chunked
Fietje · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
DA
danube3-4B
Danube · BoscoTheDog
Params3.96B
Context8K
Q4_K_M4.8 GB
GGUF
AP
Apollo medical
Apollo · BoscoTheDog
Params2.51B
Context8K
Q4_K_M1.6 GB
GGUF
AP
Apollo-6B
Apollo · BoscoTheDog
Params6.06B
Context4K
Smallest7.0 GB
GGUF
LL
llama3 therapybeagle chunked
Llama · BoscoTheDog
Params8.03B
Context8K
Smallest3.2 GB
GGUF
ON
oneirogen-1.5B
Oneirogen · BoscoTheDog
Params1.54B
Context128K
Smallest1.9 GB
GGUF
PH
phi 3 mini 128k instruct ru
Phi · BoscoTheDog
Params3.82B
Context128K
Smallest1.7 GB
Coding
UK
Ukranian Q2 chunked
Ukranian · BoscoTheDog
Params7.24B
Context32K
Smallest2.7 GB
GGUF
MI
ministral it 3b q5 k m chunked
Mistral · BoscoTheDog
Params3.32B
Context128K
Smallest2.3 GB
GGUF
MI
open buddy mistral 7B 32k chunked
Mistral · BoscoTheDog
Params7.28B
Context32K
Smallest4.4 GB
GGUF
PH
Phi 3.1 mini 4k instruct Q4 K M gguf chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.4 GB
GGUF
GE
Gemma 2 2b It
Gemma · BoscoTheDog
Params2.61B
Context8K
Smallest1.6 GB
GGUF
MI
bio mistral xxs
Mistral · BoscoTheDog
Params7.24B
Context32K
Smallest2.0 GB
GGUF
CO
cooking
Cooking · BoscoTheDog
Params121M
Context1K
Smallest0.1 GB
GGUF
FI
fietje 2b instruct Q5 K M chunked
Fietje · BoscoTheDog
Params2.78B
Context2K
Smallest2.0 GB
GGUF
BI
bitnet_b1_58-3B
Bitnet · BoscoTheDog
Params3.32B
Context2K
Q4_K_M5.0 GB
GGUF
GE
gemma mental health 2b gguf chunked
Gemma · BoscoTheDog
Params2.51B
Context8K
Smallest5.3 GB
GGUF
QW
code qwen 1 5 7B chunked
Qwen · BoscoTheDog
Params7.25B
Context64K
Smallest3.5 GB
Coding
MI
inspire mistral 7B 32K chunked
Mistral · BoscoTheDog
Params7.24B
Context32K
Smallest3.2 GB
GGUF
PH
Phi 3 mini 4k instruct q4 gguf chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Smallest2.4 GB
GGUF
RE
RedPajama INCITE Chat 3B gguf chunked
Redpajama · BoscoTheDog
Params2.78B
Context2K
Smallest1.6 GB
GGUF
PH
bling phi 3 RAG chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Smallest2.4 GB
GGUF
TI
TinyLlama-1.1B-Chat-v1.0
Tinyllama · BoscoTheDog
Params1.1B
Context2K
Smallest0.6 GB
GGUF
NE
neuralReyna mini 1.8b Q5 chunked
Neuralreyna · BoscoTheDog
Params1.84B
Context32K
Smallest1.4 GB
GGUF
FI
fietje 2b chat Q5 K M chunked
Fietje · BoscoTheDog
Params2.78B
Context2K
Smallest2.0 GB
GGUF
PH
phi3 mini 128k gguf chunked
Phi · BoscoTheDog
Params3.82B
Context128K
Smallest2.2 GB
GGUF
QW
Qwen2 1.5B Instruct Q4 K M chunked
Qwen · BoscoTheDog
Params1.54B
Context32K
Q4_K_M1.0 GB
GGUF

Keep browsing