GGUF models by BoscoTheDog

58 GGUF models quantized by BoscoTheDog on Hugging Face. Sizes run from 121M to 9.24B parameters. The longest trained context is 128K tokens.

Most downloadedNewestLargestLongest contextSmallest
PH
phi 3 mini 128k it russian Q4 K M chunked
Phi · BoscoTheDog
Params3.82B
Context128K
Q4_K_M4.8 GB
Coding
DA
h2o-danube3-500m-chat
Danube · BoscoTheDog
Params514M
Context8K
Q4_K_M0.6 GB
GGUF
BI
bitnet_b1_58-xl
Bitnet · BoscoTheDog
Params1.27B
Context2K
Smallest1.6 GB
GGUF
DA
h2o-danube3-500m-chat
Danube · BoscoTheDog
Params514M
Context8K
Q4_K_M0.3 GB
GGUF
MI
Bitnet-Mistral.0.2-330m-v0.2-grokfast-v2.9
Mistral · BoscoTheDog
Params330M
Context4K
Q4_K_M0.2 GB
GGUF
QW
qwen2 5 coder 1 5b chunked
Qwen · BoscoTheDog
Params1.78B
Context32K
Smallest1.1 GB
CodingTool use
PH
phi 3 mini 4k it spanish pofi Q4 K M
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M4.6 GB
GGUF
PH
phi 3 mini 128k italian v2 Q4 K M chunked
Phi · BoscoTheDog
Params3.82B
Context128K
Q4_K_M2.4 GB
Coding
PH
Phi 3.5 mini it Q4 K L chunked
Phi · BoscoTheDog
Params3.82B
Context128K
Smallest2.5 GB
Coding
GE
gemma2 it 2b jpn Q4 K M chunked
Gemma · BoscoTheDog
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
GE
gemma 2b sauerkraut gguf chunked
Gemma · BoscoTheDog
Params2.51B
Context8K
Smallest3.1 GB
GGUF
PH
phi 3 mini 4k it french alpaca v1 Q4 K M
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M4.6 GB
Coding
LL
llama 3 2 it 1b q4 k m chunked
Llama · BoscoTheDog
Params1.24B
Context128K
Q4_K_M0.8 GB
GGUF
TI
tiny therapy
Tiny · BoscoTheDog
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
QW
polanka qwen2 Q4 0 chunked
Qwen · BoscoTheDog
Params1.54B
Context128K
Smallest0.9 GB
GGUF
QW
qwen2 5 coder 7b chunked
Qwen · BoscoTheDog
Params7.62B
Context128K
Smallest3.0 GB
CodingTool use
LL
Llama3 8B 1.58 100B Tokens
Llama · BoscoTheDog
Params8.03B
Context8K
Smallest2.2 GB
GGUF
LL
llama3 2 it 3b medical conversational Q4 0 chunked
Llama · BoscoTheDog
Params3.21B
Context128K
Smallest1.9 GB
Tool use
GE
gemma-2-9b-it
Gemma · BoscoTheDog
Params9.24B
Context8K
Smallest3.1 GB
GGUF
BI
bitnet_b1_58-large
Bitnet · BoscoTheDog
Params729M
Context2K
Smallest0.8 GB
GGUF
GE
vikhr gemma 2b it Q4 K M chunked
Gemma · BoscoTheDog
Params2.61B
Context8K
Q4_K_M1.7 GB
GGUF
TR
TriLM BitNet 3 9B
Trilm · BoscoTheDog
Params3.99B
Context2K
Smallest1.0 GB
GGUF
BE
Bessie bitnet instruct 100k
Bessie · BoscoTheDog
Params729M
Context2K
Smallest0.8 GB
GGUF
GE
gemma2b therapy Q4 0 chunked
Gemma · BoscoTheDog
Params2.61B
Context8K
Smallest1.6 GB
GGUF
MI
ministral 3b it q4 k m chunked
Mistral · BoscoTheDog
Params3.32B
Context128K
Q4_K_M2.0 GB
GGUF
SA
SambaLingo Hungarian Chat Q2 K chunked
Sambalingo · BoscoTheDog
Params6.95B
Context4K
Smallest2.7 GB
GGUF
PH
phi 3 mini 4k it portuguese tom cat Q4 K M chunked
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.4 GB
Coding
LL
llama 3 2 it 3b q4 k m chunked
Llama · BoscoTheDog
Params3.21B
Context128K
Q4_K_M2.0 GB
Tool use
PH
phi-3-mental-health-q4
Phi · BoscoTheDog
Params3.82B
Context4K
Q4_K_M2.3 GB
Coding
ME
medical bot
Medical · BoscoTheDog
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF

About BoscoTheDog on Hugging Face

BoscoTheDog has published 58 GGUF models that ggufmodels.com tracks, across families such as Phi, Gemma and Mistral. The most downloaded are phi 3 mini 128k it russian Q4 K M chunked, h2o-danube3-500m-chat and bitnet_b1_58-xl.

The quants BoscoTheDog publishes most often are Q4_K_M (23), Q4_0 (11), Q8_0 (10) and Q5_K_M (9). Their Q4_K_M files range from 0.2 to 5.0 GB. Parameter counts run from 121M to 9.24B, and the longest trained context window is 128K tokens.

Frequently asked questions

What GGUF models has BoscoTheDog published?

58 models are listed here, mainly from the Phi, Gemma and Mistral families. The most downloaded is phi 3 mini 128k it russian Q4 K M chunked.

Which quants does BoscoTheDog publish?

The most common are Q4_K_M (23 models), Q4_0 (11 models), Q8_0 (10 models) and Q5_K_M (9 models).

Keep browsing