GGUF models by itlwas

475 GGUF models quantized by itlwas on Hugging Face. Sizes run from 5M to 46.7B parameters. The longest trained context is 4192K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Hermes 2 Pro Llama 3 8B Q4 K M
Llama · itlwas
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
NE
Nemotron 4 Mini Hindi 4b Base
Nemotron · itlwas
Params4.19B
Context4K
Q4_K_M2.7 GB
Tool use
AY
Aya Expanse 32b
Aya · itlwas
Params32.3B
Context8K
Q4_K_M19.8 GB
GGUF
LL
MMed Llama 3 8B EnIns
Llama · itlwas
Params8.03B
Context8K
Q4_K_M4.9 GB
GGUF
QW
Qwen2.5 14B Instruct
Qwen · itlwas
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
LL
Einstein v6.1 Llama3 8B Q4 K M
Llama · itlwas
Params8.03B
Context8K
Q4_K_M4.9 GB
Tool use
SE
SeaLLMs v3 7B
Seallms · itlwas
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
SE
SeaLLMs v3 7B Chat
Seallms · itlwas
Params7.62B
Context128K
Q4_K_M4.7 GB
GGUF
AU
Aura MoE 2x4B
Aura · itlwas
Params7.23B
Context128K
Q4_K_M4.4 GB
RoleplayTool use
SA
saiga nemo 12b Q4 K M
Saiga · itlwas
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
AH
Ahma 7B
Ahma · itlwas
Params7B
Context2K
Q4_K_M4.3 GB
GGUF
SE
SeaLLM 7B v2.5 Q4 K M
Seallm · itlwas
Params8.54B
Context8K
Q4_K_M5.3 GB
GGUF
CH
ChatWaifu v1.0 Q4 K M
Chatwaifu · itlwas
Params7.24B
Context4K
Q4_K_M4.4 GB
Roleplay
QW
Qwen2.5 3B
Qwen · itlwas
Params3.09B
Context32K
Q4_K_M1.9 GB
Tool use
OL
OLMo 2 1124 7B
OLMo · itlwas
Params7.3B
Context4K
Q4_K_M4.5 GB
GGUF
AT
Athena 1 14B Q4 K M
Athena · itlwas
Params14.8B
Context32K
Q4_K_M9.0 GB
Tool use
QW
Qwen2.5 32B
Qwen · itlwas
Params32.8B
Context128K
Q4_K_M19.9 GB
Roleplay
QW
Qwen1.5 0.5B Chat
Qwen · itlwas
Params620M
Context32K
Q4_K_M0.4 GB
GGUF
LL
Llama 3.1 Nemotron 8B UltraLong 2M Instruct Q4 K M
Llama · itlwas
Params8.04B
Context2096K
Q4_K_M4.9 GB
Tool use
EU
EuroLLM 9B
Eurollm · itlwas
Params9.15B
Context4K
Q4_K_M5.6 GB
GGUF
MI
Mistral 7B v0.1
Mistral · itlwas
Params7.24B
Context32K
Q4_K_M4.4 GB
GGUF
CU
Custom KoLLM 13B v6 Q4 K M
Custom · itlwas
Params13.2B
Context2K
Q4_K_M8.0 GB
GGUF
MI
Mini Magnum 12b v1.1
Mini · itlwas
Params12.2B
Context1000K
Q4_K_M7.5 GB
GGUF
QW
Qwen1.5 32B
Qwen · itlwas
Params32.5B
Context32K
Q4_K_M19.7 GB
CodingTool use
LL
Llama 160m
Llama · itlwas
Params162M
Context2K
Q4_K_M0.1 GB
GGUF
WO
WoonaV1.2 9b Q4 K M
Woonav · itlwas
Params9.24B
Context8K
Q4_K_M5.8 GB
GGUF
TI
TinyLLama v0
Tinyllama · itlwas
Params5M
Context2K
Q4_K_M0.0 GB
GGUF
QW
Qwen3 0.6B
Qwen · itlwas
Params752M
Context40K
Q4_K_M0.5 GB
ReasoningTool use
TI
TinyLlama_v1.1
Tinyllama · itlwas
Params1.1B
Context2K
Q4_K_M0.7 GB
GGUF
OP
openchat 3.5 1210 Q4 K M
Openchat · itlwas
Params7.24B
Context8K
Q4_K_M4.4 GB
Coding

Keep browsing