GGUF models by PJRM

60 GGUF models quantized by PJRM on Hugging Face. Sizes run from 1.54B to 4.51B parameters. The longest trained context is 256K tokens.

Most downloadedNewestLargestLongest contextSmallest
LL
Llama NextGenmix 4b Q4 0
Llama · PJRM
Params4.01B
Context128K
Smallest2.4 GB
Tool use
QW
Qwen3 4B Sky High Hermes
Qwen · PJRM
Params4.02B
Context256K
Smallest2.4 GB
ReasoningTool use
EC
Echo 3B
Echo · PJRM
Params2.8B
Context4K
Smallest1.6 GB
GGUF
NO
Nomi 1 Creative
Nomi · PJRM
Params3.09B
Context32K
Smallest1.8 GB
Tool use
RO
RocketHermesZephyrBoros 3B Q4 0
Rockethermeszephyrboros · PJRM
Params2.8B
Context4K
Smallest1.6 GB
Tool use
RO
Rosa NSFW Niche 3B Q4 0
Rosa · PJRM
Params2.8B
Context4K
Smallest1.6 GB
GGUF
LL
Hermes 3 Llama 3.2 3B Q4 0
Llama · PJRM
Params3.21B
Context128K
Smallest1.9 GB
RoleplayTool use
QW
Qwen1.5 1.8B Chat Heretic
Qwen · PJRM
Params1.84B
Context32K
Smallest1.1 GB
GGUF
ST
Stablelm Zephyr 3b
Stablelm · PJRM
Params2.8B
Context4K
Smallest1.6 GB
GGUF
QW
Qwen1.5 1.8B Chat Heretic
Qwen · PJRM
Params1.84B
Context32K
Q4_K_M1.2 GB
GGUF
NO
Capybara 3B V1.9
Nous · PJRM
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
QW
EVA Qwen2.5 1.5B v0.0
Qwen · PJRM
Params1.54B
Context128K
Smallest0.9 GB
Roleplay
LL
Llama 3.1 Minitron 4B Chat Q4 0
Llama · PJRM
Params4.51B
Context128K
Smallest2.6 GB
Tool use
LL
Llama 3.1 Nemotron Nano 4B v1.1
Llama · PJRM
Params4.51B
Context128K
Smallest2.7 GB
Tool use
KA
kappa 3 phi abliterated Q4 0
Kappa · PJRM
Params3.82B
Context4K
Smallest2.2 GB
GGUF
SM
Smolgpt Fables
Smolgpt · PJRM
Params1.71B
Context8K
Smallest1.4 GB
GGUF
LL
Llama 3.2 3B Instruct Chatml
Llama · PJRM
Params3.21B
Context128K
Smallest1.9 GB
GGUF
RO
RocketHermesZephyrBoros 3B Q4 K M
Rockethermeszephyrboros · PJRM
Params2.8B
Context4K
Q4_K_M1.7 GB
Tool use
ST
StableHermes 3b IQ4 NL
Stablehermes · PJRM
Params2.8B
Context4K
Smallest1.6 GB
Tool use
MC
L MChat Small Q4 0
Mchat · PJRM
Params2.78B
Context2K
Smallest1.6 GB
GGUF
LI
LittleInstructionMaker 4B v0.2
Littleinstructionmaker · PJRM
Params3.96B
Context8K
Smallest2.3 GB
Coding
QW
Qwen2.5 3B Instruct
Qwen · PJRM
Params3.09B
Context32K
Smallest1.8 GB
Tool use
SM
Smolgpt Fables
Smolgpt · PJRM
Params1.71B
Context8K
Smallest1.0 GB
GGUF
LI
LittleInstructionMaker 4B v0.2
Littleinstructionmaker · PJRM
Params3.96B
Context8K
Smallest2.3 GB
Coding
BK
4Bkto Q4 0
Bkto · PJRM
Params4.41B
Context256K
Smallest2.6 GB
GGUF
NY
Nytheria 3b
Nytheria · PJRM
Params3.09B
Context32K
Smallest1.8 GB
Roleplay
SO
Soul 3B Q4 K M
Soul · PJRM
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
MI
MiniButler 4B
Minibutler · PJRM
Params4.02B
Context32K
Smallest2.4 GB
ReasoningTool use
AU
Aura 3B Q4 K M
Aura · PJRM
Params2.8B
Context4K
Q4_K_M1.7 GB
GGUF
LL
LlamaTronic 3.1 4B IQ4 NL
Llama · PJRM
Params4.51B
Context128K
Smallest2.7 GB
GGUF

Keep browsing