GGUF models by unsloth

391 GGUF models quantized by unsloth on Hugging Face. Sizes run from 33M to 2780B parameters. The longest trained context is 10240K tokens.

Most downloadedNewestLargestLongest contextSmallest
QW
Qwen2.5 Coder 0.5B Instruct
Qwen · unsloth
Params494M
Context128K
Q4_K_M0.4 GB
Coding
FA
Falcon-H1-7B-Instruct
Falcon · unsloth
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
LL
Cogito-V2-Preview-Llama-109B-Moe
Llama · unsloth
Params108B
Context256K
Q4_K_M65.4 GB
Vision
GR
Granite-4.0-1B
Granite · unsloth
Params1.63B
Context128K
Q4_K_M1.0 GB
GGUF
IN
Januscoder-8B
InternLM · unsloth
Params8.19B
Context32K
Q4_K_M5.0 GB
CodingVision
AP
Apriel-1.5-15B-Thinker
Apriel · unsloth
Params14.4B
Context256K
Q4_K_M8.8 GB
ReasoningVision
QW
Qwq-32B
Qwen · unsloth
Params32.8B
Context32K
Q4_K_M19.9 GB
Reasoning
DE
DeepSeek V3 BF16
DeepSeek · unsloth
Params671B
Context160K
Q4_K_M404.4 GB
GGUF
LI
Ling-1T
Ling · unsloth
Params1000B
Context32K
Q4_K_M604.7 GB
GGUF
LF
Lfm2-700M
Lfm · unsloth
Params742M
Context32K
Q4_K_M0.5 GB
GGUF
FA
Falcon-H1-1.5B-Instruct
Falcon · unsloth
Params1.55B
Context128K
Q4_K_M0.9 GB
GGUF
QW
Qwen3-Coder-480B-A35B-Instruct-1M
Qwen · unsloth
Params480B
Context1024K
Q4_K_M290.1 GB
Coding
QW
Qwen3-Vl-235B-A22B-Thinking-1M
Qwen · unsloth
Params235B
Context977K
Q4_K_M142.2 GB
ReasoningVision
QW
Qwen3-Vl-32B-Thinking-1M
Qwen · unsloth
Params32.8B
Context977K
Q4_K_M19.8 GB
ReasoningVision
KI
Kimi-K2-Instruct-0905
Kimi · unsloth
Params1030B
Context256K
Q4_K_M621.2 GB
GGUF
RN
Rnj-1-Instruct
Rnj · unsloth
Params8.3B
Context32K
Q4_K_M5.1 GB
GGUF
LL
Cogito-V2-Preview-Llama-70B
Llama · unsloth
Params70.6B
Context128K
Q4_K_M42.5 GB
GGUF
IN
Internvl3-2B
Internvl · unsloth
Params1.78B
Context32K
Q4_K_M1.1 GB
CodingVision
NE
Acereason-Nemotron-14B
Nemotron · unsloth
Params14.8B
Context128K
Q4_K_M9.0 GB
CodingReasoningTool use
FA
Falcon-H1R-7B
Falcon · unsloth
Params7.59B
Context256K
Q4_K_M4.6 GB
GGUF
GR
Granite-3.3-8B-Instruct
Granite · unsloth
Params8.17B
Context128K
Q4_K_M4.9 GB
GGUF
IN
Internvl3-14B-Instruct
Internvl · unsloth
Params14.8B
Context32K
Q4_K_M9.0 GB
CodingVision
IN
Internvl3-1B-Instruct
Internvl · unsloth
Params630M
Context32K
Q4_K_M0.5 GB
CodingVision
TR
Trinity-Large-Preview
Trinity · unsloth
Params399B
Context256K
Q4_K_M239.9 GB
Tool use
GR
Granite-3.3-2B-Instruct
Granite · unsloth
Params2.53B
Context128K
Q4_K_M1.5 GB
GGUF
GE
Gemma-3-4B-It-Qat-Int4
Gemma · unsloth
Params3.88B
Context128K
Q4_K_M2.3 GB
Vision
DE
DeepSeek R1 Zero BF16
DeepSeek · unsloth
Params671B
Context160K
Q4_K_M404.4 GB
Reasoning
GL
cerebras/GLM-4.6-REAP-268B-A32B
GLM · unsloth
Params269B
Context198K
Q4_K_M162.5 GB
ReasoningTool use
IN
Internvl3-2B-Instruct
Internvl · unsloth
Params1.78B
Context32K
Q4_K_M1.1 GB
CodingTool useVision
IN
Internvl3-1B
Internvl · unsloth
Params630M
Context32K
Q4_K_M0.5 GB
CodingTool useVision

Keep browsing