Q6_K GGUF models
6-bit k-quant. Very close to the original model for most uses. 112449 models have a Q6_K file. Sizes run from 1M to 1030B parameters. The longest trained context is 4194304K tokens.
QW
Qwen3.8 27B TWIN TURBO Fable Cold Fusion 709 ULTRA HERETIC Uncensored NM DAU NEO MTP
Qwen · DavidAU
Params27B
Context256K
Q4_K_M18.0 GB
LL
Models Meta Llama Meta Llama 3.1 70B Instruct
Llama · MaziyarPanahi
Params70.6B
Context128K
Q4_K_M42.5 GB
QW
Qwen Image 2.1 PE T2I Heretic
Qwen · pottokao
Params8.95B
Context256K
Q4_K_M5.9 GB
LL
Llama-4-Scout-17B-16E-Instruct
Llama · unsloth
Params108B
Context10240K
Q4_K_M65.4 GB
QW
Qwen3.5 35B A3B
Qwen · bartowski
Params35.5B
Context256K
Q4_K_M22.3 GB
OR
Ornith 1.5 9B
Ornith · AtomicChat
Params8.95B
Context256K
Smallest2.8 GB
GL
Glm-4.6V-Flash
GLM · unsloth
Params9.4B
Context128K
Q4_K_M6.2 GB
QW
models--Qwen--Qwen2-7B-Instruct
Qwen · MaziyarPanahi
Params7.62B
Context32K
Q4_K_M4.7 GB
OR
Orion 26B A4B v1.1
Orion · bartowski
Params25.2B
Context256K
Q4_K_M18.0 GB
TR
Translategemma 4b It
Translategemma · mradermacher
Params3.88B
Context128K
Q4_K_M2.5 GB
QW
Qwen2.5 Coder 1.5B Instruct
Qwen · bartowski
Params1.54B
Context32K
Q4_K_M1.0 GB
MI
Models Mistralai Mistral Small Instruct 2409
Mistral · MaziyarPanahi
Params22.2B
Context32K
Q4_K_M13.3 GB
LT
LTX 2.5 GGUFs
Ltx · realrebelai
Params21B
Context—
Q4_K_M15.1 GB
LL
Models Meta Llama Meta Llama 3.1 8B Instruct
Llama · MaziyarPanahi
Params8B
Context128K
Q4_K_M4.9 GB
MI
Models Mistralai Mistral Small 24B Instruct 2501
Mistral · MaziyarPanahi
Params23.6B
Context32K
Q4_K_M14.3 GB
DE
deepseek-ai_deepseek-coder-6.7b-instruct
DeepSeek · TheBloke
Params6.74B
Context16K
Q4_K_M4.1 GB
QW
Qwen2.5 Coder 14B Instruct Abliterated
Qwen · bartowski
Params14.8B
Context32K
Q4_K_M9.0 GB
DE
Models Deepseek Ai DeepSeek R1 0528 Qwen3 8B
DeepSeek · MaziyarPanahi
Params8.19B
Context128K
Q4_K_M5.0 GB
SA
Sakura GalTransl 7B v3.7
Sakura · SakuraLLM
Params7.72B
Context128K
Smallest4.3 GB
DE
Deepseek-R1-Distill-Llama-70B
DeepSeek · unsloth
Params70.6B
Context128K
Q4_K_M42.5 GB
QW
Qwen2.5 Coder 14B Instruct
Qwen · lmstudio-community
Params14.8B
Context32K
Q4_K_M9.0 GB
GE
Gemma-4-E2B-It
Gemma · ente-ai
Params4.65B
Context128K
Q4_K_M3.1 GB
GE
Models Google Gemma 3 27b It
Gemma · MaziyarPanahi
Params27B
Context128K
Q4_K_M16.5 GB
NE
Nemotron-3-Nano-30B-A3B
Nemotron · unsloth
Params31.6B
Context1024K
Q4_K_M24.6 GB
QW
Models Qwen QwQ 32B
Qwen · MaziyarPanahi
Params32.8B
Context128K
Q4_K_M19.9 GB
GE
Gemma-3-27B-It
Gemma · unsloth
Params27B
Context128K
Q4_K_M16.5 GB
QW
Qwen3.6-27B-Uncensored-HauhauCS-Balanced
Qwen · HauhauCS
Params27B
Context256K
Smallest10.0 GB
QW
Qwen3.6 35B A3B Uncensored Heretic
Qwen · llmfan46
Params34.7B
Context256K
Q4_K_M21.2 GB
IN
Models PrimeIntellect INTELLECT 2
Intellect · MaziyarPanahi
Params32.8B
Context40K
Q4_K_M19.9 GB
PH
phi 4
Phi · MaziyarPanahi
Params14.7B
Context16K
Q4_K_M9.1 GB