Step 3.5 Flash Ablitirated needs about 133.3 GB of memory at Q4_K_M (a 118.6 GB file) with an 8K context window, which is more than a 48 GB GPU holds. This repository has 24 quants, from 0.5 GB to 161.7 GB (Q6_K).
Step 3.5 Flash Ablitirated – a 197B parameter Sparse MoE model
Step 3.5 Flash Ablitirated is a 197B parameter Sparse Mixture‑of‑Experts model. It has been abliterated to remove refusal vectors, enabling unrestricted generation. The model is available in multiple GGUF quantizations.
Good at
- Reasoning
- Tool usage
- Unfiltered content generation
Limits
- No vision support
- Requires large VRAM for high‑quality quants
- Apache‑2.0 license
Best for
- Research on large MoE architectures
- Testing unfiltered generation
- Fine‑tuning on custom datasets
Who it’s for. Researchers and developers who need a large, sparse MoE model for experimentation. The abliterated version is for advanced users aware of the risks of removing safety filters.
Quants & memory
| Quant | File | Est. memory | vs VRAM | Result | |
|---|---|---|---|---|---|
| — | 0.5 GB | 1.0 GB | Fits | Get file → | |
| IQ1_S | 39.4 GB | 44.6 GB | Too big | Get file → | |
| IQ1_M | 43.9 GB | 49.7 GB | Too big | Get file → | |
| IQ2_XXS | 51.4 GB | 58.1 GB | Too big | Get file → | |
| IQ2_XS | 57.5 GB | 64.9 GB | Too big | Get file → | |
| IQ2_S | 58.0 GB | 65.4 GB | Too big | Get file → | |
| IQ2_M | 64.0 GB | 72.2 GB | Too big | Get file → | |
| Q2_K_S | 65.9 GB | 74.3 GB | Too big | Get file → | |
| Q2_K | 71.8 GB | 80.9 GB | Too big | Get file → | |
| IQ3_XXS | 75.8 GB | 85.3 GB | Too big | Get file → | |
| IQ3_XS | 80.1 GB | 90.2 GB | Too big | Get file → | |
| Q3_K_S | 85.0 GB | 95.7 GB | Too big | Get file → | |
| IQ3_S | 85.0 GB | 95.7 GB | Too big | Get file → | |
| IQ3_M | 85.7 GB | 96.5 GB | Too big | Get file → | |
| Q3_K_M | 93.8 GB | 105.6 GB | Too big | Get file → | |
| Q3_K_L | 102.0 GB | 114.8 GB | Too big | Get file → | |
| IQ4_XS | 105.0 GB | 118.1 GB | Too big | Get file → | |
| Q4_0 | 111.3 GB | 125.2 GB | Too big | Get file → | |
| Q4_K_S | 111.5 GB | 125.4 GB | Too big | Get file → | |
| Q4_K_M | 118.6 GB | 133.3 GB | Too big | Get file → | |
| Q4_1 | 123.4 GB | 138.7 GB | Too big | Get file → | |
| Q5_K_S | 135.6 GB | 152.4 GB | Too big | Get file → | |
| Q5_K_M | 139.5 GB | 156.7 GB | Too big | Get file → | |
| Q6_K | 161.7 GB | 181.6 GB | Too big | Get file → |
For 8 GB VRAM, use the unknown quant (0.47 GB, ~1 GB memory). For 16 GB VRAM, IQ1_S (39.41 GB, ~44.6 GB memory) is too large; instead use IQ2_XXS (51.41 GB, ~58.1 GB memory) if you have a high‑end GPU. For 24 GB VRAM, Q3_K_M (93.8 GB, ~105.6 GB memory) is a good balance between size and performance. Estimates are approximate.
What GPU do you need for Step 3.5 Flash Ablitirated?
These figures add the file size, an fp16 KV cache at an 8K context window and about 0.5 GB for the runtime. They are estimates, so leave a little headroom, and expect to use less context or a smaller quant if you are right at the limit.
- 4 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 6 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 8 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 12 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 16 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 24 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 32 GB VRAM: the best quant that fits is the main file (0.5 GB file, about 1.0 GB in use).
- 48 GB VRAM: the best quant that fits is IQ1_S (39.4 GB file, about 44.6 GB in use).
Which Step 3.5 Flash Ablitirated quant should you choose?
Q4_K_M is the usual starting point for Step 3.5 Flash Ablitirated: it balances file size and quality well. Here it is a 118.6 GB file.
If memory is tight, the smallest file is the first one in the table at 0.5 GB, with the quality trade-offs that come with fewer bits.
New to quant names? Read GGUF quantization explained.
Specifications
Architecture from the GGUF header
Source
Frequently asked questions about Step 3.5 Flash Ablitirated
How much VRAM does Step 3.5 Flash Ablitirated need?
About 133.3 GB at Q4_K_M with an 8K context window. That is the 118.6 GB file, plus the KV cache and roughly 0.5 GB of runtime overhead. Smaller quants need less and longer context needs more.
Can I run Step 3.5 Flash Ablitirated on an 8 GB GPU?
Yes, with the smaller files: the 0.5 GB file needs about 1.0 GB at 8K context. Larger quants in this repository do not fit in 8 GB.
What is the file size of Step 3.5 Flash Ablitirated GGUF?
It depends on the quant: file is 0.5 GB, IQ1_S is 39.4 GB, IQ1_M is 43.9 GB, IQ2_XXS is 51.4 GB, IQ2_XS is 57.5 GB, IQ2_S is 58.0 GB and 18 more.
What context length does Step 3.5 Flash Ablitirated support?
The GGUF header lists a trained context length of 262,144 tokens (256K).
Where can I download Step 3.5 Flash Ablitirated GGUF?
The files are on Hugging Face in the mradermacher/Step-3.5-Flash-Ablitirated-i1-GGUF repository. The download buttons on this page open the file on Hugging Face. ggufmodels.com does not host any model files.
What license does Step 3.5 Flash Ablitirated use?
Hugging Face lists the license as apache-2.0. Licenses can differ between the quantized files and the original model, so check the repository and the base model before commercial use.
Can I run the unknown quant on 8 GB VRAM?
Yes, the unknown quant is 0.47 GB and requires about 1 GB of VRAM at 8 k context.
What is the memory requirement for IQ1_S?
IQ1_S is 39.41 GB and requires about 44.6 GB of VRAM at 8 k context.
Is the model vision capable?
No, the model does not support vision tasks.