Qwen3.6-35B-A3B Abliterated: Which GGUF Fits Your GPU?
Qwen3.6-35B-A3B abliterated builds range from 11 GB GGUF to 72 GB BF16. Compare HauhauCS, huihui-ai, and Bahushruth by file size, VRAM fit, and refusal claims.
Key points
- Qwen3.6-35B-A3B is a 35B mixture-of-experts model with about 3B active parameters, so it needs 35B of memory but computes like a much smaller model.
- HauhauCS, huihui-ai, and Bahushruth v4 remove refusals in different ways. Their GGUF sizes range from 11 GB to 44 GB, with BF16 weights around 72 GB.
- A 12 GB GPU needs IQ2_M, 16 GB needs IQ3_M, and 24 GB can run Q4_K_M. Context length and the vision projector consume memory beyond the weights.
Editor's note
We checked the primary sources linked in this note and organized only what they support. Unless stated otherwise, we did not run independent measurements.
Editorial policy and how we use AI →Contents
Qwen3.6-35B-A3B abliterated builds fit GPUs from 12 GB to 48 GB, but the model name alone tells you little. The base is a 35B mixture-of-experts model with about 3B active parameters, so it needs memory for all 35B while doing the compute of a much smaller model.
Community releases differ in method, quantization, and format: HauhauCS ships K_P GGUF quants from an 11 GB IQ2_M to a 44 GB Q8_K_P; huihui-ai offers MTP-tagged GGUF from 13.2 GB to 71.1 GB; Bahushruth v4 publishes full BF16 weights built with norm-preserving abliteration. For 12 GB, IQ2_M is the only realistic starting point; 16 GB handles IQ3_M; 24 GB handles Q4_K_M. If you want the shortest setup, use Ollama with huihui-ai's 35b-a3b-q4_K tag. If you want to inspect the weight edit, start with Bahushruth v4.
What is Qwen3.6-35B-A3B?#
Qwen3.6-35B-A3B is an open-weight mixture-of-experts model that Alibaba's Qwen team released in April 2026 under Apache 2.0. It has 35B total parameters, about 3B active per token, 40 layers, 256 experts (8 routed plus 1 shared), a 262,144-token context, and native text, image, and video input.
MoE means the model keeps every expert in memory but routes each token through a small subset. You budget memory for 35B and compute for roughly 3B. That split is why a 35B model can run at a usable speed on a 24 GB card even though the weights fill most of the memory.
By November 6, 2026, the base repository had 4,388,118 downloads and 3,088 likes. The architecture alternates 30 linear-attention layers with 10 full-attention layers. Source: Qwen3.6-35B-A3B model card.
Which abliterated builds are available?#
The main builds fall into four groups: HauhauCS Aggressive (K_P GGUF, 11–44 GB), huihui-ai MTP (GGUF, 13.2–71.1 GB), Bahushruth v4 (BF16 weights, norm-preserving), and MLX or FP8 builds for Macs and vLLM servers. They remove refusals with different methods, and their published refusal rates come from different test sets.
| Build | Method and format | File size | Best for |
|---|---|---|---|
| HauhauCS Aggressive | Abliteration, K_P GGUF | 11–44 GB | 12–48 GB GPUs, Ollama |
| huihui-ai MTP | Abliteration, MTP GGUF | 13.2–71.1 GB | Ollama, MTP-capable runtimes |
| Bahushruth v4 | Norm-preserving, single direction | ~72 GB BF16 | Weight-edit research |
| MLX / FP8 builds | 4–8 bit MLX, FP8/AWQ | Varies | Apple Silicon, vLLM |
HauhauCS Aggressive#
Published on April 17, 2026, this build had 1,256,641 downloads and 4,103 likes by November 6. Its model card claims 0 refusals across 465 prompts.
K_P is a custom quantization profile that preserves quality where the model is most sensitive, at roughly 5–15% larger files than the base quant. The range runs from IQ2_M at 11 GB to Q8_K_P at 44 GB. The files work in llama.cpp and LM Studio without a special build.
On Ollama, the repository is fredrezones55/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive, with tags latest, Q2_K_P, Q4, IQ2_M, and IQ4_XS.
The card also notes that the model may append short disclaimers. That is not a refusal, but a 0% refusal rate does not make the output entirely unshaped.
huihui-ai#
The base abliterated weights appeared on April 18, 2026; the MTP GGUF followed on May 19. The MTP repository had 4,693,464 downloads but only 77 likes.
Sizes run from Q2_K at 13.2 GB to F16 at 71.1 GB, with Q4_K at 21.7 GB. A separate 899 MB mmproj file handles image input.
MTP stands for multi-token prediction, a training objective that predicts several future tokens. Runtimes that support it can use it for speculative decoding; check your tool's version before expecting a speedup.
Ollama tags are 35b-a3b, 35b-a3b-q4_K, 35b-a3b-q8_0, and 35b-a3b-bf16.
The download gap between huihui-ai and HauhauCS reverses when you look at likes: 77 versus 4,103. Automated pulls inflate download counts, so likes and update activity are better signals of actual use.
Bahushruth v4#
Bahushruth v4 removes a single refusal direction with norm-preserving orthogonalization. It estimates that direction from 512 pairs drawn from 7,356 harmful prompts, across all 40 layers.
The card reports 0 refusals on 16 harmful prompts and 0 false refusals on harmless prompts. It also documents the path: v1 removed refusals but damaged capability, v3 preserved capability but left 94% refusals, and v4 combined both.
Sixteen prompts is not the same scale as 465. Read the test setup, not just the percentage.
The BF16 weights are about 72 GB, so they do not fit a 24 GB card. Requantized GGUF builds are available separately.
Mac and vLLM#
On Apple Silicon, MLX builds are the practical route. leonsarmiento's 6-bit XL MTP and froggeric's Heretic 4-bit are two examples. For a server, FP8 (coolthor) and AWQ or GPTQ builds target vLLM.
Which quantization fits 12 GB, 16 GB, or 24 GB?#
For 12 GB, HauhauCS IQ2_M at 11 GB is the smallest option; 16 GB handles IQ3_M or Q2_K_P at 15 GB; 24 GB handles Q4_K_M at 21 GB. These are weight sizes only. KV cache, the mmproj vision encoder, and context length all consume additional memory, so leave headroom.
| GPU memory | Practical quantization | File size | Context starting point |
|---|---|---|---|
| 12 GB | HauhauCS IQ2_M | 11 GB | 8K |
| 16 GB | HauhauCS IQ3_M / Q2_K_P | 15 GB | 8K–16K |
| 24 GB | HauhauCS Q4_K_M / huihui Q4_K | 21 GB / 21.7 GB | 16K–32K |
| 32 GB | Q5_K_P / IQ4_XS | 28 GB / 19 GB | 16K–32K |
| 48 GB | Q6_K_P / Q8_K_P / Q8_0 | 31 GB / 44 GB / 37.8 GB | Long context |
The context figures are starting points, not guarantees. How far you can extend depends on the runtime and its KV-cache settings.
262,144 tokens is not realistic on a consumer GPU even when the weights fit. Start at 8K or 16K and raise it only if you need the length.
Q2-class quants trade quality for size. If the output matters, move up to IQ3 or Q4 and shorten the context instead.
| IQ2_M | 11GB | |
|---|---|---|
| IQ3_M | 15GB | |
| IQ4_XS | 19GB | |
| Q4_K_M | 21GB | |
| Q5_K_P | 28GB | |
| Q6_K_P | 31GB | |
| Q8_K_P | 44GB |
At 24 GB, Q4_K_M uses 21 GB for weights. Add the 899 MB vision projector and KV cache, and a 16K context leaves little room. For a 32K context, drop to Q3_K_P or IQ4_XS, or use a larger card.
At 12 GB, IQ2_M leaves almost nothing after the weights. Text-only use with a short context is the realistic case.
How do you run it with Ollama or llama.cpp?#
The shortest path is Ollama: run huihui_ai/Qwen3.6-abliterated:35b-a3b-q4_K or the HauhauCS Q4 tag. For llama.cpp, pass the GGUF with -m, the vision projector with --mmproj, --jinja for the chat template, and set -c and -ngl for context and GPU layers.
ollama run huihui_ai/Qwen3.6-abliterated:35b-a3b-q4_K
ollama run fredrezones55/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive:Q4
Ollama tags are case-sensitive in practice; copy the spelling from the model page.
llama-cli -m "Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf" --mmproj "mmproj-Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-f16.gguf" --jinja -c 16384 -ngl 99
-c sets context length and -ngl sets how many layers go on the GPU. On a 24 GB card with Q4_K_M, start at -c 16384 and lower it if you hit out-of-memory errors.
Image input needs the mmproj file. For text-only use, leave it out to free memory.
On a Mac, open an MLX build in LM Studio. The 4-bit build suits a 32 GB machine; 6-bit needs more headroom.
For more detail, see the Ollama setup guide and GGUF quantization explained.
What should you check before choosing a build?#
Check how each publisher measured refusals, how hard the quantization is, whether your runtime supports MTP, and what license the derivative carries. HauhauCS reports 0 refusals on 465 prompts; Bahushruth reports 0 on 16. Those numbers describe different tests, not the same claim.
Q2-class quants can change instruction following and writing stability. Prepare five prompts from your own work and compare outputs across quants.
K_P is HauhauCS's own quant format. LM Studio may show the quant name as "?"; the card says this is a display issue, not a loading failure.
MTP only helps in runtimes that support it. Check the version before expecting faster generation.
The base license is Apache 2.0. Check each derivative's license page, and check the rights and regulations that apply to the output if you use it commercially.
An uncensored model will output wrong information as readily as right information. Removing refusals does not improve accuracy. For medical, legal, or safety decisions, verify against primary sources.
To test behavior before downloading 20 GB, use a browser service. Deepent Research's free trial runs Qwen-family abliterated models without a GPU or install.
Frequently asked questions#
Is Qwen3.6-35B-A3B better than Qwen3.8-27B for local use?#
It depends on memory and speed. Qwen3.8-27B is a dense 27B model; Qwen3.6-35B-A3B is a 35B MoE with about 3B active parameters. The MoE needs more memory for weights but less compute per token. With 24 GB, Qwen3.6 at Q4_K_M works; with 16 GB, Qwen3.8 at 4-bit or Qwen3.6 at IQ3_M fits better. See the Qwen3.8 comparison.
Can a 12 GB GPU run Qwen3.6-35B-A3B abliterated?#
Yes, but only with the smallest quantization and short context. HauhauCS IQ2_M is about 11 GB, leaving little room for KV cache and the vision projector. Start with an 8K context, skip image input, and expect some quality loss from the 2-bit quantization.
Are the 0% refusal claims reliable?#
They are self-reported and measured on different sets. HauhauCS tested 465 prompts; Bahushruth tested 16. Neither is an independent audit, and neither predicts how the model will behave on your prompts. Test the questions you actually care about and inspect the answers rather than relying on a headline number.
Which Ollama tag should I pull?#
For 24 GB, use Q4 on the HauhauCS build or 35b-a3b-q4_K on huihui-ai. For 16 GB, use IQ3_M or Q2_K_P; for 12 GB, use IQ2_M. Check the tag list on the model page before pulling, because the names differ between publishers.
Does it work in Japanese?#
Qwen models are trained on multilingual data and handle Japanese prompts, but quantization strength and prompting change the output. If Japanese is your main use, compare a few summarization or rewriting prompts across quants before committing to a 20 GB download.
Try it yourself
Deepent Research is an LLM chat service that refuses almost nothing. Everything covered in this note can be tried without downloading anything.
This note is provided for educational and research purposes. It does not encourage illegal activity.
How this note was edited and verified
Our editors check primary sources and organize only what the facts support. What we checked, and what we did not, is listed in the editor's note on each article.
Related notes
Abliterated Qwen3.8-27B: Which of 4 Editions to Run
Compare the huihui-ai, orcarouter, Heretic and RVN Qwen3.8-27B abliterated editions, then pick a quant for a 16GB GPU, a 24GB GPU or a Mac, with file sizes.
GGUF Quantization: How to Choose Q4 vs Q8 for Local LLMs
Learn what GGUF quantization is, how to read Q4_K_M, how Q4, Q6 and Q8 compare in accuracy and speed, and what fits your PC, using published Llama 3.1 8B data.
How to Run Uncensored LLMs With Ollama (2026 Commands)
Run an uncensored LLM with Ollama in two commands. Pick an abliterated build for 8GB–24GB VRAM, choose Q4_K_M quantization, and fix load errors.