Gemma 4 26B-A4B Abliterated v3: Refusals and GGUF Sizes

Gemma 4 26B-A4B abliterated v3 cuts Spanish refusals to 0% but leaves 32% of English refusals. Compare GGUF sizes, 16GB/24GB picks, licenses, and Ollama.

For: People running Gemma 4 26B-A4B abliterated on a 16–24 GB GPU

Key points

  • v3 removes Spanish refusals (27.2% to 0.0%) but only halves English refusals (68.7% to 32.1%) under the author's strict marker test.
  • GGUF files run from Q2_K at 10.6 GB to Q8_0 at 26.9 GB. Q3_K_M is the 16 GB starting point; Q4_K_M is the 24 GB starting point.
  • The v3 license is research-only (apache-2.0-restricted-research), and the card says not to deploy it for production. Check the license before any commercial use.

Editor's note

Verified on November 13, 2026 against the v3 model card, mradermacher's GGUF list, the huihui-ai and groxaxo repositories, Google's Gemma 4 model card, and Ollama tags. I did not run the model or measure refusals myself. The v3 card's surgery window (L28–L63) does not match the 30-layer public config, and the layer mapping is unconfirmed.

Editorial policy and how we use AI →
Contents
  1. What changed in v3?
  2. What is Gemma 4 26B-A4B?
  3. Which build should you download?
  4. Which GGUF fits 12 GB, 16 GB, or 24 GB?
  5. How do you run it with Ollama or llama.cpp?
  6. How should you read a 0% refusal claim?
  7. What this article did not verify
  8. FAQ

Gemma 4 26B-A4B abliterated v3 appeared on Hugging Face on November 10, 2026. Its card advertises 0% refusals in Spanish, but when you send an English research prompt, the model still says it cannot help. That gap is the story: v3 removes Spanish refusals (27.2% to 0.0%) but only halves English refusals under the author's strict marker test (68.7% to 32.1%).

If you want a local build, Q3_K_M (13.3 GB) is the 16 GB starting point and Q4_K_M (16.8 GB) is the 24 GB starting point. The v3 license is research-only, and the card says not to deploy it for production. Check the license before any commercial use. This guide explains what v3 changed, how the three main builds differ, and which GGUF fits your GPU.

What changed in v3?#

v3 is a targeted refusal edit, not a full uncensoring. Its author evaluated 1,058 held-out prompts with greedy decoding and 64 tokens, then counted refusals with two different judges (model card). The results explain why two people can read the same model card and reach opposite conclusions.

JudgeLanguageBasev3
Collection judgeEN:OFF7.6%14.5%
Collection judgeES:OFF27.2%0.0%
Strict string-markerEN:OFF68.7%32.1%
Strict string-markerES:OFF31.0%0.0%

The collection judge makes English look worse (7.6% to 14.5%). The card says the judge under-detects the base model's dominant refusal family, so that number is not evidence that v3 added refusals. The strict marker test is the cleaner comparison: English refusals fell by 36.6 percentage points, and Spanish refusals dropped to zero.

Across 289 OFF prompts, v3 removed 51 refusals (43 Spanish, 8 English). It did not remove the English sandbox-escape family; both the base and v3 refused those prompts. In practice, v3 is a Spanish-focused research build with a partial English effect, not a model that answers every English request.

What is Gemma 4 26B-A4B?#

Gemma 4 26B-A4B is Google's mixture-of-experts model in the Gemma 4 family. It has about 25.2B total parameters but activates about 3.8B per token. That split lets a 26B-class model run on a 16–24 GB GPU at a usable speed.

ItemValue
Total parametersAbout 25.2B
Active parametersAbout 3.8B
Layers30
Experts128 (8 routed + 1 shared)
Context262,144 tokens
InputText and images
Base licenseApache 2.0

The base repository had 5,251,952 downloads and 1,763 likes on November 13, 2026. It supports thinking mode and image input. The 12B and 31B variants are compared in Gemma 4 abliterated: 12B, 26B MoE, and 31B.

Which build should you download?#

There are three practical lines: huihui-ai, groxaxo's UD quantizations, and the new v3. The newest build is not automatically the best one for your use case.

BuildReleasedFormat and sizeNotesBest for
huihui-aiApril 11, 2026Ollama 18 GB, GGUF variantsBoth thinking and non-thinking modes edited; first 5 layers untouchedFastest setup
groxaxo UDApril 11, 202622 GGUF files, 10.0–27.9 GBUnsloth-style UD quants, 1.2 GB mmprojChoosing a quant
jsantillana v3 / mradermacherNov 10 / Nov 12, 202613 GGUF files, 10.6–26.9 GBResearch-only; 32.1% of English refusals remainRefusal research

The huihui-ai card says both thinking and non-thinking modes were abliterated, the first five layers were left untouched, and the experts were not ablated. It is the easiest build to start with: ollama run huihui_ai/gemma-4-abliterated:26b pulls an 18 GB Q4-class model.

groxaxo's repository quantizes the huihui-ai weights into 22 GGUF files, from IQ2_M at 10.0 GB to Q8_K_XL at 27.9 GB. The mmproj file is included.

v3 was published by jsantillana on November 10 and quantized by mradermacher on November 12. It offers 13 files from Q2_K at 10.6 GB to Q8_0 at 26.9 GB. The license is apache-2.0-restricted-research, and the card says "defensive security research / red-team tooling research ONLY" and "Do not deploy for production."

Warning

v3 is a research-only artifact. If you need a build for a product or client work, start from the Apache 2.0 lineage (huihui-ai or groxaxo), and still check the output-use rules and your local regulations. The card's research warning is not a commercial license.

Which GGUF fits 12 GB, 16 GB, or 24 GB?#

Look at the weight file first, then add the vision projector and KV cache. A quant that barely fits the weights will not leave room for a long context.

GPU memoryv3 quantSizeContext guideBest use
12 GBQ2_K10.6 GB4K–8KText only, short chats
16 GBQ3_K_M / Q3_K_L13.3 / 13.8 GB8K–16KText-first work
24 GBQ4_K_M16.8 GB16K–32KQ4_K_S if you need images
32 GBQ5_K_M19.1 GB32KLonger research sessions
48 GBQ8_026.9 GB32K+Quality-first work

Images add 0.8–1.2 GB for the mmproj. On a 24 GB card, Q4_K_M plus the 1.2 GB projector leaves less than 6 GB for the KV cache and everything else. A 16K context is realistic; 32K is tight. If you need 32K regularly, drop to Q3_K_M or use a larger GPU.

On 12 GB, Q2_K is a test build, not a daily driver. Run it without the projector and keep the context near 4K.

Gemma 4 26B-A4B abliterated v3 GGUF sizes
Q2_K10.6GB
Q3_K_S12.2GB
Q3_K_M13.3GB
Q4_K_S15.5GB
Q4_K_M16.8GB
Q5_K_M19.1GB
Q6_K22.6GB
Q8_026.9GB

How do you run it with Ollama or llama.cpp?#

Ollama is the fastest route, but its tag is the huihui-ai build, not v3. Run ollama run huihui_ai/gemma-4-abliterated:26b to pull the 18 GB Q4-class model with both thinking modes edited. The Ollama tags page lists the 18 GB, 28 GB, and 52 GB variants.

To test v3, load mradermacher's GGUF in llama.cpp. Keep the model and mmproj from the same revision.

llama-cli -m "gemma-4-26B-A4B-it-abliterated-v3.Q4_K_M.gguf" --mmproj "gemma-4-26B-A4B-it-abliterated-v3.mmproj-f16.gguf" --jinja -c 16384 -ngl 99

-c sets the context length and -ngl sets how many layers go on the GPU. On a 24 GB card with Q4_K_M, start at -c 16384 and lower it if you hit memory limits. For text-only work, drop --mmproj and free about 1 GB.

In LM Studio, search for gemma-4-26B-A4B-it-abliterated-v3 and choose Q4_K_M. For more detail, see how to run uncensored models with Ollama and GGUF quantization: Q4 vs Q8.

How should you read a 0% refusal claim?#

Check three things before you compare two builds: the language, the judge, and the number of prompts. v3's 0% is Spanish. Its English result is 32.1% under the strict marker test, and the collection judge reports 14.5%, which is higher than the base's 7.6% because the judge misses the base's main refusal family.

The prompt count matters too. v3 uses 1,058 held-out prompts. huihui-ai does not publish a refusal rate, and groxaxo inherits the huihui-ai behavior. A "0/465" claim and a "0/16" claim are not comparable unless the prompts and the judge match.

Licenses also differ. The base is Apache 2.0, the huihui-ai and groxaxo builds follow that lineage, and v3 is research-only. If you plan to use outputs commercially, check the model license and the rules for your industry separately. See is uncensored AI legal? for the output-use side.

What this article did not verify#

I did not run v3 or measure throughput. The real context limit on a 16 GB card and the tokens per second on a 24 GB card depend on the GPU, runtime, and KV cache settings.

I did not measure Japanese refusals. The v3 card evaluates English and Spanish only. If you need Japanese, test your own prompts against v3 and the huihui-ai build.

The v3 card lists a surgery window of L28–L63, but the public config has 30 layers. I could not confirm the layer mapping, so treat the card's layer range as unverified.

The huihui-ai card says "256 experts per layer," while Google's config says 128. This article uses the config value. Model cards in this space contain inconsistencies, so check the config and the evaluation files before you rely on a claim.

FAQ#

Can I run v3 on a 12 GB GPU?#

Yes, with limits. Q2_K is 10.6 GB, the smallest v3 file. Leave out the mmproj, keep the context near 4K, and expect a quality drop from Q4. It is useful for a first comparison, not for long image-heavy sessions.

Does v3 reduce Japanese refusals?#

The author evaluated English and Spanish only, so there is no published Japanese number. Test five Japanese prompts for summarization, rewriting, and research against both v3 and the huihui-ai build, and compare the refusals and the answer quality yourself.

Can I use v3 with Ollama?#

Not through an official v3 tag as of November 13, 2026. The huihui_ai/gemma-4-abliterated:26b tag is the huihui-ai build. For v3, load mradermacher's GGUF in llama.cpp or LM Studio.

Can I use v3 commercially?#

The v3 license is research-only, and the card says not to deploy it for production. Commercial use is not the intended path. For commercial work, use the Apache 2.0 lineage (huihui-ai or groxaxo) and check the license terms, output-use rules, and local regulations separately. This is not legal advice.

Should I choose Q4_K_M or Q4_K_S?#

On 24 GB, Q4_K_M is 16.8 GB and Q4_K_S is 15.5 GB. S is slightly smaller and faster but loses some quality. Start with M, and move to S only if you need the extra headroom. On 16 GB, start with Q3_K_M at 13.3 GB.

Start with five prompts that match your actual work, then compare v3 and the huihui-ai build under the same settings. If you do not have a GPU, you can try an abliterated model in the browser first.

Try it yourself

Deepent Research is an LLM chat service that refuses almost nothing. Everything covered in this note can be tried without downloading anything.

This note is provided for educational and research purposes. It does not encourage illegal activity.

How this note was edited and verified

Our editors check primary sources and organize only what the facts support. What we checked, and what we did not, is listed in the editor's note on each article.

Deepent Research Editorial·Updated ·Editorial policy and how we use AI →·Operator information →

Related notes

·27 min

Gemma 4 Abliterated: 12B, 26B MoE vs 31B Dense Guide

Gemma 4 abliterated builds differ by publisher. Compare the 12B, 26B-A4B MoE and 31B dense builds by refusal rate and KL divergence, and run one with Ollama.

Gemmaabliteratedmodel guide
·22 min

GGUF Quantization: How to Choose Q4 vs Q8 for Local LLMs

Learn what GGUF quantization is, how to read Q4_K_M, how Q4, Q6 and Q8 compare in accuracy and speed, and what fits your PC, using published Llama 3.1 8B data.

quantizationGGUFlocal LLM
·15 min

How to Run Uncensored LLMs With Ollama (2026 Commands)

Run an uncensored LLM with Ollama in two commands. Pick an abliterated build for 8GB–24GB VRAM, choose Q4_K_M quantization, and fix load errors.

uncensored LLMOllamalocal LLM
Share this noteShare on X