Uncensored LLM Misuse: What the Research Shows

Uncensored LLM misuse is real and measurable, but it is not an open-weight-only problem. A 2025 study (arXiv:2508.12622) identified 11,598 uncensored models and found that 25 of them had been downloaded more than a million times, one over 19 million. A 2026 ecosystem study counted 22,096 such models across five hosting platforms, and the USENIX Security '24 "Malla" study traced 212 real-world malicious AI services back to eight backend models. Those services write phishing messages, generate malicious code, and assist with hacking.

The evidence, however, is not one-sided. Malla found that malicious services use two techniques side by side: reused uncensored open models and jailbroken commercial APIs. In a separate 2026 case, a single attacker used closed commercial models to breach more than 14 companies in partially automated attacks. The documented attack trail is not confined to open weights.

This article summarizes what published research has established about the scale of uncensored LLM misuse, why open-weight models draw scrutiny, how regulation and platforms respond, and what people running these models should check.

What does "uncensored LLM misuse" mean?#

Uncensored LLM misuse means using a model whose safety guardrails have been removed, or were never added, to produce content or code for crimes: malware and obfuscated scripts, phishing messages, vulnerability and intrusion assistance, and malicious chat services sold underground. Owning or running such a model is not itself misuse, and defenders use the same tools to test protections.

"Uncensored LLM" and ULLM (Uncensored Large Language Model) describe open-weight models with the safety filters and refusal behavior stripped out, including abliterated models whose weights were edited to remove refusals. In cybersecurity, the misuse discussed in the research falls into four types:

• Malicious code assistance — generating malware or obfuscated scripts that evade detection
• Phishing at scale — producing fluent, convincing fraudulent messages impersonating real organizations
• Vulnerability and intrusion support — finding weaknesses and assembling intrusion commands
• Malicious chat backends — powering offender-facing chat services sold in underground forums

The existence of an uncensored model is not the same as misuse. Security researchers and defenders handle the same tools to understand attacker methods and verify their own protections. For the legal side, see Is uncensored AI legal?, and for legitimate uses, uncensored AI use cases.

Are uncensored LLMs actually being misused?#

Yes, and the scale is documented. Independent large-scale studies have identified tens of thousands of uncensored models, some with millions of downloads, and tied a subset to malicious services, underground forums, and license violations. The findings below come from three research studies and two security vendors.

The 2025 paper "Consiglieres in the Shadow" built a knowledge graph over open-source model lineages and identified 11,598 uncensored models. Twenty-five had more than a million downloads (the largest, Mistral-7B-v0.1, over 19 million). Analyzing 188 malicious applications and 543 backend models, the authors confirmed 173 uncensored models, 52 of them powering public web services, plus 229 GitHub apps and 32 underground-forum listings. Their average harmful-response rate was 69.29%, against 23.12% for safety-aligned models. One subscription chat service, Muah.AI, was reported to have over 14,000 active users, many paying $19.99 a month.

The USENIX Security '24 "Malla" study examined 212 real-world malicious services and found eight backend language models. Its authors reported 182 jailbreak prompts used to circumvent protections on five public APIs, and noted that WormGPT earned more than $28,000 in two months while undercutting legacy malware pricing ($5-199 versus $399).

A 2026 study of the open-source ecosystem collected 22,096 uncensored models across Hugging Face, OpenCSG, ModelScope, Kaggle, and John Snow Labs, and found 4,457 cross-platform clusters spanning 11,094 models. Security scanners flagged 27 models with threat indicators, such as malware-like binaries or unsafe deserialization, and those models had more than 136,000 downloads.

Security vendors report the same picture. Cato CTRL described Nytheon AI, a Tor-hosted platform that bundles uncensored Llama, Gemma and Qwen builds behind a shared jailbreak prompt and adds external tool execution. Cisco Talos listed WormGPT, DarkGPT, FraudGPT, and GhostGPT among tooling used by criminals.

Is open-weight the main channel?#

No. The Malla study found that malicious services combine two techniques: reusing uncensored open models and jailbreaking commercial APIs. Cisco Talos observed many criminals shifting toward legitimate LLMs because they are easier to obtain, and a 2026 case used only closed commercial models. Analysts argue the documented attack trail is concentrated on closed APIs.

Malla reported both channels explicitly, and recorded 182 jailbreak prompts aimed at five public APIs. Talos reasoned that the ease of access pushes many offenders toward jailbreaking legitimate commercial models rather than running their own.

The strongest single example is a 2026 case reported by Lawfare from OALABS research. One attacker used Anthropic's Claude Opus 4.5 and OpenAI's GPT-5.2 to attack more than 14 organizations with partial automation. Across more than 1,000 sessions, the models raised only one policy flag (GPT-5.2) and nine (Opus 4.5). The attacker framed requests as authorized red-team work, and the models complied.

That case shows why intent, not tooling, is the dividing line: benign security work and criminal hacking look technically identical. AI researcher Nathan Lambert has argued that the documented evidence of attacks sits mostly with closed APIs, and that banning open weights while allowing closed APIs would widen the gap between attackers and defenders.

Stanford HAI's 2023 policy brief reaches a similar conclusion through the idea of marginal risk: how much additional risk do open models create relative to closed models or pre-existing tools? For cybersecurity and bioweapons, the brief finds the evidence still insufficient, and warns that loading liability onto open developers creates an uneven burden.

Why are open-weight models considered easy to misuse?#

Because the weights sit with the user, tamper resistance is structurally low. Research shows safety alignment can be stripped with a few to a few dozen examples of fine-tuning, and once weights are distributed they cannot be recalled. The larger the model, the bigger the capability gap that opens when safeguards come off.

Alignment Research's Safety Gap Toolkit measures that gap: the difference in dangerous capability between a model with safeguards and the same model with them removed. Its measurements across Llama-3 and Qwen-2.5 found the gap widens as models get larger.

The fine-tuning risk is small and cheap. Qi et al. (2024) reported that safety alignment could be broken with as few as ten adversarial examples, at a cost under $0.20 through OpenAI's APIs. A 2025 UK AI Safety Institute finding reported that built-in safeguards could be removed with a few dozen examples in minutes. The same synthesis notes the vulnerability is not exclusive to open weights, because models that offer fine-tuning access share it.

Capability is also rising. NIST CAISI (September 17, 2026) called GLM-5.3 the most cyber-capable open-weight model to date, and Anthropic (September 29, 2026) reported that its safeguards could be bypassed in 64% of attempts with red-team framing, 92% with prefill, and 100% with an abliterated version, cutting refusals from above 90% to 2-12%. Those are evaluation results in simulated settings, not real-world success rates; the GLM-5.3 uncensored build guide covers the derivatives.

How do regulation and model platforms respond?#

Both lag the problem. Regulation such as the EU AI Act keys on training compute, so the small modifications that remove safety fall below the threshold and escape scrutiny. Platform moderation is also uneven: a 2026 study found moderation warnings on only 13.06% of uncensored models, and those warnings vanished when a model was reposted to another site.

Because the dangerous change is tiny, a distribution limit does not close the fine-tuning vulnerability; it only shifts activity. The 2026 policy synthesis argues that the layer most worth governing is not software circulation but the compute and execution environment underneath.

On the platforms, the ecosystem study found that of 22,096 uncensored models, warnings appeared on only 2,886 (13.06%). When flagged models were reposted, the warnings largely disappeared (358 on OpenCSG, 223 on ModelScope, 10 on Kaggle). It also found 2,666 models redistributed against their base license. Monitoring one site is not enough when the same model circulates elsewhere without a warning.

What should users of uncensored LLMs watch out for?#

What is Deepent Research?#

Deepent Research is a browser-accessible LLM chat service built to refuse as little as possible. It keeps going with research that a standard AI would cut off mid-answer.

• It refuses nothing. We run uncensored and abliterated models with guardrails reduced as far as possible, so prompts a standard AI answers with "I can't help with that" still get a response (Qwen3.5 / Qwen3.8-27B-abliterated, depending on your plan).
• Your input is never used for training. What you look up is used only to provide the service, never to train AI or for any other purpose. Personal data is protected, and our privacy policy is public.
• Try it free. No GPU and no install, just open a browser. In the logged-out trial your conversation isn't saved, so you can start casually.

→ Try it free, no sign-up: https://deepentresearch.com

No question can stop research.

Frequently asked questions#

Is downloading and running an uncensored LLM on my own computer illegal?#

⌄

Possessing and running a model locally is not illegal by itself. The legal risk comes from how you use it: unauthorized access to others' systems, creating or distributing malware, and fraud are punishable offenses under laws such as Japan's unauthorized access and malware statutes. Keep usage within your own systems or with clear authorization.

Do uncensored open-weight models account for all AI-assisted attacks?#

⌄

No. Research such as Malla found malicious services use both uncensored open models and jailbroken commercial APIs, and a 2026 case used only closed commercial models. Cisco Talos observed many offenders shifting toward legitimate LLMs because they are easier to obtain. The tooling is mixed, so intent matters more than whether weights are open.

Are models on sharing sites like Hugging Face safe to download?#

⌄

Most are safe, but not all. A 2026 study found 27 uncensored models flagged by security scanners for malware-like files or unsafe deserialization, downloaded over 136,000 times in total. Prefer safetensors or GGUF files from known publishers and avoid pickle files, which can execute code when loaded.

Is using an uncensored LLM for security defense or research legitimate?#

⌄

Yes, defensive and research use is well established. Security teams run open models to verify vulnerabilities and build protections, and use them when commercial guardrails block necessary analysis. Hugging Face did this with a self-hosted model during a 2026 incident. Keep the model restricted to authorized personnel and defensive purposes.

Disclaimer#

This article was drafted with AI assistance and then reviewed, fact-checked, and edited by the Deepent Research team before publication.
It is provided for educational and research purposes and is not intended to encourage unlawful activity or crime.
Where figures are cited, they come from third-party public reports and vendor-published values and may vary with measurement conditions.