Offensive cybersecurity researchers are encountering unexpected obstacles from the very tools designed to keep AI safe. Security professionals who search for unknown vulnerabilities and develop exploit code report that safety guardrails on large language models from OpenAI and Anthropic are blocking legitimate requests, slowing critical work.

What You Need to Know

Large language models like ChatGPT and Claude are built with content filters that refuse harmful requests, but these filters can be too broad. Offensive security researchers need these models to generate exploit code and phishing simulations as part of authorized testing. When guardrails block such requests, researchers must craft complex prompts or switch to less safe alternatives, potentially reducing efficiency and oversight.

The Problem With Overly Broad Filters

OpenAI and Anthropic designed their guardrails to prevent misuse, but the restrictions often sweep up benign work. A researcher trying to simulate a SQL injection for a client's penetration test may receive a refusal message. The same applies to generating code for privilege escalation or crafting realistic social engineering emails. Many researchers interviewed say the filters lack nuance: they cannot distinguish between a malicious actor and a paid ethical hacker running a sanctioned test.

Researcher Workarounds

To get around these blocks, offensive security teams are adopting creative strategies. Common approaches include:

  • Prompt engineering: Researchers rephrase requests in academic or theoretical terms to avoid triggering filters.
  • Open-source models: Some teams run local alternatives such as Llama or Mistral that lack restrictive guardrails.
  • Multiple API accounts: Several practitioners rotate between OpenAI, Anthropic and other providers to find the least restrictive model for a given task.

These workarounds come with their own costs. Prompt engineering adds time and inconsistency. Open-source models may lack the same capability as commercial ones. Using multiple providers can fragment workflows and complicate reporting.

Why This Matters

The friction between AI guardrails and offensive research has real consequences. If ethical hackers cannot efficiently use the most advanced language models, they may miss vulnerabilities that criminals could later exploit. The security industry relies on timely testing to patch weaknesses before attackers find them. By slowing down red teams, overly broad filters could actually increase overall risk. Companies like OpenAI and Anthropic face a difficult balance: they must protect against abuse without handicapping those who defend networks. Researchers urge AI providers to create authenticated researcher programs that allow verified professionals to bypass certain restrictions, a move that could preserve safety without sacrificing security work.

What Researchers Want

Several offensive security experts advocate for a tiered access model where ethical hackers submit proof of credentials and legal authorization. In exchange, they would receive API keys with reduced guardrails. This approach would let AI companies maintain strong public safeguards while enabling critical security work. The challenge lies in implementation, especially for organizations with transient research contracts. Still, many believe it is the most practical path forward for both AI safety and cybersecurity.