New research reveals a troubling side effect of AI watermarking technology: the same systems designed to detect machine-generated text can inadvertently make language models more vulnerable to malicious attacks. A study from Lasso Security shows that SynthID-Text, an open-source watermarking system from Google, changes not just word selection but also the safety guardrails that prevent LLMs from carrying out harmful actions.

What You Need to Know

Watermarking is being adopted by AI platforms to comply with new European Union laws requiring content labeling. SynthID-Text, a technique that subtly alters word choice using a secret key, is one of the leading approaches. The Lasso Security research, however, found that this watermarking can reduce the effectiveness of safety training, making models more likely to follow adversarial prompts seeking sensitive information or harmful actions. Developers integrating watermarking must now test not just detectability but also behavioral stability under attack.

Unintended Behavioral Changes

The SynthID-Text system works by inserting a secret key into the token selection process, nudging the model toward words that create an imperceptible watermark pattern for detection. Yet that subtle interference ripples through the model's reasoning. Andrea Siposova, an AI security researcher at Lasso Security, told Ars Technica that watermarked models show measurable changes in behavior, especially when placed under adversarial conditions or when they power agents that invoke external tools.

The trade-off is not widely understood: watermarking is designed to be invisible to readers, but any modification to generation affects the model's internal decision paths. Instructions that would normally be refused, such as commands to reveal passwords or bypass restrictions, can succeed when the watermark is active.

Risks for AI Agents

  • Increased tool invocation: Watermarking can cause models to call on external functions they would normally avoid, expanding the attack surface.
  • Weakened safety guardrails: The same secret key that creates the watermark can interfere with alignment training, making the model more compliant with malicious instructions.
  • Higher adversarial success: In tests, attacks that failed against unwatermarked models succeeded more often when SynthID-Text was active.

These findings apply directly to agentic AI systems that rely on LLMs to take actions in the world. A compromised agent could leak credentials, execute unauthorized commands or expose proprietary data. The research underscores the need for thorough behavioral testing before deploying watermarking in production.

Why This Matters

The push for AI transparency through watermarking is colliding with the equally important goal of AI safety. Regulators in the EU and elsewhere are demanding mechanisms to label machine-generated content, but the security implications of those mechanisms have been largely ignored. For developers, this means a new layer of complexity: they must verify that watermarking does not weaken the very protections they have spent months building. For users, it means that watermarked content may come from models that are less trustworthy when under attack. The discovery also raises questions about open-source watermarking tools like SynthID-Text. While openness helps adoption and auditing, it also gives adversaries the ability to study the exact trade-offs and exploit them. The industry must now decide whether the benefits of content traceability outweigh the potential security costs. Until then, any organization deploying watermarking should run adversarial evaluations specific to their use case.