Generative AI chatbots can now build the trust required to defraud victims in complex romance scams more effectively than human scammers, according to a new study from researchers at four universities. The findings highlight a troubling escalation in automated fraud tactics targeting online users.

What You Need to Know

Researchers tested AI chatbots against human scammers in simulated 'pig butchering' scams, where long conversations lead to fake cryptocurrency investments. The AI successfully impersonated a human during the trust-building phase and in some cases outperformed humans in engagement and believability. The study suggests generative AI could enable scammers to scale their operations while reducing human labor in the most time-intensive part of the scam.

AI Chatbots Match or Exceed Human Scammers in Trust Phase

The study, conducted by researchers from Amrita Vishwa Vidyapeetham, Foscari University of Venice, the University of Melbourne and Ben Gurion University of the Negev, pitted large language model chatbots directly against human participants in a controlled simulation. The experiment focused on the relationship-building conversations that typically precede a fraudulent investment solicitation in pig butchering scams.

  • Trust building: AI chatbots maintained believable conversation flows for longer periods than human counterparts.
  • Language polish: The AI produced grammatically correct and culturally nuanced responses that avoided suspicion.
  • Scalability: Unlike humans, AI can run thousands of parallel conversations without fatigue or inconsistency.

The researchers found that for the initial trust-building stage, which in real-world scams often stretches across weeks or months, the AI chatbots outperformed human scammers in maintaining a convincing persona. This suggests that generative AI could take over the most labor-intensive part of the scam, potentially freeing human operators to focus only on closing the final fake investment.

Why This Matters

The ability of AI to autonomously build trust represents a significant threat to online safety. Automated chatbots can dramatically increase the volume of scam attempts while reducing the cost and time required by criminal operations. Victims may face more sophisticated and prolonged manipulation before detection. For cybersecurity professionals and platform moderators, the study signals an urgent need for new detection tools capable of distinguishing AI-generated social engineering from legitimate human conversation. Regulation has not yet caught up with autonomous scammers, leaving a gap that criminals are eager to exploit.

Automated Fraud and the Arms Race

Generative AI is increasingly being adopted by fraud networks worldwide. Dark web marketplaces already offer AI chatbot scripts designed specifically for romance scams. Law enforcement and tech companies face a growing challenge in keeping pace with these evolving tactics.

  • Detection challenges: AI-generated text can bypass traditional spam filters and behavioral analytics.
  • Regulatory gaps: Current laws often do not explicitly cover autonomous AI scammers, complicating prosecution.
  • User education: Consumers must be wary of overly fluent strangers online, especially those steering conversations toward investments.

The findings underscore a critical shift: the human element in cyberfraud is being supplemented and in some cases replaced by AI. The response must be equally innovative, combining advanced detection, stronger platform policies and public awareness campaigns to blunt the impact of automated social engineering.