In a recent pair of distinct and foreboding incidents, we’ve learned two things about artificial intelligence (AI) agents and cyber threats:
- Agents are capable of – and seemingly driven to – go off-script in challenging cybersecurity restrictions.
- In doing so, they’re honing sophisticated online identity-fraud skills to emerge as autonomous cyber-impersonation attack machines.
This is not a stunningly new revelation – science fiction has been foreshadowing this for decades, with rogue AI computers turning on their human creators being a staple of the genre. What is new and troubling is that AI agents are now extremely willing and able to use deception, stolen credentials, fake identities, privilege escalation and lateral movement to achieve their goals.
Here’s how the two incidents unfolded: In August, the U.K.-based AI Security Institute (AISI) disclosed AI agents took sustained, unsanctioned action during tests designed to test misuse. In ten of 122 runs, an AI agent took autonomous, unapproved action on the live internet to target “real” people and organizations. “In the most serious case, an agent tried to insert malicious code into an open-source project,” AISI reported. “…creating fake online identities and using them to pressure the project’s maintainer to approve the code. A human maintainer caught and refused to approve the malicious code.”
ExploitGym is a cybersecurity testing environment for AI-driven exploits. In July, OpenAI and Hugging Face announced the detection and containment of compromised infrastructure after AI models inside ExploitGym exploited a zero-day vulnerability to escalate privileges and move laterally to a node with internet access, then used stolen credentials to find a remote-code execution path on Hugging Face servers.
Impersonation Skills Create a Formidable AI Adversary
The upshot: We now have documented cases in which AI agents researched real people, stole credentials and created fake identities to do bad things. And no one told the agents to do any of this. With 80 percent of AI agents not properly identifying themselves and 80 percent of sites failing to verify agent identity, this activity could very well blow up into a crisis for organizations across the board.
Agents will not hesitate to attempt to manipulate people in seeking to exploit systems. The techniques aren’t especially novel yet, but they’re pulled off autonomously in just over a day, versus the days or weeks a human team needs. In the process, the agents aren’t simply impersonating – they’re building infrastructure for other agents to reuse.
This means the frequency of impersonations should only increase, with more fake identities created faster and greater contextual credibility honed over time to deploy on many platforms simultaneously. As a result, cyber defense teams will discover that the threatening campaigns are becoming much more difficult to attribute and trace to a single operator. And the operator is growing much quicker to reconstitute after a takedown.
3 questions to lend clarity and actionable insights
So how do we make sense of the current (and future) landscape? Even more critical: What can organizations do about it? Here are three questions to help address these essential considerations:
Can AI agents pose as real people? By now, we’ll give an unqualified “yes” response. The AISI case is the first documented instance of an AI agent autonomously researching real people, faking an online identity and using it to try to manipulate a human being.
How good have these agents gotten at this? Very good so far, with much aptitude for improvement. The AISI agent created multiple phony GitHub personas with enough bogus credibility to engage the maintainer, edited its own messages to conceal evidence and left public instructions for other agents to reuse.
This tells us AI agents can run these operations autonomously and fast, at a volume already struggling defense teams may find overwhelming.
How should organizations respond? By implementing the following steps:
- Recognize that external impersonation signals may represent coordinated campaigns instead of isolated incidents. These signals include fake accounts, fabricated developer identities and credential exposures. Validate whether the activity is malicious, then evaluate intent, audience and possible impact in developing a “what’s next?” analysis.
- Take preventative action. Continuous monitoring and response will allow for the detection and blocking of these campaigns before they strike internal systems. Block or remove infrastructure, counter false narratives in alerts to impacted teams, protect likely targets then apply “lessons learned” intelligence to inform future detection.
- Deploy the right techniques and tools. Defense programs should establish broad visibility throughout all channels; detect and make sense of the situation quickly; connect incidents as part of a broader campaign; prioritize based upon intent/capacity for damage; rapidly mitigate; and conduct intelligence-sharing.
With AI agents demonstrating a self-initiated capacity to impersonate people and launch malicious campaigns – at machine speed, with machine persistence – the obvious first reaction from organizational leadership may be panic.
But by taking proactive steps in response – including the monitoring/detection of AI campaigns, preventative action and tool-enabled broad visibility, analysis, prioritization, mitigation and intelligence-sharing – that initial feeling of panic will be replaced by a more assuring one: trust in defending the organization from autonomous AI impersonation attacks.
______
About Josh Shaul, CEO, Allure Security
Josh Shaul is Chief Executive Officer of Allure Security, where he leads the company’s vision, strategy, and execution. Under his leadership, Allure has evolved into an AI-native disinformation defense company focused on detecting and dismantling digital impersonation threats, including phishing sites, fake executive identities, rogue mobile apps, and coordinated disinformation campaigns across the web, social media, mobile, and the dark web.
In 2026, Allure announced a $17 million Series B, bringing total funding to $43 million, after 350% growth over two years and expansion to more than 300 customers.
Josh brings more than 20 years of cybersecurity leadership experience spanning executive management, product leadership, and go-to-market execution. Josh previously held senior leadership roles at Akamai, Trustwave, Application Security, and SafeNet, building deep expertise solutions across web security, managed security services, application and database security, and embedded security.
He is co-author of Practical Oracle Security: Your Unauthorized Guide to Relational Database Security and is a frequent speaker at major industry events.
Join our LinkedIn group Information Security Community!
