When attackers hijacked Instagram accounts early June by tricking Meta’s AI support chatbot, most of the coverage focused on the breach itself. But this incident is a great illustration of a broader and more critical problem: the security industry has invested heavily in controlling what AI says, while largely ignoring what AI is authorized to do.
Meta’s bot verified nothing about who was asking. It just helpfully did what it was told to do — up to and including sending the attacker a confirmation code to make sure a new email address was valid. Until we start applying more mature authorization frameworks to AI agents, we’ll have more incidents like this.
What Actually Happened
The attack itself was straightforward. The attacker spoofed the location of the victim using a VPN, which circumvented certain protections that would have triggered if the attacker’s location was far from the victim’s. The attacker then asked an experimental Meta chatbot to add a new email address to the account. The chatbot emailed verification codes to confirm the new address was valid. It was trying to be helpful! The attacker verified the new email address, was presented with an opportunity to reset the password, and thus gained control of the account.
Most attacks are not one simple hole that can be patched. They string together vulnerabilities to escalate privileges or take over valuable accounts. Based on the attack details that have been publicly shared from this incident, the failures in this vulnerability chain included: relying on IP location to determine if additional security measures are taken; allowing a chatbot to modify a user’s primary email; requiring verification codes only from the new email address and not the old; and treating those verification codes as enough to allow for a password reset, which the chatbot facilitated. Guardrails around any of these would have stopped this version of the attack.
Authentication vs. Authorization — and Why It Matters for AI
Authentication is who someone is. Authorization is what they can do. Authentication is a comparatively better understood issue with AI agents, but authorization decisions reach deep into the bowels of applications and are usually business-model specific. They were often designed either for software designed by humans or slow-moving humans. AI agents combine the speed of software with the innovation of humans, finding edge cases and holes at scale.
Even with perfect authentication, the deeper failure in the Meta incident is that the agent was authorized to perform account-takeover-equivalent actions. And that’s the part the industry is underinvesting in.
Why AI Projects Are Especially Prone to This
Stapling an AI chatbot into a support system didn’t introduce a new class of vulnerability. But it makes such holes more likely to exist, because the efforts to make an AI project successful biases systems toward over-permissioning.
The larger problem is that we’re exposing services, functionality, and APIs to AI agents without properly addressing the actual helpfulness of them, nor how attackers can leverage them to find and exploit existing holes. In this case, Meta wanted the chatbot to be helpful and useful, which requires access. But they gave too much access.
This pattern is already showing up elsewhere. In 2024, an AI agent was tricked by users into sending $47,000 in crypto even though it was explicitly instructed not to. A Lenovo chatbot was manipulated into exposing session cookies based on a crafted product query — not quite direct account access, but the same underlying goal.
These aren’t isolated flukes. They’re early signals of a broader problem that will only grow as agents are granted more authority. If this attack pattern were applied to a healthcare application, the account takeover motion would be similar, but the consequences would be dire — private health data shipped off in violation of privacy rules, for example.
What Needs to Change
This incident is not a common problem – yet. But based on the trajectory of AI rollout, the number of incidents seems certain to grow as more agents are given more authority.
There are a few authorization controls that should be non-negotiable before an agent goes live. First, limit agents to the bare minimum that is useful, and expand the scope cautiously. Second, use both manual and automated evaluations to determine the true bounds of what an agent can do. Third, evaluate the access provided and determine what needs human review and how that review should take place.
Old Mitigations, New Difficulty
Security is multi-faceted and always has been. The same mitigations that worked in the past such as defense-in-depth, principle of least privilege, and human oversight, work here too. They just need to be applied systematically. The principles are old and well-understood. What’s new and hard is applying them to systems that are non-deterministic, probed at scale, and with surprisingly broad access.
The industry is pretty focused on keeping AI from saying bad things. That’s fine, as long as we don’t completely overlook whether AI should be allowed to do what it’s trying to do. Until we start holding ourselves to more mature authorization frameworks, we’ll continue to have more incidents like this.
_____
Author Bio
Dan Moore is the senior director of CIAM strategy and identity standards at FusionAuth, where he drives B2C product direction and serves as a primary spokesperson on authentication, developer identity, and the emerging challenge of securing AI agents. With over 25 years of software engineering experience — including stints as a CTO, AWS certification instructor, and engineering manager — he is a regular speaker at identity and security conferences and a sought-after podcast guest on authentication and developer security. He is the author of Letters to a New Developer (Apress) and a contributor to 97 Things Every Cloud Engineer Should Know (O’Reilly).
Join our LinkedIn group Information Security Community!
