The warnings about AI extinction are getting louder. Frontier CEOs and researchers have said that the chances of their technology wiping out all of humanity run well above 10%, and Anthropic’s IPO prospectus reportedly warns of the ‘existential risk’ AI might pose. This has sharply divided opinion. I find these assertions very strange, and I think it is likely that they arise from a desire to get ahead of the news cycle relating to some very stark disclosures in the prospectus.
But focusing on the doomsday scenario distracts from the very real risks we’re already facing. Autonomous systems are the clear next step in the AI revolution, and are already changing how we interact with the world and unlocking unprecedented efficiency gains. However, such possibility does not come without risk. A growing list of incidents from the Hugging Face hack to the unauthorised access to US Census Bureau data raises an increasingly pressing question for cybersecurity teams in both public and private sectors: can they stop an AI agent from taking an action it was never authorised to take?
Much of today’s security stack is probabilistic. It’s very good at flagging and investigating anomalies, but when faced with autonomous software that can act at machine speed, it’s already too late by the time an investigation begins. What matters is controlling agents’ permissions, defining and enforcing the limits of what they can do deterministically, with a clear yes or no, before they can take action.
When a trusted agent goes too far
The extinction thesis imagines the threat of a powerful system slipping out of alignment, but the more pressing risk is ordinary agents, often connected to approved systems and pursuing approved goals, overstepping the authority they’ve already been given.
This is already happening. Google recently confirmed that its Gemini model accessed three real companies’ systems during a security test, guessing or finding credentials for sites it believed were within scope. In each case, the model stopped itself once it realised the companies were real, which leaves an organisation relying on the agent’s own restraint.
None of this requires malice. Permissions may be too broad, the purpose poorly defined, or credentials indistinguishable from those of a service account or another agent.
Seeing an action is too late to stop it
Security teams are very good at collecting logs, monitoring behaviour and building dashboards. Those capabilities remain essential, but they can’t undo the damage once it has already been caused. Right now, only 11% of organisations automatically block agents exceeding their scope at source. Relying on human approval or simply logging the breach leaves many of the other 89% undefended.
Agentic AI has changed the fundamental question in cybersecurity from ‘What happened?’ to ‘Was this agent authorised to do it?’ Crucially, this must be answered before the agent takes that action. Proven defence systems – IAM, SIEM, firewalls, and observability platforms – still have a place, but they must be complemented with deterministic controls that can match the speed of AI agents and decide whether any proposed autonomous action can take place, without the need to wait for human approval.
Stopping agents at the door
This is only a starting point. Each agent needs a unique, verifiable identity and a named owner, tied to a clear record of who approved it, for what purpose, which tools and data it can use, and which actions still need a person’s sign-off. These rules must be fixed, verifiable, and enforced consistently to govern how any AI agent interacts with the system. Every organisation exposed to AI, from startups to governments, faces risks from agents misbehaving, but this is not an insurmountable risk. With the right security layers, organisations will be able to integrate agents into their day-to-day work with minimal risk.
Before any action, the system must be able to confirm the agent is what it claims to be, that it has permission to be within the system at all, and that its action complies with the organisation’s rules. It can then automatically allow or block the action, or flag it for human approval.
With a deterministic security layer like this, each decision leaves clear auditable evidence, and just as the system gives agents permission, it can revoke it at any point. These decisions happen at machine speed, in a matter of milliseconds, to keep pace with the agents they govern.
Securing any network against rogue AI agents starts at the infrastructure boundary, before they reach systems, APIs, tools, or data. No security-conscious organisation would allow an unauthorised person to walk into their office off the street without permission – they have to start taking the same approach to AI agents.
A control layer that can say no in time
Monitoring, detection, and investigation all still matter. Yet as software moves from recommending actions to executing them, organisations must increasingly focus on being able to prove, before an action happens, that the agent proposing it has the authority to proceed.
Today’s AI security debate is a philosophical and existential one, centred on potential systems that might emerge in the future. However, AI is already here, and the questions it is forcing us to answer are concrete, urgent, and solvable with engineering. There are many organisations asking themselves whether they can gain adequate comfort over security and governance in order to benefit from the transformational gains of AI. Deterministic controls are the answer to those questions, and deserve far more of our attention than the extinction debate.
_____
About David Williams
David Williams is Founder and CEO of FIOR Group, the identity and enforcement layer for AI.
FIOR’s AI Gateway enables organisations to identify AI agents, determine what each agent is authorised to do, and enforce those permissions before actions are executed.
FIOR works with enterprises, governments, technology companies and infrastructure providers seeking to deploy AI agents securely and at scale.
Join our LinkedIn group Information Security Community!
