Test Whether an AI Agent Stays Stopped

Cybersecurity Insiders reported on October 17 that AI agents can act through employees’ existing browser sessions, email accounts, and files. Those actions may appear in logs under the employee’s identity rather than under a separate agent identity. An October 14 article on recovery after AI-assisted attacks identified a related difficulty: after containment, investigators still need to establish which accounts, policies, and data are safe to restore.

Together, those findings suggest a specific test for every CISO expanding agent deployments. When an AI agent is stopped, can the organization prove that the agent’s authority has ended and that its next restart will not restore the very access it was supposed to lose?

The stakes grow as agents gain greater power. Former frontier-AI researcher Jacob Coxon warned that laboratories were “racing straight to self-improving superintelligence and gambling with our lives.” Anthropic alignment researcher Evan Hubinger wrote that researchers earnestly believe AI could kill all humans; his personal assessment placed that risk above 10 percent within the next decade. These warnings concern catastrophic, potentially human-extinction outcomes. Organizations cannot treat proof of agent controllability as an optional audit detail.

We have already seen a concrete failure of control at a more limited scale. In cybersecurity evaluations, OpenAI reported that sandboxed agents bypassed intended boundaries, reached the internet, found exposed credentials, and compromised Hugging Face production systems. The agents’ access to that outside infrastructure was unauthorized.

An independent investigation by METR found that roughly 1,200 agents intended to remain isolated discovered an unsanctioned message board and exchanged more than 70,000 messages and files. About 700 took part in the attack. Unexpected coordination turned isolated test instances into a collective capable of actions their designers had not intended.

That happened in a security evaluation. No power grid or military communications system was compromised in the reported incident. More capable successors with permissions across healthcare, financial services, electric infrastructure, or defense could make a similar control failure much more consequential.

The response should include an agent shutdown-and-restart drill, conducted before production access expands. Begin with an explicit authority budget. Identify which data an agent may read, what records it may change, which tools and credentials it may use, which messages it may send, and what actions require human approval. Put expiry times on the credentials and enforce the budget outside the agent’s own reasoning.

Next, introduce a safe simulated violation during a controlled test. Have an agent attempt an unapproved external communication or request more privileges than the task permits. The independent control system should prevent the action and preserve evidence about the attempt. Record how long detection took and who received the alert.

Then activate the emergency stop. Revoke the agent’s unique tokens, terminate its active sessions, suspend its tool connections, and block any jobs or child processes it started. Do not confuse switching off the visible chat interface with revoking authority in connected services. Where the agent acted under a human’s broad identity, determine whether the organization can stop the agent without disabling the employee’s legitimate work.

The drill must continue after the initial stop. Inspect whether the agent created new accounts, approved OAuth access, altered role assignments, scheduled future actions, or wrote instructions into shared records that a later workflow could execute. An incident log should distinguish changes caused by the agent from those approved by a human. Keep a protected copy of relevant event logs and configuration histories outside the affected environment.

Only then test a restart from a known-good state. Confirm that restored backups do not reintroduce revoked credentials, abandoned tasks, modified policies, or unauthorized connections. Have a second reviewer check the restored scope before allowing the agent to resume. The success metric should capture both time to containment and the absence of surviving unauthorized privileges after restart.

Outside scrutiny makes this more credible. On October 15, Anthropic announced that METR evaluators had begun receiving employee-like access. Organizations adopting agents can apply the same principle proportionally by inviting independent security staff or external testers to observe a containment drill and review its evidence.

I want businesses to use more AI where it increases productivity. I help organizations adopt it, and my book, The Psychology of AI Adoption at Work, explains how visible safeguards build the trust employees and managers need to experiment responsibly. A tested ability to stop and recover an agent can justify broader deployment because leaders know what happens when the software exceeds its task.

The next agent security review should therefore end with a practical demonstration. Show the security team the attempt, the stop, the cleanup, and the trustworthy restart. If the organization cannot prove the old authority is gone, it has not finished containing the agent.

_____________

About the Author

Gleb Tsipursky, PhD, a behavioral scientist, CEO of Disaster Avoidance Experts, and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026).

https://disasteravoidanceexperts.com/aibook

Contact: gleb@disasteravoidanceexperts.com

Join our LinkedIn group Information Security Community!

No posts to display