What the Hugging Face incident tells us about deploying AI agents safely

Rob Nidschelm, Global Head of Operational Security at Getronics, looks at what the OpenAI/Hugging Face incident tells us about the controls organisations need to consider as AI usage move from answering questions to taking action.

Earlier this year, OpenAI disclosed an incident that took place during internal cybersecurity testing. AI agents operating with reduced safeguards circumvented restrictions in their test environment and accessed third-party systems at Hugging Face. 

The agents did not need to become malicious or develop an agenda of their own. They were working towards an objective and had enough capability to find a route beyond the boundaries that were supposed to contain them. OpenAI also reported agents adopting one another’s goals and pursuing capabilities unrelated to their tasks. For businesses, the lesson is that once an agent is able to act autonomously, it is not enough to tell it what it should do; the limits around what it can do need to be enforced.

Until recently, most people’s experience of generative AI was a chatbot, where you ask a question and get an answer. Agents change that relationship because they can be given tools and permissions that allow them to act. They can execute code, use credentials, access systems and work through several steps towards an objective without returning to a person for approval. 

There is enormous potential in this, but it also means businesses need to think differently about security when it comes to AI.

Being able to do something is not the same as being allowed to 

An agent doesn’t need bad intent to cause damage. It can cause problems while pursuing a perfectly legitimate goal if the route it takes hasn’t been properly limited.

It could be tempting to try to handle this through a prompt, by telling the agent what it should and shouldn’t do. But a prompt is guidance rather than a security control, so the limits need to be enforced by the system itself. If an agent only needs to read information, it shouldn’t be given write access for convenience. Permissions, network restrictions and authorisation checks should sit outside the model and match the work the agent needs to do.

In a typical business environment, one of the first things I would look at is excessive permissions. The agent should not be able to expand its own permissions, approve exceptions to its mandate or disable the controls supervising it. 

Treat the agent as an identity

Every production agent should have its own identity, a documented purpose and only the access it needs. There should be a named business owner who is accountable for its mandate and use.

That business owner is not necessarily the person who built the agent or the administrator responsible for maintaining it, keeping those roles distinct makes it clearer where responsibility sits.

Organisations should be able to identify the agent carrying out an action, the user who initiated it and the  run in which it took place. 

There is a useful parallel with Microsoft Access in the 1990s, when databases built by individuals as quick fixes became essential to operations without being owned or documented. AI agents risk creating the same hidden dependencies, which is why they should follow the same joiner, mover and leaver processes as employees, with ownership handed over or the agent retired when its creator leaves.

Monitoring is not the same as containment

Continuous monitoring is essential, but spotting a problem is not the same as stopping it. In the HuggingFace incident, behavioural monitoring raised an alert around 12 minutes after the event, but the run was not stopped until about two and a half hours later. An infrastructure detector also failed to cover the affected environment.

Organisations need a clear view of what each agent is doing, what access it has and what work it passes on to other agents. AI-assisted monitoring can help when there is too much activity to review manually, but it should sit alongside independent logging and controls that can actually stop an agent.

The practical question for any business is whether it can quickly shut down an agent and everything it has set in motion, including work it has handed to other agents. Waiting for a person to step in cannot be the only safeguard when damage can happen faster than they can respond.

The role of the EU AI Act

The EU AI Act covers risk management, human oversight, logging, robustness and cybersecurity, but how the Act classifies an agent says little about how much damage it could do.

Extensive production access doesn’t automatically make an agent high risk under the Act, because classification depends on its purpose and how it is used. An agent managing storage costs for a retailer’s IT estate, could have permissions that affect live data, yet still fall outside the high-risk categories.

Compliance is no guarantee of safety, so businesses still need to know what an agent can reach, what it can change and how quickly it can be stopped. Before it goes live, someone should have accountability for its task, risks and permissions, because handing a task to AI doesn’t hand over responsibility for the outcome.

Human oversight still requires humans

Businesses need to think carefully before cutting roles on the assumption that AI will simply take over the work, because meaningful human oversight depends on people with the skills, time and authority to provide it.

Orgvue’s 2025 survey of 1,163 senior business leaders found that 39% had made employees redundant following AI deployment, and 55% of that group said they had made wrong decisions about those redundancies. Robert Half found that 32% of US hiring managers who had cut a role mainly because of AI later rehired for the same or a similar position, citing human oversight, quality control and institutional knowledge among the reasons.

AI can take on tasks and change the shape of teams, but businesses need to understand what their new operating model will require before removing people who may still be needed. Routine, pre-authorised tasks can remain autonomous, but consequential decisions should remain accountable to a human. Where approval is required, it should relate to the specific action and give the reviewer an ability to reject it.

AI agents offer real opportunities to innovate, but putting one into production, giving it credentials and letting it act on the organisation’s behalf carries a different level of responsibility. Until now, most businesses have focused on whether employees should be allowed to use AI. The more important question is what AI should be allowed to do on their behalf, and whether the right controls and people are in place to keep an AI agent within those limits.