The AI industry is beginning to confront a question that security leaders know all too well: what happens when innovation moves faster than the ability to control it? Calls to slow down and carefully pace AI progress by Dario Amodei, Sam Altman, and other AI leaders are intensifying, accompanied by much darker warnings from researchers about the catastrophic consequences facing humanity if powerful systems outpace the guardrails designed to keep them in check. But while much of that debate is focused on what AI could become, CISOs are already seeing the security implications of rapid AI adoption play out across the enterprise today.
AI is rapidly multiplying the risk CISOs must manage. Every new agent can mean another identity to secure, another set of credentials to track, another web of privileges stretching across enterprise systems to control, another sprawl of APIs, tools and systems to monitor, and another portfolio of models and LLMs to worry about. Unlike the human identities security teams have spent decades learning to wrangle, agents can be created, connected, and given access at a pace traditional access controls were never built to handle.
That problem is now harder because agents no longer come from one centralized AI or engineering team. Sales teams are deploying them to automate operational tasks. Developers are using coding agents to accelerate productivity. Other business functions are connecting agents to SaaS applications, databases, and internal systems. Agents are being deployed in a democratized manner, and the barrier to deploying an agent is getting lower. The number of identities and privileges security teams must govern is moving in the opposite direction with the latest ratio of non-human identities to humans being 144:1.
Access can start small. In some cases, an agent may be acting autonomously and might initially receive permission to perform a narrow task. Then it needs another dataset, application, or tool to complete something new, so another permission is added. In other cases, it may act on behalf of a user and inherit that person’s changing access. In more advanced scenarios, an agent may even reason that it needs additional privileges to complete an assigned task and seek a path to acquire them. Access does not always have to be explicitly granted or inherited, either. For example, earlier this year, an AI coding agent used by startup PocketOS discovered AWS keys embedded in code and used them to delete the company’s entire database and its backup in just nine seconds, demonstrating how agents can discover and misappropriate credentials they were never intended to have. And an agent, by itself, could spawn an entire swarm of agents which may inherit the parent agent’s accesses and privileges, and further acquire their own.
The result is “privilege creep,” where an agent becomes progressively more powerful over its lifetime, often without anyone intentionally designing it that way. But access is not the only thing that can quietly accumulate. A similar dynamic is emerging with the resources agents consume, creating a financial dimension to the same problem.
Token creep and privilege creep are two sides of the same coin
As an AI conversation or workflow grows, more contextual information can accumulate and be passed back to the model, even when all of it is no longer necessary. That token creep creates inefficiency and drives costs. We’ve all heard about the Uber CTO publicly talking about how they blew through their entire annual AI budget in 4 months.
Privilege creep follows a similar pattern. Long-lived agents interact with more systems, perform more actions, and acquire more permissions. In some cases, they can discover credentials (passwords, secrets, tokens) and misappropriate them. Over time, those credentials and privileges tend to remain in place. Access gets added far more easily than it gets removed.
Both problems expose a fundamental weakness in how organizations manage long-running AI systems: accumulation without continuous reassessment. Organizations need to keep asking what an agent actually needs, both in terms of the access it holds and the resources it consumes.
Establish a persistent identity for every agent
Agents cannot be governed effectively if they aren’t consistently identified. The challenge is that an agent’s composition and its AI bill of materials (AI-BOM) can change over time. It may begin using a different unsanctioned model, connect to new MCP servers local or remote, or operate with different credentials with varying permissions and scopes. Security teams therefore need a persistent, cryptographically secure identity that allows them to uniquely identify the evolving agent and unambiguously attribute actions to it as its underlying components evolve.
That consistent identity gives security teams a foundation for access decisions. Before they can determine what an agent should be allowed to do, they need to know which agent is asking and trust that identity.
Evaluate the risk of every agent in real-time
An agent’s risk profile can evolve significantly over time. Its identities, credentials and scope of access can keep changing, becoming long-lived in some cases or weak in others. Similarly, the data an agent accesses can vary in sensitivity as well. In some scenarios, the agent may have access to Personally Identifiable Information (PII), Payment Card Industry Data Security Standard (PCI-DSS), or Protected Health Information (PHI), which can introduce greater levels of risk. The posture of the agent itself also could be a source of risk, including the use of unsanctioned LLMs and MCP servers. Runtime access matters as well, such as whether the agent has access to more tools than necessary.
All of these risk dimensions and signals need to be captured and fed into the governance model of the agent. This enables organizations to continuously evaluate risk and adjust governance as conditions change.
Treat privileges as dynamic, not permanent
Traditional access controls were largely designed around relatively predictable human roles and behaviors. Agents can operate much faster and across a much broader collection of systems and tools, making controls designed around human activity difficult to apply at the same speed and scale.
Instead of treating permission as a one-time decision, organizations should continuously evaluate whether an agent still needs its privileges.
If an agent gained access to a database for a task it completed three months ago, should that access still exist today? If the employee the agent operates on behalf of changes roles, should the agent automatically inherit every new permission? If its behavior or composition changes, should its existing access remain untouched?
An agent’s access and agency should evolve with its responsibilities. As those responsibilities change, security teams need to reassess what the agent still needs and remove privileges that no longer serve a purpose.
Monitor what agents actually do at runtime
Knowing what an agent can access is only half of the equation. Security teams also need visibility into what it does with that access.
That means continuously observing which systems an agent connects to, which credentials and privileges it uses, the tools it calls, and whether its behavior deviates from established patterns. That runtime information can help organizations build a more complete risk model around each agent, incorporating both what the agent is made of and how it behaves.
With this level of visibility, controls become contextual. Instead of simply asking whether an agent has permission, security teams can ask whether an action makes sense for that agent right now, based on how it is behaving and the risk involved.
Enforce least privilege at the point of action
Visibility alone is not a silver bullet for preventing privilege accumulation. Enterprises also need granular policies that can govern agent activity wherever it occurs, including MCP tools, command-line tools, files, and other systems.
Runtime enforcement lets you limit what an agent can do without removing the autonomy that makes agents valuable in the first place. The goal should not be to add a human approval step before every action. Instead, establish guardrails that let agents operate freely within appropriate boundaries while preventing unnecessary privilege escalation.
Those boundaries should extend to what an agent can consume. An exploited agent could burn through an organization’s AI budget or quota, making token-consumption limits another important part of agent governance.
Unchecked accumulation is the real risk
As agents operate for longer periods and become more deeply connected to enterprise systems, some change is inevitable. Their responsibilities will expand. Their context will grow. Their access requirements will evolve. What cannot become inevitable is unchecked accumulation.
Securing AI agents requires moving beyond the question, “What access did we give this agent?” Enterprises need to keep asking, “What access does this agent need right now? What is it actually doing with that access? What should it no longer have?”
And access is only part of the equation. As agents become more autonomous, organizations also need limits on what they can consume and, ultimately, what they can cost. Those boundaries will become even more critical to maintain as agents take on broader responsibilities, make more decisions, and consume greater resources. Maintaining visibility and control over these dimensions is key to preventing unchecked accumulation as agent autonomy grows.
_____
About Archit Lohokare
Archit Lohokare is the Chief Executive Officer at AppViewX, the market leader in AI Agent and Machine Identity Security. He joined AppViewX through its acquisition of Eos, a venture backed AI Agent Security company that he founded and led as its CEO.
Archit brings over two decades of experience as a senior enterprise software product and strategy executive. Prior to founding Eos, he led the Workforce and Endpoint Security business at CyberArk (acquired by Palo Alto Networks) as its Senior Vice President & GM. Archit joined CyberArk through the acquisition of Idaptive, where he was the Chief Product Officer.
Throughout his career, Archit has led go to market teams and driven new product innovation across Cybersecurity and SaaS, from Zero trust, Identity, and Data security to Sales Performance Management and CRM at larger organizations like Symantec (now Broadcom), IBM and smaller startups, three of which have seen successful exits.
Archit’s mission is to envision, build, and take to market products that solve complex business problems, create value for customers, and drive growth. Archit holds an MBA from UC Berkeley-Haas School of Business and a Bachelor’s degree in Computer Engineering from NTU, Singapore. He lives in the San Francisco Bay Area with his wife and two children.
Join our LinkedIn group Information Security Community!
