AI adoption is accelerating across organisations, from governments to banks to various private companies. Internal copilots, automated search, decision support systems, and agent-based tools are being deployed at speed. Yet the most serious risk AI introduces is not in the model, the algorithm, or the output. It lies with data access, visibility, categorisation, and management.
Most organisations do not have a clear, operational understanding of their own information environments. They cannot say, with confidence, what information they hold, where it resides, which parts are sensitive or what their AI systems can actually access or infer.
AI systems rarely create risk in isolation. They amplify whatever data environment they are placed into. If that environment is fragmented, poorly classified and only partially understood, risk scales silently.
From invisible weakness to regulatory reality
The EU AI Act is the first regulation to force this exact issue into the open. Its implications extend far beyond the EU and are already being felt in the US. Any organisation operating in Europe, selling into European markets or supplying European customers will be affected, either directly through compliance obligations or indirectly through procurement pressure, as European buyers increasingly require demonstrable data control from vendors, suppliers, and technology partners upstream.
For high-risk AI systems, the Act requires demonstrable control over data quality, governance and handling. In practice, this means being able to show, in operational terms, what data feeds an AI system, where that data originates and how access is controlled at runtime. This is where many organisations falter. AI is being deployed on top of unindexed file systems, legacy archives, SaaS platforms, collaboration tools and vendor environments that were never designed for machine-level access. The AI Act is forcing attention first through regulation, but the operational risk is universal.
The questions most organisations cannot answer
When AI governance moves from policy documents to real systems, the gaps become obvious.
Most organisations cannot reliably answer:
• What information they actually hold across internal systems and third-party platforms
• Where that information resides and how it moves between systems and vendors
• Which data is sensitive, regulated or mission-critical versus incidental or obsolete
• What internal AI tools can access, retrieve, infer, or present without explicit user intent
Without these answers, governance exists only on paper.
Why AI governance efforts are quietly breaking down
From work with public institutions, municipalities, banks and mid-sized organisations, the same failure modes appear repeatedly.
Here are the failure modes seen most often:
1. No reliable inventory of information
Organisations cannot govern what they cannot itemise. Data sprawls across email systems, file shares, SaaS tools, archives, backups and supplier platforms. Inconsistent labelling makes operational vs critical distinction difficult. One-off audits capture a moment in time, and subsequently become outdated.
2. Sensitivity is assumed, not classified
Few organisations can consistently distinguish public, confidential, personal, regulated, and mission-critical data across systems. Policies exist, but enforcement is uneven. With overlapping rules and labels, data is stored everywhere and labels are not consistent. Vague guidance, data overload, fragmented tools and inconsistent enforcement prevent organisations from maintaining a clear operational understanding of what data requires protection and why.
3. AI systems do not respect assumptions
AI tools operate on permissions and retrieval logic, not intent. If a system can see data, it will use it.
4. Governance is imposed after AI is embedded
Copilots and AI features arrive bundled into productivity platforms. By the time governance frameworks are written, access paths already exist.
5. Risk is evaluated theoretically, not operationally
AI governance often stops at documentation, committees, and training. Very few organisations test what actually happens when AI interacts with real data under stress: misconfigurations, compromised accounts, hostile inputs or supplier failures.
These are not hypothetical risks. IBM reports that a measurable share of organisations have already experienced breaches involving AI models or applications, with the overwhelming majority lacking proper AI access controls. Researchers have demonstrated cases where AI systems retrieved and exposed sensitive internal data without user interaction. This occurred because the ingestion and retrieval processes happened automatically.
In the public sector, supplier breaches have shown how centralised platforms can expose millions of records at once, not because AI was misused, but because no one had visibility into what data was being collected, stored and made searchable downstream.
A different starting point for AI governance
In practice, many organisations begin AI governance at the wrong layer. They focus first on model selection, prompt restrictions, and usage policies. They assume that the underlying information environment is already known, mapped and governed.
Effective AI governance must start one layer earlier, with data visibility and control. That means:
• Automated discovery of information across internal systems and external platforms
• Continuous classification of data by sensitivity, regulatory exposure and operational criticality
• Enforceable guardrails that define what AI systems can access, retrieve, infer from, or act upon
This is also where “dark data” becomes visible: information organisations did not know they possessed, or did not realise was exposed to machine access. This often includes legacy client records, archived emails, historical case files, old contracts and data copied forward through system migrations, all of which can quietly be brought to light once AI systems begin indexing and retrieving information at scale. By keeping this security risk (and its prevention) in mind, AI adoption can then accelerate more safely because risk is constrained at the data layer.
From compliance to control
Procurement processes, regulators, insurers and boards are converging on a simple demand: proof of control. Organisations that cannot demonstrate data visibility, classification and enforceable access controls will increasingly struggle to deploy AI at scale.
The future of AI governance will not be decided by better policy language. It will be decided by whether organisations can see, classify and control their own information environments before various AI systems turn secure nontransparency into exposure.
___
About: Andreas Malik is the founder of Risk and Decision and a digital resilience specialist with over 20 years of experience working with public institutions, municipalities, and financial organisations on risk, continuity, IT security, and recovery. His work focuses on helping organisations gain operational control over their data and systems before incidents occur.
Join our LinkedIn group Information Security Community!
