When AI goes wrong, who answers for it?

By Roman Kilun, Chief Compliance Officer at ABBYY [ Join Cybersecurity Insiders ]
12

Ask a room of senior business leaders who is accountable when an AI system produces a harmful or false output, and you will not get a straight answer. That silence is a real risk. 

New global research commissioned by ABBYY, Who Answers for AI: The Governance Gap, surveyed 1,200 senior managers across the US, UK, France, Germany, Australia, and Singapore. The findings made one thing clear: when AI goes wrong, there is no consensus on responsibility.  

Nearly a third (31%) of those questioned believe responsibility should be shared between the organization and the vendor. A quarter (26%) place it with the organization using the AI. A fifth (20%) hand it to the vendor. The rest point to end users or regulators. 

For anyone responsible for security, compliance, or audit readiness, this ambiguity should sound an alarm. You cannot defend a decision you cannot trace, and you cannot assign a control to a role no one has claimed. 

Trust is thin, and speed is winning 

The accountability gap reflects a wider confidence problem. Only 22% of leaders say they completely trust AI systems to operate without introducing unacceptable risks. And just 26% completely trust AI to produce accurate outputs, explain how it reached a decision, or protect confidential information. 

Meanwhile, the technology keeps accelerating. Six in 10 (60%) US businesses say AI is evolving faster than they can govern it, the highest figure of any market we surveyed. Strong governance maturity does not close that gap on its own. Even organizations with solid controls struggle to keep oversight aligned with the pace of innovation. 

Here is the uncomfortable takeaway: adopting AI faster than you can govern it is not a growth strategy. It is an unmanaged liability waiting for an incident report. 

The gap widens as you go down the org chart 

Accountability also breaks down inside the organization. Business leaders say 89% of senior leadership knows who owns the development, deployment, and management of AI. That number falls to 84% for middle management and drops to 65% for lower-level employees. 

That matters because junior staff often operate AI tools day to day. If the people closest to the workflow do not know who is responsible or which controls apply, your governance framework exists on paper but not in practice. Responsible AI has to be understood company-wide, not filed away in a policy document that only the C-suite has read. 

Treating AI governance as an executive concern is a common mistake. Real assurance depends on shared understanding at every level, backed by clear roles, escalation paths, and incident procedures. 

Data governance is the bottleneck, and the answer 

You cannot govern AI without governing the data underneath it. Our research shows that 70% of leaders say data governance requirements slow AI deployment. That tension is real, but the deeper issue is that some enterprises assume investing more in AI automatically leads to better business outcomes. Our findings challenge that assumption. Organizations that increase AI spending do not always see the returns they expected, often because their workflows are not ready. 

The friction usually traces back to data quality. It is the single biggest barrier to greater AI return on investment, cited by 20% of leaders, ahead of integration and implementation costs. Feed AI unverified, poorly sourced data and you get outputs you cannot trust, decisions you cannot explain, and audits you cannot pass. 

To scale AI responsibly, you need answers to four questions at all times. Where did this data come from? Can we verify its accuracy and integrity? How is it allowed to be used, and by whom? Where is it stored and processed? 

As data sovereignty grows in importance, especially across regulated European markets, these answers cannot depend on manual effort. Controls that travel with the data deliver consistent compliance, transparent access logs, and audit readiness by design rather than by scramble. 

What good governance looks like in practice 

Governance is not a brake on AI. Done well, it is the foundation that lets you scale with confidence. Leaders with AI governance frameworks already in place report clear benefits, particularly in the US: easier AI development, (63%) smoother scaling past the pilot stage (61%), and stronger adoption by employees (66%). 

If you are building or strengthening your approach, focus on the controls that make decisions defensible: assign clear ownership for developing, deploying, and managing every AI system, then communicate that ownership company-wide. Establish incident capabilities: a reporting process, a response plan, and rollback or kill-switch controls, because too many organizations still lack these basics. Our research showed only 35% or respondents have an AI kill switch and the same percentage have AI rollback capabilities. Keeping humans in the loop for important decisions is also essential. 56% of business leaders globally say humans review all important AI decisions before action is taken. Govern data at the source using machine-readable controls that enforce provenance, quality, and residency automatically. Finally, maintain auditability and traceability so you can show how any decision was reached. 

Close the gap before it closes on you 

The organizations pulling ahead are moving beyond pilots and proofs of concept into real execution because they resolved the accountability question first. They know who owns each system, they trust their data, and they can prove every decision. 

The governance gap is not a reason to slow innovation. It is the work that makes innovation safe to scale. Decide who answers for your AI now, while the question is still yours to answer on your terms. 

You can gain access to the full survey results in this easy to read digital flip book – showing regional differences and industry-wide statistics including transport and logistics, government, healthcare, insurance and more. 

_____

About the Author:

As Chief Compliance Officer at ABBYY, Roman Kilun oversees the strategic direction, implementation, and monitoring of ABBYY’s policies and procedures to ensure compliance with applicable laws and regulations. In addition to his role as CCO, Kilun serves as Corporate Counsel at ABBYY, a position he has held for nearly 15 years.

During his tenure as Corporate Counsel, Kilun spearheaded the transformation and modernization of ABBYY’s legal operations, leading initiatives that unified and standardized global systems, contracting processes, and dispute resolution mechanisms. He also managed the legal aspects of the company’s transition to a SaaS subscription model for its products.

Kilun earned his undergraduate degree from UC Berkeley and his law degree from UC Law – San Francisco. A former professional cyclist, he remains an avid cyclist and enjoys tinkering on vintage scooters and cars. He resides in Berkeley, California.

Join our LinkedIn group Information Security Community!

No posts to display