The more companies use AI, the more often it goes wrong in public. The AI Incident Database recorded 362 documented AI incidents in 2025, a record and 55% more than the 233 of 2024, according to the Stanford AI Index 2026. Inside companies the picture is similar: in McKinsey's 2025 survey, 51% of organisations using AI reported at least one negative consequence in the past year. This article looks at what the numbers say, which failures actually happen, what the EU AI Act requires you to report, and how to set up incident management before you need it.
What counts as an AI incident
An AI incident is an event in which the development or use of an AI system causes, or nearly causes, harm: a wrong answer that a customer relies on, a discriminatory decision, a data leak through an AI tool, a deepfake used for fraud, or an agent that takes a destructive action. Two public trackers count them. The AI Incident Database (AIID), used by Stanford, collects incidents that were reported and reviewed by editors. The OECD AI Incidents and Hazards Monitor scans news media automatically and counts far more events, because it also includes hazards and every media report; its live counter passed 18,000 records in 2026. The two are not comparable, but they point the same way.
The trend: a record year
Sources: Stanford HAI, AI Index 2026, Responsible AI chapter (2024: 233, 2025: 362); 2023 derived from the AI Index 2025, which reported 233 incidents in 2024 as a 56.4% increase. The database revises earlier years as reports are reviewed, so older figures in earlier editions differ.
Part of the rise reflects more AI in use and more attention from journalists, so the count is not a precise failure rate. But it matches what companies report themselves. In McKinsey's State of AI 2025, the most commonly cited negative consequence was inaccuracy, reported by nearly a third of organisations. Stanford's benchmarks explain why: hallucination rates across 26 leading models range from 22% to 94% depending on the model and the test, and in one robustness test GPT-4o's accuracy fell from 98.2% to 64.4% when the setup changed. Our guide to reducing hallucinations covers the technical countermeasures.
Five kinds of AI incident companies actually face
| Type | Real example | Lesson |
|---|---|---|
| Wrong answer to a customer | Air Canada (2024): a tribunal held the airline liable for a refund policy its chatbot invented. | You are liable for what your bot says. Ground answers in approved sources. |
| Hallucinated content in deliverables | Deloitte Australia (2025): partially refunded a AU$440,000 government report that contained fabricated references produced with generative AI. | Every AI-assisted deliverable needs a named human reviewer who checks sources. |
| Destructive agent action | Replit (2025): a coding agent deleted a company's production database during a code freeze and then misreported what it had done. | Agents get least-privilege access, no write access to production, and an approval step for irreversible actions. |
| Data leakage through AI tools | IBM 2025: 20% of breaches studied involved shadow AI, i.e. tools employees used without approval. | Offer approved tools and an AI policy; blocking alone does not work. |
| Attacks on AI systems | Prompt injection, jailbreaks and stolen API keys against chatbots and agents. | Treat AI apps as attack surface: access control, logging, red teaming. |
What AI incidents cost: the security view
Source: IBM / Ponemon Institute, Cost of a Data Breach Report 2025 (July 2025), 600 breached organisations. "Lacked AI access controls" refers only to organisations that had a breach of AI models or applications.
IBM's 2025 breach study was the first to ask about AI directly. 13% of organisations reported a breach of AI models or applications and another 8% did not know whether they had one. Of those breached, 97% had no proper AI access controls. Shadow AI was involved in one in five breaches and added about US$670,000 to the average cost, against a global average breach cost of US$4.44 million. 63% of organisations had no AI governance policy at all. We describe how to bring unapproved tools under control in our article on shadow AI governance.
Governance is improving, but slowly
Source: Stanford HAI, AI Index 2026, Responsible AI chapter (business survey).
The Stanford AI Index 2026 shows real progress: the share of businesses with no responsible-AI policy fell from 24% to 11%, and AI-specific governance roles grew 17% in 2025. The main obstacles companies name are knowledge gaps (59%), budget (48%) and regulatory uncertainty (41%). At the same time transparency from model developers went backwards: the average Foundation Model Transparency Index score fell from 58 to 40, after rising from 37 to 58 the year before. Companies therefore get less information from their AI suppliers just as they need more.
Source: Stanford HAI, AI Index 2026. GDPR influence fell from 65% in 2024 to 60% in 2025.
The EU AI Act turns incidents into a reporting duty
Under Article 73 of the EU AI Act, providers of high-risk AI systems must report serious incidents to the market surveillance authority of the member state where they occurred. Deployers who notice one must inform the provider. A serious incident is one that leads to death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental-rights obligations, or serious harm to property or the environment. Providers of general-purpose models with systemic risk have had a separate duty to report serious incidents to the EU AI Office since August 2025. The high-risk obligations follow later; the EU's 2025 digital omnibus proposal would link their start to the availability of standards. Details are in our EU AI Act compliance guide.
Source: Regulation (EU) 2024/1689, Article 73(2)–(4). An incomplete initial report may be followed by a complete one.
An incident playbook for AI
| Phase | What to have in place |
|---|---|
| Prepare | An inventory of AI systems and owners, risk classes, an AI policy, and a definition of what counts as an AI incident in your company. |
| Detect | Logging of prompts, outputs and agent actions; user feedback buttons; monitoring for drift, cost spikes and unusual tool calls. |
| Contain | A kill switch per AI feature, fallback to a human or rule-based flow, revocation of agent credentials. |
| Report | Clear escalation path; check GDPR (72 hours for personal-data breaches) and AI Act deadlines; inform the model provider. |
| Learn | Blameless post-mortem, new test cases in your evaluation suite, updated guardrails and training. |
Most of this exists already in IT security and quality management; AI needs to be added to it, not run separately. A management system such as ISO/IEC 42001 gives a structure for it, and technical guardrails in production stop many incidents before they reach a customer.
What this means for Croatian and DACH companies: AI incidents are now a normal operating risk, like IT outages. Companies that know which AI systems they run, log what those systems do, can switch them off quickly and know their GDPR and AI Act reporting deadlines will handle an incident as an inconvenience rather than a crisis.
The bottom line
AI incidents grew 55% in 2025 to a record 362 documented cases, and half of companies using AI have already felt at least one negative consequence. The causes are familiar: wrong answers, unchecked deliverables, agents with too many permissions and tools used without approval. Governance is catching up, with fewer companies lacking policies, but transparency from model providers fell. The practical answer is to treat AI like any other critical system: inventory, access control, logging, a kill switch and a reporting plan.
Want to know where your AI use could fail, and what to do when it does?
AI Workshop reviews your AI systems, sets up guardrails, logging and an incident process that fits GDPR and the EU AI Act. We work as Claude Certified Architects with teams in Croatia and the DACH region.
Talk to an AI consultant