London

June 28–29, 2027

New York

September 15–16, 2026

Berlin

November 9–10, 2026

The hallucination crisis: Who owns agentic AI outcomes in production?

An AI hallucination exposed a critical accountability gap and inspired a governance playbook for safely deploying high-stakes automation.

Speakers: Sibani Mishra

Register or log in to access this video

Create an account to access our free engineering leadership content, free online events and to receive our weekly email newsletter. We will also keep you up to date with LeadDev events.

Register with google

We have linked your account and just need a few more details to complete your registration:

Terms and conditions

 

 

Enter your email address to reset your password.

 

A link has been emailed to you - check your inbox.



Don't have an account? Click here to register
November 17, 2026
NYC 27 Pre sale ticket block image

When a production AI agent hallucinated, the true crisis wasn’t technical—it was organizational. Discover the battle-tested governance playbook we built to map accountability and safely deploy high-stakes automation anywhere.

Every engineering leader has shipped something that kept them up at night. In most industries, a bad release means a degraded UI or a temporary drop in conversion metrics. In healthcare, it means something else entirely.

This case study details the rollout of AI agents into clinical workflows handling prior authorizations, discharge summaries, and medication triage across more than 2 million patient interactions annually. Built on AWS Agent Core with Agent Squad orchestration across specialised clinical agents, the architecture was sound — the ownership model was not.

Three weeks into production, a discharge-summary agent hallucinated a medication continuation instruction. No patient was harmed, but the fallout was immediate: clinicians lost confidence, overrides skyrocketed, and our engineering teams entered a finger-pointing loop. Platform engineering, ML, and clinical operations had all assumed someone else held final escalation authority. Nobody did.

This talk shares the operating playbook we built to pull our AI system out of crisis and back into production safely — and more importantly, to make sure we never ended up in that room again. We’ll cover how we drew strict ownership boundaries for agentic systems where no single service owns the full outcome, how we engineered human-in-the-loop patterns that preserve genuine accountability without becoming rubber stamps, and how we rebuilt clinical trust through review structures that gave humans meaningful control rather than the illusion of it.

The governance framework that came out of this incident has since been adopted by two teams outside healthcare entirely — because it turns out, regulated AI is just a more honest version of the AI governance problem every engineering leader is quietly avoiding.
You do not need to work in healthcare for this talk to matter- this is a blueprint for any leader deploying high-stakes automation.

Key takeaways

  • A reusable playbook for defining escalation authority before an incident forces the conversation.
  • A decision-ownership model for agentic systems where multiple specialised agents share a workflow but no single service owns the final outcome.
  • A governance structure for regulated AI that gets systems out of endless pilot phases and back into production with organisational sign-off.
  • A trust-rebuild framework for restoring stakeholder confidence after a high-stakes AI incident — applicable far beyond healthcare.
  • A human-in-the-loop design pattern that preserves genuine clinical accountability without collapsing into rubber-stamp override behaviour at scale.