Building got easy. Shipping didn’t. Someone on your team can stand up a working AI demo in a weekend, and it will look good enough to show the board. Then it stops, because a demo and a system a clinician depends on are two different things.
In this session we walked through five healthcare AI systems our team built that reached production, and what separates them from the pilots that stall. Almost none of it is about the model.
The 95% figure describes a symptom
The number has been in every vendor deck for two years and it has never helped anyone decide what to do on Monday. The research that gets cited also covered all industries, not healthcare specifically, and the pilots that worked were the ones somebody embedded in one specific workflow rather than run from a central AI team.
The useful question is not why 95% fail. It is which decisions the other 5% made first.
Real PHI changes the design, not just the paperwork
HIPAA, the BAA, the security review, the audit trail, access control. Either those were in the design from the start or they were not. Teams treat them as a gate at the end, and that is the most common reason a pilot has to be rebuilt rather than promoted.
Testing only on generated data hides the problems. On one build we ran both real patient records and generated records through the same pipeline. That pass caught two things before they reached production, including a shortcut in the error handling that would have truncated real data. Our engineer built that gate as part of the work, not as a QA phase bolted on afterwards.
Integration has an owner, a change window, and an opinion
Epic and Cerner are not technically mysterious. Each integration has someone who owns it, a window when changes are allowed, and a view on whether yours matters. Add the parts of the workflow that still arrive by fax and the integration surface is usually larger than the AI surface.
So pick a first workflow that needs fewer integrations. Starting from one or two inputs is a feature. It is the difference between something the team sees working in weeks and a six-week project that becomes an eighteen-month one.
If you cannot name the owner, you have a demo
Somebody’s phone rings at two in the morning and a nurse is waiting. If you cannot name that person, you do not have a production system. You have a demo with good uptime.
The model reads. Your rules decide. A person signs off.
A black box is not acceptable in a clinical or financial workflow, and it is not necessary. The model reads and extracts, your written policy decides, a person signs off, and everything is logged. The model never has the final call.
Your policy decides the same way every time, which is what lets you explain why any given result happened.
What it looks like when it works
One example, anonymized. In post-acute referral intake, hospitals send the same referral to four or five facilities at once and the first to respond usually wins the admission. Before, a staffer had to notice the email arrive, open dozens of pages of attachments, read them, and decide.
Now the agent reads the discharge requirements and the provider data when a referral arrives, and a person on staff accepts or declines with the source document on screen. Every action gets logged. There is a review step, so it is not automated end to end. About a month after their first email to us, they had it running. The workflow barely changed. What changed is that the reading and matching stopped being done by hand.
You own the result
Ask this of anyone you bring in, including us. Does the code run in your cloud account, under your BAA, with the model inside that boundary? Does patient data stay there, and stay out of anyone else’s training set? Do you get open standards, with nothing you cannot export? Does it go into the systems you already run rather than a new platform?
We work build, operate, transfer, and we have completed that transfer twice. Owning it is a commitment rather than a trophy, though. These systems have to be maintained and updated with new business rules as the operation changes, the same as any other production software.
Where to start
One workflow, chosen on purpose, that can pay back this year. Fewer integrations, one or two inputs, a number you already track, and a result visible in weeks rather than quarters. Decide the PHI boundary, the integration owners, and the accountable person before anyone writes code.
If your pilot has stalled, the question is usually not whether the model is good enough. It is which of those decisions has not been made yet.
