Why 95% of Healthcare AI Never Ships

0:00Welcome, everyone. This is From Pilot to Production: Why 95% of healthcare AI never ships and what the other 5% do differently. Thanks for joining me this afternoon. The title of this has a statistic in it. You've seen that statistic It's been in every vendor deck for 2 years now, and it's never really helped anybody. So let me get out of the way in the first minute It's an MIT Media Lab report from last August 95% of generative AI pilots show no measurable return. They counted getting to production with real KPIs as the bar All industries, not just healthcare, were included And the sample is really quite small, I think roughly 52. So the authors just said it was directional. So I'm not going to lean on this. What's really worth your time is what MIT found underneath it

0:53The failures weren't the models, they were really workflow and adoption And the pilots that worked were the ones somebody embedded in one specific workflow rather than run from a central AI team. I'm Bob Klein. I'm CEO of Digital Scientists. I'm not going to read all this to you, but really the point of putting all this together is to kind of share lessons learned in this space. And it's up to you to decide the kind of the value of it.

1:25But let me move on. So 5 Healthcare AI systems, all five reach production. Medical coding, MDS, ambient documentation, vitals, monitoring in the OR And referral intake, all five belong to the client It's not changing So again, this slide's really five healthcare AI systems. They all reached production. All five belong to the client, not to us Two of them are products that companies sell, and we were the team that built them engineers and designers who did that work One of them on screen. That's the list. Here's one of them. So it really isn't just a list. It's health context AI. It's ambient documentation running inside never alone today. NeverAlone is a virtual care platform That's really Never Alone's navigation behind the panel. Nobody opens a second tool to get to this

2:44It's listening to a telehealth call, tag the clinical terms as they heard them. One of those tag types is PHI because it has to know what it's handling It suggested ICD-10 codes, and there's a save and cancel because a person decides Generally, it generates a SOAP node as well. It's HIPAA compliant, encrypted at rest in a transit, and the whole thing runs inside the boundary The point is, it reads your rules, decides, a person signs off, everything is logged, remember those four, they come back.

3:18So before I go on and show anything, there's a couple of us on the call labs, risk adjustment, triage, pharmacy, virtual care. My assumption is the folks on the call are trying to learn from others and with lessons learned. Just wanted to throw that out there and see if somebody might want to we could go around the room and get any kind of feedback Kind of, you know, one of the reasons why demos stall is building got easy and shipping didn't And in healthcare, the gap between those two is the widest it has ever been. That gap is the 95% failure rate It is an incompetence, it isn't a bad team building did get easier. Nothing downstream of building got easier And in healthcare, downstream is really where all the work actually is.

4:39So I wanted to ask, you know, there are a couple things I keep hearing, and these are really stand-ins Of these four things I keep hearing, I guess put the one that's closest to what you're hearing in the chat and I'll read them back. Right? And see if there's something there. Referrals still come in as faxes and somebody reads every one Our best coder does it in their heads, nobody else can do it. The nurses tried it for 2 weeks and went back to the phone Every health system wants a different version of it So, has anyone else kind of hearing those kinds of things or seeing those things in your organizations?

5:25So the point about prototypes, why they fail is your prototype will never touch PHI It isn't a smaller version of the real thing. It's a completely different object. And it's not because your team isn't good, because it's really Nothing about it was built to touch PHI. There's no BAA, no audit trail, no EHR on the other end. It can demo forever on synthetic data It will never really, on any timeline, become the real thing, because it's not a version, a smaller version of the real thing, it's a different object altogether Has anyone here on the call gotten around, you know, gotten a BAA around a pilot Anyone kind of willing to volunteer something about that?

6:19Then if you have, then you're past the first problem, right, is dealing with real Real PHI, and the one you have left to deal with is integration and who owns it when When you're done. So the four things that we can prototype never has, and it's never the code Is real data Production data is PHI dirty, contradictory, late kind of legally radioactive. Most demos die here about three months in. That's the faxes one that I read earlier So how do you deal with the faxes? Real data is a challenge. Real integration, epic point click here, Cerner, each with an owner, a change window and an opinion That's the every health system wants a different version one The next one is real guardrails.

7:14HIPAA, the BAA, Security Review, Audit Trail, access control, either it was in the design from the start or it wasn't Lovable doesn't really start with that, does it? In healthcare, this is what turns a 6-week project into an 18-month nightmare. Sorry, did I say nightmare? 18-month one Real ownership, somebody's phone rings at two o'clock in the morning and the nurse is waiting. If you can't name that person, you don't have a production system, you have a demo with a good uptime That's the nurse's one, right?

7:48And here's the part that matters for this room. If you sell software, all four of these belong to your customer the health system Their security review, their integration queue, their CMIO, which makes it harder, more repetitive And tie directly to how fast you close deals So The two things so what do the 5% do differently? It's two things First, one, they decide before they build, not more strategy One workflow chosen on purpose that can pay back this year When building was hard, difficulty made the choice for you You could only do one thing, so you did the obvious one Now you can do 10 things, which means you can do 10 wrong things.

8:40Two, they start from the constraints, not the demo. PHI, the BAA, the audit trail, who signs off, which EHR is on the other end? The ones that ship design for those in week one So the ones that stall meet them after the demo works when it is a rebuild and not a fix. So two things, not five and not a framework Okay, so that's what the 5% do differently. So my question for the folks on the call, one question, can you name that workflow right now?

9:17And you name it the data it runs on and the person who owns it If you can't That's the first one Right? And if you can, and it still isn't live, that's the second one, the second question, right? So you have to be able to name it and then you've got to to understand the person who owns it So I don't… I don't know your businesses the way you do, but if I had to guess where I'd start in each one of them, it looks a bit like this. So I've taken the liberty of putting some details out there around where you might find the workflows that generate the value that you're after. So whether it's diagnostics at volume, you know, you've got incomplete test orders and you're Chase the ordering office and escalate the rest. The health plan we all know about prior auth prior auth case assembly. You pull the evidence, check the policy

10:14flag what's missing before a clinician ever opens it Risk adjustment, of course, pre-visit prep Nurse triage, post-call follow-through, did the visit happen? Did the medication arrive Long-term care pharmacy has to worry about prescription exceptions A post-acute network, referral coordination, who has capacity, who answered, who's still sitting unplaced Listen for it in your own building. Every morning someone checks. We're always waiting for after the handoff, nobody knows whether it's like that's just how organizations run.

10:47And it helps narrow your focus onto the right kind of workflow to start with. So I've included a little more detail here about how to pick the workflow that pays back And I just cover some of these details, some of this information around evaluating. I think I include a link to a resource at the end about this, but it's really five tests, a strong choice passes all five manual load, you actually spend hours on it today. There's clear return, doing it faster moves money, risk, or capacity.

11:23There's contained downside. If it's wrong, it's only a recommendation. A person can override it There are fewer integrations, you can start from one or two inputs. It's a visible when the team sees it working in weeks, not quarters. So give me one of yours and say it like this if you can, when this trigger happens, the agent reads this, decides this one bounded question, and sends the exceptions to this person So here's mine, so you can hear the shape When a post-acute referral arrives, the agent reads the discharge requirements and the provider data Identifies qualified facilities, requests capacity, tracks responses, and sends uncertain placements to the discharge coordinator.

12:10That's an example of a workflow Here's a real life example I wanted to include. A client in Los Angeles runs a recuperative care center. They have roughly 30 beds. It's a medical respite for homeless patients coming out of Southern California hospitals Okay. Hospitals send the same referral to four or five facilities at once and the first to respond wins I didn't know this actually existed this way, but it creates a little kind of a stressful time period where the center has to evaluate a lot of information and respond. So the window is 15 to 20 minutes Whether you know you can win, you know, you basically get a new patient or not And before this, the staffer had to notice that the email came in, open dozens of pages of attachments, read them, and decide

13:14On 30 beds, you know, you miss a referral, it is a real share of your capacity against a payroll that just doesn't change. So, and in this case, there, you know, people were out, they're on vacation. The staffing was a challenge About a month after their first email to us, they had this running end-to-end The referral arrives, AI reads the packet, their rulebook applies, their own admission policy Staff gets a secure alert A person on that staff accepts or declines with the source document on screen, and every action gets logged Right, so one of our senior engineers paired directly with the operator and his referral manager Not a whole project team with handoffs, just one engineer, and the two people

14:05Who knew the word cold, right? Know how it works today So why it worked as a first project, the worst it could do was recommend something a person could then override Okay. Yes, there's a couple questions about referral intake. Absolutely. There's a chance to review, right? So it's not automated. They actually have to sign off on it The point here is that AI reads, your rules decide. A person signs off. Everything is logged The model never decides and never has the final call. So those are good questions It's… your written policy has to decide the same way every time, so you can always explain why. And this has always been a challenge. And it's like, it's not acceptable to have a black box These are really kind of feels more like automated business rules, right? But the rules have to be applied consistently and can be overridden

15:03So, two decisions kept it explainable You know And there you have to be able to respond. So in this case, for example, when someone tried to let the AI judge whether an infection was stable on antibiotics, it proved unreliable So we made it a hard rule that sends every such cases to a person Right, so it's slightly changed the workflow And we launched it in shadow mode, you know, while people still decided so we can compare the two before it held any authority You know, and for a health system, that's… it's really about safety For others, it's really a sales story That's… that's the sentence your, you know, your customer CMIO repeats to their own risk committee Which is the difference between a deal that closes this quarter and one that sits in review.

15:54Now, the health systems need to be able to be able to rely on what's there and be able to adjust it as needed. So One more topic on that. The test gate, every release had to pass a live end-to-end test, not a lab test. On every change, a real email with fake patient content into the live inbox, the actual cloud pipeline processing at the recommendation checked against the known correct answer in the staff screen opened in a real browser to confirm the card appeared.

16:30For some of these things, we had real patient data and we had generated data for patients, right? So We were able to build the AI on on historical information from all kind of previous records. So even going through this, you know this process, we caught two things before they reached production There is a payer matching bug that would have accepted a patient whose insurance wasn't covered. Right, so some of that might have been some alignment between back and forth with insurance companies And there's a shortcut in the error handling that would have truncated real data. So we had to address that But 51 test referrals with known answers covering every path a real referral could take Nothing reached production without passing.

17:24Our engineer built that gate as part of the work, not as not as a QA phase kind of bolted on afterwards. It was That's kind of what separates the two. It's not my discipline, it's really it's theirs. It's We know that we have to get it right, and it was a collaboration with a client So And I think this is where it comes down to what it is that gets built or delivered, whether it's in your application or in your services, or something you deliver to clients And that's something I would advise you to ask of anyone if you're looking for these kinds of services is The code, the rules, the pipelines, open standards, you know What is it that you own? What is it that you keep?

18:20Do you get to keep all this under your own cloud account, under signed BAA, with a model inside that boundary Patient data never leaves it and never trains somebody else's model, and no new platform It goes into the system that you already run And that's… for us, that's really function as build, operate, transfer The challenge for some of these applications that they have to be maintained right so they have to be updated. This isn't a SaaS platform that just lives on its own. You know, clients have to be able to maintain it And update it with new business rules And for some To achieve the kind of the output that they're after, they have to… they just have to continue to do that. So there's a support requirement

19:05for anything that gets built in this way And if they're not maintained and updated, then you're not going to get the value that they can provide. And that's honestly, that's the case with any piece of software. It's got to be supported, it has to maintain be maintained, has to be updated So so when I say own it, I mean it literally. Owning it is a commitment and not a trophy Right? So yes, in this example, the client does own the IP It's their IP And We're really the partner and It's… that's the whole point of build, operate, transfer. So I think I think that's important. That's not necessarily how every Every company works these days. I would say that's not the Palantir model So the Anyway, I'll climb down off my soap box, but it's I think there is a lot of value there that's obtainable

20:08But there is also investment of energy and time to build out these workflows and maintain them. So it's also possible if that client asked if they just wanted us to manage and support it, that's something we do as well, which is At a very reasonable cost, right? So Kind of budget for the second year, name the person who owns it. You know, we're usually trying to get things into production and live and supported And I like to say I like to be part of somebody else's success story So Three ways to start, right? And all of it is free, and none of it needs your email. I put together some resources for the folks on the call. We'll start Q&A after this.

20:55If you might be doing it yourself in terms of scoring your own workflows, what kind of opportunities are there internally? So I have a link here to a AI workflow opportunities scorecard. There are 5 tests. It's a worksheet you can download You can check also your own bench And I include what I think are the capabilities that you need to bring to one of these projects, and you need, you know to use your teams in-house Or the last one is really, you know, a chance if you want to come, if you want to have a conversation with me, I'm happy to chat about it, bring me the one that you're stuck on The point is that is to get started, you know, and To decide this is kind of build it internally or work with a partner, score a partner. These are just I thought were resources anybody could use

21:52Whether you work with us or not, it's fine. Okay. One thing I wanted to put in there next, just really the last last slide is the model is about 20% of the build and there are other pieces of things that have to get done Around assurance and delivery, orchestration, data integration, PHI and compliance. The Some of the goal there's Some model agnosticism, right? So we're not overly dependent on any one model. We try and optimize what's the best model to use, right, and be transparent about model selection There's other areas here around cost and things like that, but I just wanted to include that in there. Because it is something to think about The effort that's involved We'll go through Q&A now. I've got

22:54For everyone on the call, or anyone who registered will include access to this recording And the slides, so you're welcome to take a look at those. As I mentioned, you're welcome to book a working session or just to chat with with me, and I would probably pull in our principal engineer And also just keep in touch. I want to thank everybody for coming today Okay, I got a couple questions here. Let's see Is there a human in the loop to review the referral intake?

23:29There is a human in the loop. The referral intake, it's more of refer, you know, reviewing the decision. Of course, the human has access to everything that the The AI had access to But some of the aspect, the value there is being able to turn around a decision about that prospective patient very quickly. So you have 15 minutes and they might multiple ones might come in at the same time. And I want to say that I think they had probably five employees that did this And a couple of them were out at the same time. So, yes, there's a human in the loop making the final decision. Some of what got involved here was Emails back and forth, review of all the documentation, validation against The kind of patients they were able to support

24:24Like levels of wound care, you know, all these kind of business rules that had been documented and I think what was great about this client, and they're a small nonprofit, I'm really a big believer in what they do but They needed the efficiency, they needed the automation And yes, there are other things that they're going to want that come out of this in terms of of reporting and things like that, but they're… all those things are easy… I mean, are not difficult to generate from running a workflow like this, but It's not that we changed everything about the way that they worked, we just automated some aspects of it that were being done kind of almost like in a rote way By employees. And I think

25:11By freeing up these employees, they're able to do something else, or they're able to make more kind of strategic decisions. They're given information at the right time But yes, they have to learn to trust it and know like where that's coming from. What was the other question Right, so I enter the reporting Shadow mode is… is kind of like a parallel mode, I would say, right? So, it can operate in parallel, and You know, so it's not necessarily… it's kind of comparing the… your existing process to a new process to this… to this AI workflow And comparing the results, the decisions, the times, things like that. So with with one workflow You know, you're looking to get it integrated into the operation and made productive and put into production as soon as possible, but you… it's got to be trusted. You know, you want to be implemented

26:15I think that's a challenge for anything. For one of the examples we have, an MDS project that went on for years, we carried it all the way to production And Honestly, we really had to work hard to earn the trust of the nurses, the MDS coordinators were involved with it and they saw anything that was wrong, they had a problem with it. So Like I said, it can't be a black box. I need to understand where it comes from how the decisions are made, where the data's from. So that was a particularly challenging Example Are there any other questions?

27:00Well, again, thank you to everybody that came today, or registered I'm Bob Klein, this is Digital Scientists, and this has been This has been From Pilot to Production: Why 95% of healthcare AI never ships and what the other 5% do differently. And I challenge everybody on the call To flip those numbers, right, there's no reason why it shouldn't be 95% ships And let's move beyond just the prototype. Healthcare needs our help. Thanks so much.