Healthcare providers have moved beyond asking whether to adopt AI and are now focused on deciding where to apply it.

Three-quarters of U.S. health systems now use at least one AI application, up from 59% in 2025, and half run three or more, according to an industry survey published in March.

Determining where to invest can be difficult, experts told Tennr. When healthcare executives are offered a hammer as powerful as generative AI, everything can start to look like a nail.

The most common use cases still center on automating administrative tasks, like documentation, revenue cycle, and coding. But the fastest growth is in patient communication and appeals. Use of AI to draft replies to patient texts grew 80% year over year, and AI-prepopulated technical appeals grew 50%.

The best offerings deliver significant ROI ahead of schedule, according to an October analysis from Bain & Company. Healthcare buyers included in the consultancy’s analysis projected an average of 24 months to ROI for new tools. The typical team saw value in half that time.

Appeals management, treatment recommendation, clinical trial coordination, prior authorization, compliance assurance, and follow-up care solutions are currently some of the tools with the highest average ROI, according to the report.

Still, a broadly popular solution might not match a specific organization’s needs. During a recent webinar, Tennr asked three healthcare transformation leaders to share how they would evaluate new solutions, from which problems they’d address first to how they’d measure success.

Here’s what they shared.

Start where skilled labor is misallocated. Many organizations begin transformation projects with processes tied to collections and billing volume. Blake Sollenberger, managing director at Evergreen Healthcare Partners, suggests also looking for places where outdated processes pull credentialed staff away from the work they were hired to do.

Payer requirements have pushed nurses, physicians, and therapists into utilization review, documentation, and other administrative work. Moving that work off their plates can free clinician time, reduce after-hours charting and open appointment slots.

“Those people are highly skilled resources. They’re very expensive, and they should be doing the work that’s at the top of their license,” Sollenberger said.

Treat build versus buy as a capability question. Lincoln Richards, partner at Albaron Partners, said general-purpose models have advanced to the point where an AI-fluent employee can replicate much of what point-solution vendors pitch. Even when an organization decides to buy, that employee is essential to the evaluation.

“They’re going to be the right person to evaluate any tool if you do go look for one,” Richards said. “Otherwise, you’re in over your head, and you can’t really understand how to implement, track and judge outcomes.”

Building carries its own risk. Tom Antony, head of healthcare AI at Shore Capital Partners, said his team has built internal tools that staff stopped using because the output wasn’t quite right.

“If a tool is off a little bit, that means it’s harder to trust. You might see people just not using it, especially if it’s close to clinical processes or outcomes,” said Antony.

Antony’s point speaks to the fact that many providers have not yet built out robust testing for AI. A report from the Center for Connected Medicine at UPMC and KLAS Research found that 63% of health systems surveyed describe their AI strategy as developing or ad hoc. Only 44% have a dedicated platform for testing tools before rollout. Respondents named limited resources, insufficient time, and a shortage of specialized talent as the leading barriers.

Antony’s rule of thumb is to buy when speed to a proven solution matters. He builds only when the need is highly specific or the result could become a durable proprietary asset. Sollenberger added that internal tools need data engineers to maintain them, and in many markets those engineers are as hard to hire as experienced nurses.

The Bain study points the same way. Providers are moving away from internally built AI tools in 2026, and organizations report that half or fewer of those builds are still in active use. It appears teams can stand up a working prototype quickly, but few can operate, monitor, and update projects.

Design pilots that can prove out the full concept. Sollenberger said many organizations pilot new tools on their simplest workflows. A revenue cycle tool tested only on traditional Medicare claims says little about Medicare Advantage, where payers don’t follow Medicare’s prospective payment rules. Results from a visit-based specialty may not hold for procedural lines or lengthy inpatient stays.

“You have to kind of always be thinking: Will these metrics still be applicable when we want to apply it to enterprise?” he said.

Antony described organizations getting stuck in "pilot purgatory," extending trials because early results appear promising but haven't been measured against predetermined benchmarks. He stressed the importance of clearly establishing thresholds to continue, expand, or axe a project.

"The ability to cut off a pilot when it's not working is just really... it's hard, obviously," Antony said. “But it’s really important to make the decision and then move on to the next thing.”

Choose tools that make life easier. When solutions are patient- or employee-facing, they must easily fit into existing workflows, experts said, or they’ll likely be abandoned in a matter of weeks.

Sollenberger compared good AI to air conditioning.

“If you’re sitting in a room and you don’t notice the air conditioning, that means it’s working beautifully,” he said. He suggests asking vendors to shadow a staff member or patient before implementing technology to make sure they fully understand the current workflow and ideal user experience.

A technology that works wonderfully for one population might be completely inappropriate for another, the panelists said.

Voice agents proved a particularly polarizing example. Richards believes that a new patient’s first call to a provider should be with a human, because the call shapes their view of the practice.

However, Antony’s portfolio uses voice agents as the first line for new-patient scheduling, with patient satisfaction roughly equal to human staff. On referrals, automated outbound scheduling even converted better than human outreach.

“I think it’s actually the speed to contact,” Antony said.

Sollenberger described a self-pay collections vendor that A/B tested a more lifelike voice. Patients were more hesitant to share card details with an agent that sounded human.

“The robot’s not going to go use their credit card info to buy shoes,” he joked.

Phase the rollout to minimize unexpected problems. Automation changes work beyond the department that adopts it. One team becoming more efficient creates more volume for the next team in line. Phasing changes lets a team absorb new volume gradually.

Panelists’ final recommendation was to map the “blast radius” of a deployment to manage the impact of changes downstream.

Antony’s teams build a value stream map before launch to identify every function a project will touch, while expecting it to be incomplete: “You could model it out, but you’re probably gonna be wrong.”