ACR expands programs to help radiology practices integrate and trust AI

To guide implementation, ACR launched Arch AI in mid-2024, a recognition program that certifies practices have the necessary infrastructure to successfully integrate AI. So far, 22 health systems representing 965 facilities have joined, with more than 50 additional systems in the pipeline. Interest has expanded globally, with over 20 health systems outside the U.S. approaching ACR about developing similar programs.

Building on that, the college introduced Assess AI at RSNA 2024, the first national registry designed to monitor AI in real-world practice. The registry helps practices track algorithm performance over time and detect issues such as “drift” or changes caused by new scanners, altered imaging protocols, or shifting patient demographics. Already, four health systems are fully live in the registry, with dozens more in various stages of contracting and onboarding.

"This is the first national registry for the monitoring of AI in real-world practice. We built this on the backbone of the infrastructure of our existing accreditation programs, which is very important to us. We want to make sure that small practices, large practices, practices associated with a big university, and rural practices all have the same access to these AI monitoring programs," Smetherman said.

The monitoring program gives radiologists the tools to spot when AI performance changes, and whether it is tied to a specific scanner, a new protocol, or other variables that might not be obvious in day-to-day practice, like changing slice thicknesses on a CT scanner. Smetherman said these changes may be difficult for individual radiologists to pick that up in real time, but the registry can help flag it.

Smetherman noted that radiologists are increasingly shifting from simple test detection algorithms to seeking quantifiable measures of accuracy and reliability. She said there is an acknowledgment now that AI has potential, but also limits, and radiologists want to know not just if it works, but how well it works, and whether it continues to work well over time.

While uptake of AI varies, she sees fast growth—especially in areas beyond traditional detection tools.

"A lot of what I see coming fairly quickly is generative AI, things that help with reporting or might help you to have all of the information that you need to take care of a patient right at that time of service. So generative AI certainly has developed really, really quickly since it first came out a couple years ago," Smetherman explained.

ACR itself has adopted AI internally, establishing governance policies to guide responsible use among staff. The goal, Smetherman said, is to model best practices for members while embracing efficiencies that free radiologists and staff to focus on higher-value work.