AI triage software significantly reduces radiology report turnaround times, with a caveat
An artificial intelligence triage software can significantly reduce radiology report turnaround times when assessing CT scans for pulmonary embolism, according to new research published Monday.
Such devices pinpoint patient exams containing time-sensitive findings, prioritizing them for reading in a rad’s queue. This can help them to communicate urgent concerns more quickly, leading to earlier treatment and improved outcomes, experts write in JACR.
Researchers with the U.S. Food and Drug Administration and University of Chicago recently aimed to assess one such software, seeing how it could improve efficiencies. They found clear benefit, with the device helping to trim turnaround times during the regular work schedule but not on off-hours, the analysis concluded.
“This study was motivated by inconsistent findings in recent literature regarding the time-saving benefits of AI triage devices in clinical settings,” Yee Lam Elim Thompson, PhD, a senior staff fellow with the FDA, and co-authors wrote Sept. 29. “By analyzing work-hour and off-hour cohorts separately, we found that AI triage significantly reduces report [turnaround times] during high workload periods but has minimal impact when the clinic is less busy.”
The analysis utilized over 11,000 adult CT pulmonary angiography (CTPA) scans, with contrast enhancement to visualize blood vessels in the lungs, suspected of PE and logged at UChicago between 2018 and 2022. Radiologists processed reports using software from Microsoft (formerly Nuance), with the tested triage software, BriefCase, manufactured by Aidoc. The device is intended to identify suspected pulmonary embolism in CTPA scans, with a reported sensitivity (correctly identifying a disease when present) of 90.6% and specificity (pinpointing those without PE) of 89.9%. Thompson et al. retrieved over 527,000 records from the system to help set a baseline for rad read times.
Average turnaround time from before AI was about 68.9 minutes during regular work hours, falling to about 46.7 minutes afterward, a 32.2% drop. However, during off-hours, the change was less significant, falling from about 44.8 minutes before AI down to 42 minutes after adding the software, a nearly 6.3% change. Clinically observed time savings during regular work hours (about 22.2 minutes) were deemed "significant," while off-hour savings (about 2.82 minutes) were not, the authors concluded. Researchers also used a previously developed computational model called QuCAD, which simulates and predicts report turnaround times based on various factors, such as staffing levels.
Exploration of different workflow scenarios with the computational model confirmed that small variations in workflow parameters lead to different conclusions regarding time-saving benefits from AI-based triage.
“Through this modeling, we found that time savings can be significantly influenced by clinical workflow parameters, including examination interarrival time, the number of radiologists, radiologist read time, disease prevalence, and the diagnostic performance of the AI,” the authors concluded. “Our findings highlight the critical role of clinical workflow in determining the effectiveness of AI triage devices, as time-saving benefits may not occur in every clinical scenario. The computational model, QuCAD, can assist users in understanding the clinical conditions under which an AI triage device can deliver optimal time savings.”
Read much more, including potential study limitations, in the Journal of the American College of Radiology.
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.