Radiologists see significant reduction in reading times with AI-generated reports
Radiologists are seeing significant reductions in radiograph reading times with the use of artificial intelligence-generated reports, according to new research published Saturday.
Acceptability—defined as the proportion of AI reports approved by humans without requiring any revisions—also increased considerably, “suggesting growing confidence in AI-generated reports over time,”
experts detailed in JACR.
“These findings suggest that future studies should explore ways to enhance the accuracy of … preliminary AI-generated reports and examine how increasing confidence in AI impacts radiologists' decision-making and diagnostic performance,” Eun Kyoung Hong, MD, PhD, with the Department of Radiology at Mass General Brigham, Boston, and co-authors wrote Sept. 20.
For the study, researchers used a publicly available set of 756 chest X-rays interpreted by five radiologists. Those involved used a generative AI model for report generation—Kakao Brain’s KARA-CXR, which is strictly for research purposes—to create findings-only reports, without additional impressions or recommendations. Five radiologists each generated preliminary AI reports with the software, with two additional rads assessing the results.
Radiologists reading times decreased about 25%, from 25.8 seconds in initial tests down to 19.3 by the end of the experiment. Meanwhile, the “acceptability” of AI-generated reports increased from 54.6% to 60.2%. The artificial intelligence model proved more successful with chest X-rays that showed no issues (68.9%) compared to those with abnormalities (52.6%). Median agreement and quality scores stayed stable for healthy chest X-rays but varied significantly for abnormal scans, the study found.
Researchers noted that their findings underline the need for human oversight when interpreting complex cases.
“While radiologists grew more comfortable approving AI-generated reports without modifications, this did not necessarily impact the diagnostic accuracy of the final reports,” the authors noted. “Moreover, while reductions in interpretation time and increases in acceptability were statistically significant, it may not yet reflect clinically meaningful improvements in workflow or diagnostic performance. Future studies, ideally conducted prospectively in real-world clinical settings, are needed to determine whether sustained exposure to AI-generated reports alters radiologists’ acceptance thresholds and how such changes may affect clinical decision-making and patient outcomes.”
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.