New research is questioning previous studies that have attempted to assess AI’s impact on a collective group of radiologists, rather than on an individual basis.
Harvard Medical School scientists recently sought to examine the effects of AI assistance on 140 radiologists handling 15 different chest X-ray diagnostic tasks. They also aimed to identify predictors of these effects, such as how age and expertise change each physician’s response to the technology.
“Surprisingly,” the authors wrote Tuesday in Nature Medicine [1], experience-based factors such as a radiologists’ subspecialty, failed to reliably predict artificial intelligence’s impact.
“We find that different radiologists, indeed, react differently to AI assistance—some are helped while others are hurt by it,” co-senior author Pranav Rajpurkar, PhD, an assistant professor of biomedical informatics at Harvard Medical School, said in a March 19 news announcement. “What this means is that we should not look at radiologists as a uniform population and consider just the ‘average’ effect of AI on their performance. To maximize benefits and minimize harm, we need to personalize assistive AI systems.”
The study involved a total of 324 patient cases with 15 different abnormal conditions captured via chest X-ray. Rajpurkar and colleagues used advanced computational methods to help discern how AI impacted radiologists’ abilities, and the magnitude of performance change with and without assistance. But the degree of change was inconsistent and varied among physicians.
Researchers did not attempt to explain why AI tools affect human clinicians’ performance differently. They urged researchers to work with radiologists who use these tools to better define the factors that influence human AI-interaction.
“Clinicians have different levels of expertise, experience, and decision-making styles, so ensuring that AI reflects this diversity is critical for targeted implementation,” Feiyang Yu, who conducted the work while at the Rajpurkar lab, said in the Harvard announcement. “Individual factors and variation would be key in ensuring that AI advances rather than interferes with performance and, ultimately, with diagnosis,” Yu added.
Read more about the study’s results at the link below.
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.