Artificial intelligence shows promise in mitigating radiologist bias

Artificial intelligence may serve as a useful tool for mitigating radiologist bias when interpreting images, according to a new study published in Scientific Reports [1].  

CT has value in helping physicians assess patients suffering from COVID-19 (though some have criticized this practice). Quantification of pneumonia, in particular, may help to predict treatment course and outcomes, but it is “heavily reliant” on a radiologists’ subjective perceptions, Romanian researchers wrote March 25.

A survey of 40 radiologists—along with a retrospective analysis of CT data from 109 patients treated at two hospitals—showed that members of the specialty often overestimate lung involvement. To fix this, scientists conducted a randomized control trial using AI-based clinical decision support.

This was found to reduce any absolute overestimation error from 9.5% ± 6.6 down to 1% ± 5.2, the investigation found.

“These results indicate a human perception bias in radiology that has clinically meaningful effects on the quantitative analysis of COVID-19 on CT,” Bogdan A. Bercean, with Politehnica University of Timișoara in Romania, and colleagues advised. “The objectivity of AI was shown to be a valuable complement in mitigating the radiologist’s subjectivity, reducing the overestimation tenfold.”

Bercean et al. made use of a commercial medical device from Rayscape, which is based in Romania and also co-founded by the study author. The AI analysis offers radiologists an automatic suggestion of total lung involvement percentage, along with colored segmentation overlays. These are meant to help physicians visually check the validify of the suggested percentage, while also allowing for “easier mental adjustments, where needed.” Researchers randomly blinded radiologists by turning the AI tool off 50% of the time to assess its effectiveness.

They found that the AI assistance reduced the average overestimation difference, with further testing confirming the finding’s statistical significance. Bercean and co-authors attributed success of the AI in part to widespread adoption and integration into the hospital’s PACS. AI clinical decision support was particularly popular among younger radiologists, who also demonstrated the greatest bias susceptibility.

“Our study demonstrated that quantification of the involvement of the lungs in COVID-19 on CT scans is a perception-sensitive process prone to cognitive overestimation bias,” the author’s concluded. “This is of key importance given the wide use of the marker, although it was shown to be controllable with an AI decision support system. This reinforces the benefits of human-AI synergy and strengthens the need to further study the adaptability of radiology to rapid technological and methodological changes.”