Radiology residents miss more findings after night shifts, even with AI assistance
Night-shift fatigue can have a significant impact on radiology residents’ diagnostic accuracy, and new data suggest that AI assistance has little benefit in improving readers’ performance.
The findings, published in European Radiology, also revealed that readers’ reporting times did not increase, despite their reported energy depletion. Experts involved in the analysis say the results raise questions about how overnight workloads affect diagnostic performance and whether AI tools can truly help mitigate fatigue-related risks in radiology.
“There is a broad consensus that strategies to mitigate radiologist fatigue are urgently needed. One potential avenue is artificial intelligence. By automating repetitive tasks and supporting image interpretation, AI has demonstrated potential to improve efficiency, enhance accuracy, and alleviate workload,” Robert Malte Siepmann, with the department of diagnostic and interventional radiology at Aachen University Hospital in Germany, and colleagues noted. “These features suggest that AI tools could help offset fatigue-related declines in performance by allowing radiologists to conserve cognitive resources and concentrate on complex cases. However, evidence supporting this hypothesis is limited.”
Researchers evaluated 10 radiology residents with an average of 35 months of experience. Participants interpreted three sets of 33 chest radiographs before and after completing an on-site first-call night shift. Pre-shift readings were conducted in morning and afternoon sessions without AI assistance, while post-shift readings were performed with and without AI.
Eye-tracking technology measured fixation duration and saccade count, while researchers used changes in the relationship between saccade velocity and amplitude to classify fatigue. Diagnostic accuracy was assessed by comparing residents’ interpretations with an expert majority vote.
Six of the 10 residents met the study’s definition of fatigue. Among these readers, diagnostic agreement declined from a Cohen’s kappa of 0.71 before the night shift to 0.61 afterward, a statistically significant difference. Their eye-tracking results also showed shorter fixation durations and fewer saccades following the shift.
The four residents classified as non-fatigued also experienced a decline in diagnostic accuracy, with kappa values falling from 0.72 to 0.65. This reduction was considered statistically significant as well.
Despite the changes in accuracy, reporting times remained stable before and after the night shift. This suggests that residents continued to interpret images at a similar pace even as diagnostic performance declined.
AI assistance did not significantly improve diagnostic accuracy in either group. The technology did, however, affect reporting efficiency for some readers; among non-fatigued residents, AI assistance reduced average reporting time by 9.9 seconds per radiograph. Fatigued residents also read images faster with AI, by around 6.3 seconds per radiograph.
“Because workflow speed was upheld, the reduction in accuracy underscores a critical safety issue, particularly in high-throughput environments," the group cautioned. "In clinical practice, individual efficiency is often driven by worklist-related pressures, while accuracy is not continuously monitored.”
The authors cautioned that further research involving larger reader groups and different imaging tasks will be needed to definitively determine how fatigue affects diagnostic errors and whether other AI systems or workflow interventions can improve safety during overnight coverage.
Read more here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.