New study offers insight into the type of AI radiologists prefer
Radiologists are more likely to trust domain-specific artificial intelligence models when it comes to report generation, new data suggest.
Researchers with the Moffitt Cancer Center recently conducted an analysis to get a better idea of what type of AI radiologists prefer to help generate impressions—perhaps the most critical section of a radiology report. The team compared radiologists’ opinions of reports generated by a general-purpose large language model to that of a model fine-tuned on institutional and clinical data from radiologists and oncologists, comparing the outputs to reports authored by human readers.
Of the 200 CT reports produced, readers showed preference for the domain-specific model’s outputs. In general, the group indicated that the model produced impressions that aligned with the completeness, accuracy and conciseness of reports compiled by radiologists. What’s more, the team found that the domain-specific model also produced reports in significantly less time compared to the LLM.
About 70% of all FDA-cleared AI applications have been developed for radiology-related purposes, making radiologists’ opinions on their use a critical consideration for organizations looking to invest in the technology. These findings could be used to help guide decisions, experts involved in the analysis suggested.
"Impressions are the most critical part of the radiology report," study co-author Andrew Del Gaizo, chief medical information officer at Rad AI, said in a statement. "This study demonstrated that, in addition to accuracy, customization matters to radiologists in order for them to feel confident that AI clearly and effectively communicates findings from impressions—and does so in a way that aligns with their clinical workflows."
"We saw meaningful variability in how radiologists and oncologists evaluated the same outputs, which has important implications for how health systems think about deploying AI," added Trevor Rose, MD, MPH, a diagnostic radiologist at the Moffitt Cancer Center in Tampa, Florida. "The study shows that impression quality can be inherently subjective even with clinicians in the same specialty. Rather than optimizing for a single 'best' output, organizations should be prioritizing tools that can adapt to different users, workflows and clinical preferences."
Read more about the findings in npj Digital Medicine.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.