Researchers develop program that can spot falsified radiology reports written by AI
Researchers from the University at Buffalo have developed a tool they believe can help spot artificial intelligence-generated radiology reports used for fraudulent insurance claims.
With all of the good AI can bring to the medical industry, particularly within the field of radiology, there will inevitably be drawbacks. There are numerous applications that have been developed to help streamline the process of writing radiology reports, and while the perception of these tools is generally favorable, there are caveats to AI-generated reports.
“With generative AI becoming more capable of producing remarkably convincing radiology reports, there’s a greater risk of fabricated reports being used to falsify medical histories and support fraudulent claims,” lead research investigator Nalini Ratha, PhD, a SUNY Empire Innovation Professor in the Department of Computer Science and Engineering at UB, said in a news release. “Radiology reports have highly specialized structure, vocabulary and stylistic norms, making general-purpose detectors unreliable. Therefore, our goal was to build a detection framework designed specifically for radiology that can distinguish clinician-written medical documentation from synthetic text before it reaches clinical or insurance workflows.”
Ratha, alongside PhD students Arjun Ramesh Kaushik and Tanvi Ranga, trained their tool on a dataset of 14,000 pairs of radiologist-authored and AI-generated chest X-ray reports. Using real radiology reports, the team deployed both advanced large language models and vision-language models to create AI-generated reports. The group mainly used the “findings” section of radiology reports for training, as these sections contain the most descriptive language and are more likely to be exploited.
An authorship-detection framework was also integrated via a BERT–Mamba–based detection model. The purpose of this was to help the tool spot patterns known to be specific to AI.
“AI systems leave subtle stylistic fingerprints such as patterns in phrasing, punctuation and word choice that differ from how radiologists naturally write,” Kaushik said. “By disentangling style from content and treating it as its own measurable feature, our model was able to detect those patterns with exceptional precision.”
The tool performed well when tested; it was able to accurately differentiate human- from AI-written reports, yielding Matthews correlation coefficient scores of 92% and 100% for text-to-text and image-to-text assessments. It also performed well when presented with reports from LLMs it had never been trained on previously, maintaining its performance and consistency.
“What we found is LLMs tend to write in polished, expansive language, while clinicians write in concise, direct terms,” Ranga noted. “Radiologists use straightforward terms like ‘heart’ or ‘lung.’ LLMs often replace them with more elaborate phrases like ‘pulmonary vasculature,’ which became a clear stylistic signal that our model learned to separate.”
The team is continuing to work on fine-tuning the model and is optimistic that it could have applications that extend beyond radiology.
Learn more about their work here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.