ChatGPT a ‘substantial first step’ toward AI-drafted radiologist reports
Using a RSNA template and related classification system, ChatGPT was able to generate highly rated reports for distal radial fractures, according to a new Current Problems in Diagnostic Radiology study.
The findings appear to be a positive development given concerns about ongoing radiologist shortages amid an aging population, with the study’s authors noting that greater productivity is one way to achieve higher capacity.
“Text drafting tools might well support work of radiologists in the future. They would allow a radiologist to focus time on the observation of image details and patient pathology. ChatGPT can be considered a substantial step forward towards that aim,” wrote first author Wolfram A. Bosbach, MD, PhD, of the University Hospital of Bern in Switzerland.
To gain a better understanding of ChatGPT’s current capabilities, the authors tasked the artificial intelligence software with drafting appropriate radiology reports for nine cases of distal radius fractures (which were chosen because of the relative frequency and simplicity of the task), for five iterations each. Before doing so, they fine-tuned the model using human conversations and a reward model for reinforcement learning by ranking model outputs.
Then, four musculoskeletal radiologists used a Likert scale scorecard, ranging from -2 to 2, to objectively evaluate the responses in multiple categories: correctness of exam information and fracture findings, suitability of impression, grammar and style format.
Based on the radiologists’ scores, ChatGPT fared worst in the “impressions suitable” category at 1.08—still indicating agreement—and best in the “style format correctness” category at 2, indicating strong agreement.
“This reflects the reviewers’ view on the ability of ChatGPT to generate radiology report drafts on a highly competent level,” the authors concluded.
Noting that other forward-looking professionals in the field have focused on AI’s ability to read and analyze images, the authors highlight the fact that their findings offer an alternate yet also impactful way to take advantage of rapidly developing technologies.
View the full study here.