How AI can help extract follow-up recommendations from radiology reports
Artificial intelligence (AI) can help providers identify follow-up recommendations in radiology reports, according to new research published by the Journal of the American College of Radiology.
“Studying follow-up recommendations is key to understanding and monitoring their prevalence, identifying unwarranted variations in their use, tracking their impact, and developing initiatives to improve their use,” wrote author Emmanuel Carrodeguas, MD, of Harvard Medical School in Boston, and colleagues. “Nonetheless, it remains challenging to extract recommendations from radiology reports, given their free-text nature and the varied language radiologists use to make follow-up recommendations.”
The authors selected 1,000 radiology reports from an academic medical center that generates more than 500,000 reports each year, reviewing and annotating each one for follow-up recommendations. Traditional machine learning (TML) algorithms and a deep learning (DL) algorithm were then developed and trained on an additional 850 reports.
When each algorithm, as well as an existing natural language processing (NLP) system, were put to the test, the TML algorithms scored F1 scores of 0.75 (random forest), 0.83 (logistic regression) and 0.85 (support vector machine). The DL algorithm scored a 0.71, the same as the NLP system. The performance of both the TML and the DL algorithms “appeared to plateau” after 500-700 samples, the authors added.
“Our experiments indicate the feasibility of training machine learning algorithms on textual data,” the authors wrote. “A challenge of machine learning is obtaining adequate samples for training and validation. Although large data sets exist in electronic health records, most radiology reports are free text and would require manual annotation before training.”
This research also “opens the possibility” of using follow-up recommendations automatically extracted from reports to ensure there are patient receive the care they need as quickly as possible, the authors added.
“For reports containing further recommendations, a notification system may be developed that can monitor and inform providers so that necessary diagnostic follow-up is not delayed,” they wrote.
Michael has more than 18 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.