Machine learning model predicts which radiotherapy patients are most vulnerable to adverse side effects
Experts have developed a machine learning model they say can help predict which patients are most likely to require some form of acute or urgent care while undergoing a radiation therapy treatment regimen.
Radiotherapy, though effective, often causes adverse side effects. These effects range from mild fatigue, nausea and hair and skin changes to more consequential issues involving blood counts, dehydration, reproductive challenges, gastrointestinal problems and more. It is estimated that up to 20% of patients undergoing radiotherapy will require acute care in the form of emergency visits or hospitalizations at some point during their treatment.
Findings shared this week at the annual meeting of the American Society for Radiation Oncology (ASTRO) detail the utility of a machine learning model that was designed to help predict which patients may be most likely to require acute medical care throughout the course of their treatment. Researchers involved in the model’s development believe its risk classifications could arm providers with the information they need to make proactive decisions that could help patients better tolerate the duration of their treatment.
“We previously reported the results of the System for High-Intensity Evaluation During Radiotherapy (SHIELD-RT), one of the first machine learning (ML)-guided randomized controlled trials in healthcare, where ML was applied to electronic health record (EHR) data to identify patients at high risk for acute care events and direct increased clinical evaluations – reducing acute care by 45% and overall costs by 48%,” Marianna Elia, MS, BS, with the University of California San Francisco, and colleagues noted. “As there remains limited external validation of prospectively tested healthcare ML models, we sought to test the hypothesis that the model would have good external validation performance at multiple institutions with distinct patient populations, clinical practices, and different EHR systems.”
The SHIELD-RT model integrates structured EHR data spanning patient characteristics, cancer treatment, vitals, lab results, medications and prior acute care utilization to determine an individual’s risk of requiring urgent care. Recently the team completed an extensive external evaluation of the model on more than 22,000 radiation therapy courses delivered to nearly 19,000 patients from two sites throughout a 10-year period. This allowed them to compare its risk predictions against outcomes detailed in the patients’ medical records. They evaluated its performance using AUROC, Brier score and calibration plots. Those with an acute risk prediction of 10% or higher were considered high-risk.
SHIELD-RT performed well, achieving an AUROC of 0.756 at Site 1 and 0.770 at Site 2. Using the 10% threshold, the model yielded a sensitivity of 54.6% at Site 1 and 58.0% at Site 2, while specificity was stable at 80% across both groups.
Overall, it classified around 20% of the cases as high-risk. True event rates for the high- and low-risk populations were 13.9% and 2.8% at Site 1, 16.5% and 3.7% at Site 2, “demonstrating good discriminatory power,” the group suggested.
“As the SHIELD-RT study previously demonstrated the model’s ability to direct care and reduce acute care event rates, these early phase external validation results show promise for its generalizable clinical impact,” the authors concluded.
ASTRO’s 67th annual meeting is set to conclude in San Francisco on Wednesday, October 1.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.