AI spots dozens of missed incidental pulmonary embolism diagnoses at one hospital
Artificial intelligence spotted dozens of missed incidental pulmonary embolism diagnoses over the course of a year at one medical center, according to an analysis published Monday in JACR.
The investigation was retrospective, utilizing nearly 12,000 CT scans gathered at Duke University Medical Center in 2014. But scientists believe their proprietary algorithm could be prepped for pinpointing these near-misses in real-time clinical work.
“Radiologists must play a role in the design of future care algorithms to manage the balance between sensitivity and specificity in these scenarios,” Benjamin Wildman-Tobriner, MD, with Duke’s Department of Radiology in Durham, North Carolina, and co-authors wrote Feb. 15. “Nonetheless, our data suggest that prospective implementation may identify clinically significant missed [incidental pulmonary embolism] and should be considered.”
Previous studies have indicated that such unsuspected PE prevalence is about 2.6%, with radiologists likely encountering this indication often in a busy clinical practice. Yet, another analysis estimated that more than half of incidental cases went unreported during the initial interpretation, which is critical as most acute PE requires treatment, experts noted.
To address this concern, scientists created an AI algorithm trained on more than 30,000 computed tomography scans gathered from 2,000-plus U.S. hospital sites. Wildman-Tobriner and colleagues took the tool for a test drive, using it to analyze 11,913 CT exams of the chest, abdomen and pelvis for indications other than pulmonary embolism. Natural language processing and a trained abdominal rad helped to further refine the results.
All told, investigators discovered 49 (0.41%) missed incidental pulmonary embolism diagnoses in a single year. That compares to 79 (0.66%) spotted during the initial clinical interpretation, “demonstrating that incidental emboli may be missed nearly as often as they are identified,” the authors said.
“It is important to recognize that algorithms are not perfect, and that most (85.8% [289 of 337]) of the studies identified by our algorithm as possibly containing IPEs were ultimately found to be negative,” Wildman-Tobriner cautioned. “Although a ‘hit rate’ of about 15% may be considered reasonable to catch missed IPE, alarm fatigue is an important consideration, particularly if multiple AI algorithms are implemented, compounding the number of [false positive] cases to be assessed.”
Read more about the investigation in the Journal of the American College of Radiology here.
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.