A 3-step buyer’s guide for evaluating radiology AI solutions
Imaging experts are offering a three-step buyer’s guide for evaluating radiology artificial intelligence solutions.
The development and commercial release of these AI models for radiology has “exploded” in recent years. While vendors are economically incentivized to sell as many products as possible, the onus is on imaging groups to select software with optimal diagnostic performance and workflow integration, experts note.
“As a result, vendors may present a strong narrative in support of the performance of their algorithm,” Grayson L. Baird, PhD, an associate professor and vice chair of radiology research at Brown University, and co-authors detailed in the Journal of Imaging Informatics in Medicine. “It is up to radiologists and their practices to critically evaluate such advertised performance metrics, including available options for effective workflow integration, which matters.”
To date, no one has published an independent, comparative performance analysis of AI detection models in imaging, similar to Consumer Reports, the authors believe. Baird and colleagues aimed to address this with their new open-access analysis, published last month. Researchers helped illustrate the use of their buyer’s guide by sharing a case study, previously published by European Radiology in 2024.
The example describes the deployment of a large vessel occlusion-detection AI model, used in an emergency radiology setting. At the time, the vendor had reported a 97% sensitivity rate for correctly identifying such stroke cases and nearly 96% specificity for pinpointing those without this clinical concern. It also came with a high positive predictive value (or the probability a person who gets positive results actually has the disease) at 95% and negative predictive value at 98%.
“At first glance, this seems to describe a highly performing AI system: the diagnostic accuracy is high, with sensitivity and specificity exceeding 95%, and the PPV and NPV suggesting that only 5% of flags will be false alarms, and only 2% of what the algorithm does not flag will be an LVO,” the authors noted.
However, the AI algorithm was only deployed for about three weeks before the practice had to remove it, due to perceptions that it was inaccurate. This included a positive predictive value of 33%, meaning that 67% of flagged cases were false alarms, much greater than the 5% rate the vendor had claimed. Further analysis showed that the discrepancy was mainly due to highly different disease prevalence in the patient cohort at the deployment site compared to the model’s test setting. A second reason for the discrepancy was the slightly different values of sensitivity and specificity between the patient cohort compared to the model’s test performance.
“The lesson to draw from Scaringi and colleagues is that relying on vendor-reported performance metrics…may not be sufficient to anticipate performance and behavior of the model in local practice…,” the authors noted.
Baird and co-authors detailed three steps to help others avoid these pitfalls in the future. They are, in brief:
1. Evaluating if and how a given AI model will perform on the local population: Namely, what error rates clinicians can anticipate from the AI system. Baird and colleagues offer a list of sample questions practices can ask vendors. Possible queries could include: What population was the algorithm trained, tested and validated on? And what were the resulting accuracy metrics? The goal is to establish to what degree a model’s training and testing data are comparable to the clinical population and setting in which the model will be used.
2. Consider how often these error rates will impact local clinical decision-making: This includes subsequent medical, legal, financial, psychological, workflow and reputational downstream effects. Baird and c0-authors offer a matrix to help with these considerations, dubbed the Risk Assessment of Decision-making and Downstream Effects, or RADDE. In the example of the stroke algorithm, the matrix would pose: “If AI correctly predicted an LVO, then the patient receives necessary treatment. Therefore, the patient’s chance of surviving improves (medical), the patient is satisfied with the radiologist’s performance (reputational), and the radiologist has satisfied their legal duty of care (legal).”
3. Weigh the frequency of errors against the cost of those mistakes. “Only after these questions are addressed can a practice make informed decisions about whether implementation of a given AI system is justified within their practice setting or not,” the authors noted. By completing the RADDE matrix, practices can determine (1) the probability of errors occurring, (2) how those mistakes might impact decision-making, and (3) possible downstream effects.
“Of course, practices may have multiple AI systems to choose from, and results from our proposed three-step evaluation may provide significant utility in driving towards a data-informed comparison/selection method,” they concluded.
You can read much more in the official journal of the Society for Imaging Informatics in Medicine (SIIM). Other co-authors of the piece included the Mayo Clinic’s Christoph Wald, MD, PhD, MBA, who currently chairs the American College of Radiology board, and Michael A. Bruno, M.D, M.S., vice chair for quality and chief of emergency radiology at Penn State.
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.