Factors making healthcare AI devices more likely to face recall

Artificial intelligence-powered medical devices are more likely to be recalled in the U.S. based on a few key factors, according to new research. 

In particular, products with information missing from supporting clinical studies have an increased likelihood of recall. Deviations from the device’s intended use most often trigger FDA recalls, “highlighting unique safety challenges” in AI, experts write June 11 in JAMA Network Open. 

The findings are based on data from 903 healthcare AI devices cleared by the U.S. Food and Drug Administration between 1995 and 2024. Overall, about 5% (or 43) were recalled during this time, at a median interval of 458 days. Of the total, 692 devices were in radiology, with 4.3% (or 30 products) recalled during the study period.

“These findings highlight the importance of robust clinical validation and strengthened postmarket oversight for AI-enabled devices,” study co-author Daniel Windecker, MD, with the Department of Diagnostic and Interventional Neuroradiology at the University of Bern, Switzerland, and co-authors concluded. 

The study assessed AI devices from authorization to recall/administrative censoring through Aug. 31, 2024, using adverse reports from FDA manufacturer databases. Devices with missing clinical study information had a higher risk of recall than those with published analyses, the study found. Products flagged by market surveillance tools (such as Europe’s CORE-MD) also had higher risk of recall, the study found. 

Problems related to temperature occurred in about 1 of 9 recalls, while labeled issues happened in about 2 of 9. Both issues were tied to a lower risk of recall, while incorrect utilization of AI devices has a higher hazard of leading to a device being pulled from the market. Ineligibility for third-party review, radiology-related devices, and missing info about clinical performance were likewise associated with higher hazard of recall. In fact, radiology devices are about 52% more likely to face a recall than other areas of medicine, the study found. This is likely influenced by the specialty accounting for such a large percent of the total devices assessed. 

Windecker and colleagues believe theirs is the first study to systematically explore the association between AI device difficulties and the risk of recall. They warn against using AI in situations that differ from the originally intended clinical setting, or without robust supporting evidence. 

“Our findings suggest that the current premarket evaluation framework may not fully capture real-world clinical variability,” the authors noted. “Although this limitation is not unique to AI-enabled devices, their data-driven and context-dependent nature may amplify the consequences of inadequate validation, underscoring the need for more rigorous and transparent clinical performance evaluation.”