Fewer than 30% of FDA-cleared AI devices share key safety, adverse event info prior to approval
A new analysis is prompting questions regarding how rigorously many of the AI-enabled tools approved by the U.S. Food and Drug Administration are tested prior to their clearance.
According to the data, hundreds of algorithms were approved by the FDA without first providing data relevant to testing methods, safety and efficacy. This, authors of the analysis suggest, highlights the need for more stringent testing and a dedicated regulatory pathway to approval for AI devices.
“Unlike standards for pharmaceutical agents, there do not exist predefined standards for efficacy, safety and risk reporting of AI/[machine learning] devices prior to or after clearance or certification,” Ravi B. Parikh, MD, MPP, with the Department of Hematology and Medical Oncology at Winship Cancer Institute in Atlanta, and colleagues noted. “In particular, the FDA and EU have recognized safety and risk assessment as important regulatory priorities for AI/ML.”
For their study, the group analyzed data from FDA decision summaries and approval databases, the FDA Manufacturer and User Facility Device Experience Database, and the FDA Medical Device Recalls Database for all AI/ML devices cleared by the FDA from September 1995 to July 2023. They compared study design, data availability, efficacy, safety, bias assessments, adverse events, device recalls and risk classification for each device. A total of 691 devices were included, 531 of which were designed specifically for radiology.
Nearly half of the device summaries (46.7%) did not disclose their study design, while over half (53.3%) did not provide information regarding training sample size, and/or demographic information. Just 1.6% included data from randomized clinical trials, but slightly more (7.7%) shared information obtained during prospective analyses.
The team observed very few premarket summaries with data from peer-reviewed journals, and nearly 75% failed to provide data on statistical or clinical performance, such as sensitivity, specificity and patient outcomes. Less than 30% reported safety assessments and even fewer—less than 7%—provided information related to adverse health risks.
Overall, there were 489 adverse events reported involving 36 (5.2%) devices. Those resulted in 458 malfunctions, 30 injuries, 1 death and 40 device recalls. Most of these were attributed to software issues.
“Reporting of safety results and risks to health, as well as adherence to international standards, did not improve after 2021,” the authors noted. “These findings suggest that premarket safety information for AI/ML devices may be insufficient and more robust postmarket surveillance systems are necessary to capture health-related adverse events and safety concerns relevant to AI/ML devices. In particular, known degradation in performance of AI/ML algorithms complicates traditional surveillance methods.”
The group added that their findings highlight issues with a lack of diversity in training datasets. Though this is a known problem, the authors cautioned it has the potential to further widen disparities in healthcare. The group suggested a streamlined regulatory pathway to approval could include a stipulation requiring developers to meet certain criteria regarding data they use to train algorithms that could help address this issue.
“To date, the FDA does not have a dedicated regulatory pathway for AI/ML devices, the team concluded. “Given the unique considerations of these devices, implementation of a dedicated regulatory system for AI/ML devices, including more rigorous standards for study design and transparency, should be considered.”
Read more about the findings here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.