AI inspects endotracheal tube placement on chest X-rays
Radiology researchers have demonstrated the reliability of an AI system they developed to automatically check placement of endotracheal tubes on chest X-rays.
Testing their system against prespecified goals, the team found the algorithm’s performance justifies advancing it toward clinical settings.
Lead author Matthew Brown, PhD, senior author Dieter Enzmann, MD, and colleagues at David Geffen School of Medicine at UCLA describe the work in a study published online May 27 in Academic Radiology [1].
Brown and co-authors divided a retrospective dataset of 2,000 chest x-rays from ICU patients into 1,488 for training and 512 for testing.
From these data they designed semantically embedded deep neural networks to automatically identify endotracheal tubes, tracheas and carinas (part of the lower trachea) on chest radiographs.
To check placement of tube tips, the team computed a “safe zone” representing the area in which the tube tip should rest when it’s correctly placed. This zone lies inside the trachea, between 3 and 7 centimeters above the carina.
Noting that an endotracheal tube was present in 385 of the 512 test cases, the authors report their AI achieved 95% accuracy at identifying tubes and 86% accuracy at verifying correct tip placement.
Assessing overlays showing the tube’s path and distance from the tip to the carina, the algorithm correctly did not generate an overlay in 221 of 227 chest x-rays when no tube was present. This represented an overall overly accuracy of 89% (454 of 512).
“With ongoing system refinement,” the authors comment, “this overlay accuracy metric should eventually be above 90%, but 89% was considered sufficient at this stage.”
The AI also presented one of three messages—“Found” when the tip was in the safe zone, “Position Alert” when the tip was misplaced or the safe zone eluded the AI, or “Not Found.”
The alert messages had a positive predictive value (PPV) of 83% for all test cases and of 42% for intubated subgroup.
The prespecified PPV target was lower than the NPV, the authors note, “because it is crucial to alert on misplaced endotracheal tubes and we are willing to accept false positives.”
“While false positives can be inefficient and over time cause alert fatigue, they do not pose a direct risk to patient management,” the authors add. “We will ultimately strive for a low false positive rate, but this target was considered sufficient to permit moving into the initial clinical evaluation where physician acceptance can be further studied.”
Meanwhile negative predictive values were 98% for the “all” group and 99% for the intubated subgroup.
Brown and colleagues set their prespecified NPV threshold above 95% “because the biggest safety concern is a misplaced endotracheal tube, i.e., there is a very low tolerance for false negatives. … [T]he NPV will need to be maintained very close to 100% in larger studies before being cleared for clinical use.”
Brown, Enzmann and co-authors conclude:
We believe it is important for radiology AI evaluations to have prespecified targets for clinically relevant metrics if we are to advance such technology into clinical practice. The intended use in our next phase of clinical evaluation is for healthcare quality improvement providing endotracheal tube detection assist overlays and position check alerts. The system outputs will be made available to both ICU physicians at the point of care and reporting radiologists and the impact in each setting will be evaluated. The physician remains responsible for the final diagnosis and patient management decisions. Our conclusion for this phase of our development is that the AI decision support performance is sufficient to move into clinical implementation and further evaluation.”
More Coverage of Chest Imaging:
Predictive model helps reduce need for chest X-rays to diagnose pneumonia in kids
Radiologists are capable of differentiating COVID-19 from other lookalikes on chest CT
Lung damage appears to persist one year after COVID-19 pneumonia, new CT study reveals
Imaging among the costliest, most prevalent low-value services delivered at children’s hospitals
Reference:
- Matthew Brown, Koon-Pong Wong, Liza Shrestha, Muhammad Wahi-Anwar, Morgan Daly, George Foster, Fereidoun Abtin, Kathleen Ruchalski, Jonathan Goldin, Dieter Enzmann: “Automated Endotracheal Tube Placement Check Using Semantically Embedded Deep Neural Networks.” Academic Radiology, May 27, 2022. DOI: https://doi.org/10.1016/j.acra.2022.04.022
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.