A narrow miss for AI trained to find pacemakers on X-rays for MRI safety
A convolutional neural network has achieved 99.67% accuracy at flagging the presence of pacemakers on chest radiographs in patients referred for MRI.
While impressive, the U.K. radiology researchers behind the work state that anything less than 100% is not enough to unseat patient and referrer questionnaires as the gold standard for MRI safety.
The present study, published June 29 in the Journal of Digital Imaging, thus positions computer vision image classification as a highly promising work in progress, especially given that such questionnaires are similarly imperfect.
“Questionnaires are not fail-safe,” the authors of the new study point out, “as referrer responses can be unreliable and patient responses are often not available until the day of the scan” [1].
Corresponding author Mark Daniel Vernon Thurston, MBChB, of University Hospitals Plymouth NHS Trust and colleagues trained, tested and validated their model with around 8,000 chest X-rays about evenly divided between those with visible pacemakers (3,996) and those without (3,977).
Two board-certified radiologists reviewed the images to ensure correct pacemaker classification, and the researchers used the image data to retrain a pre-trained image-classification neural network, according to the study report.
Assessing the model’s final performance on a separate test set of 300 images, the team recorded the 99.67% accuracy score as the model correctly classified 299 of them. Sensitivity here was 100%, specificity 99.3%
The single mischaracterized image contained a feeding tube, which, the authors note, looks much like a pacemaker lead.
Such a false positive rate, although small, “would create additional work for a human operator,” the authors write. “We used a 50:50 split between positive and negative examples, which does not reflect the prevalence of pacemakers in the typical MRI patient population. Given the real world class-split, an anomaly detection model may be worthy of future investigation.”
More:
There is much focus on using artificial intelligence for guiding diagnosis. However, there are many possible applications of computer vision techniques for optimizing workflow and safety. In this study, we have demonstrated the potential for an artificial intelligence model to detect pacemakers on routine chest radiographs. This could be incorporated into current MRI safety processes to improve early identification, before safety questionnaire data is available.”
The study is posted in full for free.
More Coverage of MRI Safety:
Physicians work to spread the word: MRI scans are generally safe for patients with pacemakers
Could this research help prove cutting-edge MRI techniques are safe for patients?
MRIs found safe for patients with implantable cardiac devices
Implantable heart stimulator conditionally cleared for MRI
Worker dies after MRI machine plummets outside local hospital
Gadolinium can be used as substitute for iodine contrast in some interventional imaging procedures
Real-world fast brain MRI in outpatient setting offers ‘substantial’ business benefits
Startup collaborating with Yale on new point-of-care MRI system
Reference:
- Mark Daniel Vernon Thurston, Daniel Kim, Huub Wit: “Neural Network Detection of Pacemakers for MRI Safety.” Journal of Digital Imaging, June 29, 2022. DOI: https://doi.org/10.1007/s10278-022-00663-2
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.