AI can help humans inspect chest X-rays for COVID-19, but the technology is unfit to serve as a standalone diagnostic tool for that purpose, according to a prospective observational study conducted at the University of Minnesota and published online June 1 in Radiology: Artificial Intelligence.
In the study, deep learning models performed satisfactorily at identifying patients with severe COVID symptoms but could not recognize the virus in COVID-positive patients who had mild respiratory symptoms and minimal findings on chest X-rays [1].
More tellingly still, the algorithms had significantly lower overall accuracy than both of two board-certified radiologists reading similarly sourced radiographs.
Ju Sun, PhD, and colleagues trained and tested AI models with X-ray image datasets from their own institution as well as publicly available COVID-positive and COVID-negative databases. The total image count topped 91,000.
To assess the AI in real-time clinical use as a clinical decision support mechanism, the team used diagnoses confirmed with polymerase chain reaction (PCR) lab tests.
Sun and co-authors report their system made the correct call at a rate of 63.5%.
This compared with 67.8% for one experienced radiologist and 68.6% for another.
Also of note:
- The AI’s real-time performance rates remained unchanged over 19 weeks of implementation.
- Model sensitivity was higher in men than women, while model specificity was higher in women.
- Sensitivity was higher for Asian and Black participants compared with white participants.
The authors conclude:
AI-based diagnostic tools may serve as an adjunct, but not replacement, for clinical decision making of COVID-19 diagnosis, which largely hinges on exposure history, signs and symptoms. While AI-based tools have not yet reached full diagnostic potential in COVID-19, they may still offer valuable information to clinicians taken into consideration along with clinical signs and symptoms.”
The present study builds on research Sun conducted in 2020 with senior authors Erich Kummerfeld, PhD, Christopher Tignanelli, MD, and colleagues.
The new study is available in full for free (PDF).
More Coverage of COVID-19 Imaging:
Imaging shows COVID vaccines effective at warding off pulmonary embolism
VIDEO: Should women wait to get mammograms after COVID vaccination?
FDA clears software that analyzes lung function in COVID patients from single x-ray
Lung ultrasound saves providers dozens of hours compared to CT for severe COVID management
COVID-19 less severe among fully vaccinated patients, CT imaging study confirms
Reference:
- Ju Sun, Erich Kummerfeld, Christopher Tignanelli, et al., “Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study.” Radiology: Artificial Intelligence, June 1, 2022. DOI: https://doi.org/10.1148/ryai.210217
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.