6 steps for seamlessly integrating an artificial intelligence solution into daily clinical practice
Imaging stakeholders have written plenty about the promise of artificial intelligence but not much on how to integrate AI solutions into daily clinical practice. Experts with University Hospitals are attempting to fill the void, detailing their own implementation experience on Tuesday in the Journal of the American College of Radiology.
The Cleveland-based institution recently acquired an AI-powered X-ray scanner with a U.S. Food and Drug Administration-cleared algorithm for detecting collapsed lungs. After going live in July 2020, University Hospitals is now sharing its success story for those just beginning to test the waters.
“Development of an AI algorithm capable of identifying clinical findings on diagnostic imaging represents an impressive feat, but by itself offers limited value to healthcare professionals,” Jonathan Pierce, MD, a radiology resident, and colleagues wrote. “The true value of a technology of this kind relates to how quickly it can be integrated into clinical practice and leveraged to improve key metrics such as turnaround time.”
Pierce and colleagues finished installing the AI algorithm around December 2019. They recommend six steps to take when integrating the tool into radiologists’ regular work:
1. Validation. Despite the FDA blessing, University Hospitals took time internally confirming the system’s effectiveness. Scientists routed the first 150 chest X-rays with AI findings to the organization’s separate research PACS. Two tenured thoracic radiologists independently reviewed the results, which confirmed almost 100% sensitivity in detecting moderate- to large-sized pneumothorax findings and 80% for the smaller variety.
2. Integration. Around February 2020, University Hospitals began assimilating AI into the regular PACS workflow. This involved initiating conversations between the organization’s informatics team, device vendor and the thoracic radiology department. Afterward, the team identified DICOM tags tied to the new algorithm and noted whether they were compatible with the hospital PACS. Informatics team members then mapped them into the system and worklist.
“Identifying DICOM headers containing actionable information is key to the integration of a new AI tool into the clinical PACS environment,” Pierce et al. noted. “Open collaboration with PACS vendors to determine how best to utilize this DICOM data is a crucial step in this process.”
3. Experimentation. University Hospitals additionally developed another process for improving after-hours turnaround times for routine studies that were confirmed for pneumothorax. Work up to this point led to an “end-to-end seamlessly integrated system,” with positive findings sent directly to the PACS and displayed as top priorities in real time as part of on-call docs’ worklists. Researchers spent several weeks testing the AI tool in the research PACS between April and May 2020 before approval for clinical use.
4. Education. The hospital system launched a department-wide educational campaign around June, hoping to increase awareness and acceptance. This step called for conversing with administrators, techs, managers, residents and attending radiologists while also creating a comprehensive slide deck to share with residents at the start of the new academic year.
5. Initiation. University Hospitals went live with the AI solution across all physician users in August 2020. At the time of the study’s publication, the new mobile X-ray scanner had logged nearly 31,000 exams, with about 3,000 labeled as suspicious, and more than 900 of those ordered as routine/noncritical.
6. Collection. Since then, the organization has continued to track and store AI cases within its PACS. Project participants are also performing statistical analyses and tracking other secondary outcomes such as turnaround times and user engagement, with plans to potentially publish the findings in the future.
“Feedback across the department has been overwhelmingly positive,” Pierce and co-authors noted. “There is a consensus among radiologists and trainees that the tool is both accurate and helpful during the regular workday as well as on-call shifts with almost no perceived disruption to their normal workflow.”
Read the rest of their work, including sample use cases and potential limitations, in JACR here.
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.