AI quality assurance models saving lives and millions in avoided med-mal
Unrecognized imaging findings are an unfortunate, but undeniable, part of radiology. New advancements in artificial intelligence (AI) and machine learning offer a critical safety net that is improving care and saving lives — as well as avoiding millions of dollars in potential medical malpractice costs.
Our radiology practice of six million annual exams is using AI to identify potentially life-threatening conditions that were not identified by the reading radiologist — and sending them straight to our quality assurance (QA) committee. Each month we’re catching on average four SMA occlusions, two aortic dissections, 40 pulmonary embolisms (PE), and more.
To quote the landmark report, To Err is Human. We know that we make mistakes — every radiologist does. If you find yourself thinking, “nobody in our practice misses aortic dissections or PEs,” I would urge you to pause and consider — were the mistakes not made? Or did they just go unnoticed?
The question is, how do we improve?
At this year’s annual meeting of the Society for Imaging Informatics in Medicine (SIIM), I presented how two vRad AI models and QA workflows are helping to avoid adverse patient outcomes and likely accounting for $2 million/year in potential med-mal savings. I’ll share that research and more so you can see which studies should get special attention at your practice — and the potential cost of error.
But first, some context.
AI as a way to provide a powerful safety net in radiology
AI is a phenomenal technology for radiology. As vRad’s Chief Medical Officer, I work very closely with our team of AI development engineers. We were early adopters of AI and have been using it to prioritize imaging studies for years with outstanding results, ensuring the most urgent cases are invisibly pushed to the top of our radiologists’ worklists.
Our latest frontier for AI adds another layer to our already very robust Patient Safety Organization (PSO). When one of our AI algorithms picks up on something in an image that hasn’t been identified on the radiologist’s written report, the study is auto-routed to our quality assurance committee so members can quickly review, and if needed, addend the report and notify the care team. The committee also provides feedback to the reading radiologist in the same way as any other detected missed finding is communicated.
These models are focused on six potentially life-threatening pathologies and nearly all of the QA notifications are taking place within a day of the original report — in most cases within just a matter of hours. Our goal is to reduce the average notification time to under two hours and offer these vRad-developed models to our clients for use on their own, locally-read studies. This prompt feedback from the quality committee to the interpreting radiologist is a great addition to the PSO workflow.
Our AI-enabled QA process means care providers and radiologists now have a safety net for getting accurate clinical information quickly, as opposed to days or weeks later — or worse, when a lawsuit has been filed in response to a negative patient outcome.
Most likely to go unseen, most likely to progress to med-mal indemnity
Epidural fluid collections
Splenic laceration
Superior mesenteric artery occlusion
Intracranial hemorrhage
Pulmonary embolism
These are the findings that are the most likely to go unseen, can be devastating for patients, are most likely to go to trial, and will be the costliest if they do. I know this because I analyzed in great detail 220 claims against vRad radiologists between 2017-2020. It is this med-mal data, combined with our massive QA database dating back nearly 20 years that guided our team to build the AI models for these six specific pathologies. We are looking for potential findings that would be acutely life-threatening if missed — and then using AI to help us prevent that from happening.
Life-saving results from AI detection of PE and intracranial hemorrhage
Let’s look at the dramatic impact these models have on patient outcomes.
As you can see from the table below, pulmonary embolism and intracranial hemorrhage (ICH) are the highest-volume pathologies flagged by our models for QA review — which is understandable given that we read 6 million studies annually, roughly 85% of which are from emergency departments.
I would call your attention to the “Standard Miss Rate” column, which can be useful in guiding your vigilance and informing your educational program content. We can calculate these figures with a high degree of confidence based on our actual diagnosed findings plus missed findings identified by AI and our traditional QA overreads.
Aortic dissection and spinal epidural fluid are the two models I chose to present at this year’s SIIM Annual Meeting because they are our longest-running QA models that provide the most data. By adding up all the missed findings captured by our standard QA overreads, AI detection, and med-mal cases, we can normalize the data to determine that 1% of aortic dissections go unseen by the radiologist and (hold your hats) a whopping 49% of spinal epidural fluid collections.
Now at this point in my lecture I could sense people wondering, “what is wrong with vRad radiologists that they miss so many epidural fluid collections?” I’m quick to point out, however, that our accuracy is probably better than most, because our radiologists are emergency radiology specialists and more attuned to these findings than a generalist who doesn’t see the volume of emergency cases that we do. These are eye-opening figures that should guide the improvement efforts of all radiology practices.
With the miss rate and total study volume, we can then calculate the annual amount of med-mal cost saved based on our actual med-mal indemnity paid out over the years. We are saving $1,145,000 annually for aortic dissection and another $728,000 for spinal epidural fluid by using AI to get missed findings to the QA committee faster after the report is signed. Empowering the QA team with the latest technology leads to better patient outcomes and away from future litigation.
Critical epidural lesions: This past December, vRad’s Data Scientist and Machine Learning Engineer, Robert Harris, PhD, presented at the Radiological Society of North America annual meeting on the results of our AI model for critical epidural lesions — including epidural hematoma, epidural abscess, epidural phlegmon, and epidural fluid collection.
Critical epidural lesions can be catastrophic if they’re not caught, and our model is currently finding 30 to 40 per year that are not seen by the reading radiologists.
While less-commonly missed, SMA occlusions and splenic lacerations can have devastating consequences for the patient who goes home untreated, making these eight monthly misses well worth identifying.
A patient-first approach to AI with benefits for all in medical imaging
As the radiology industry faces high volume and a shortage of radiologists nationwide, healthcare providers need to use every tool at their disposal to deliver patients the best possible chance of early and effective diagnosis and treatment.
AI has proven itself to be a highly effective partner to our physicians, a safety net that is an essential (and complementary) component of the critical care provided by emergency diagnostic radiologists.
If you’re a radiologist or group interested in learning more about reading with an AI safety net, I recommend speaking with one of our physician recruiters or account managers who can be contacted here.