'Jack-of-all-trades' AI model outperforms numerous others in CT-related tasks
Researchers from Stanford University have developed a vision-language model capable of an array of diagnostic tasks that could one day make all CT scans opportunistic exams.
Dubbed “Merlin,” the machine learning model was designed to assess 3D abdominal CT scans; researchers trained it to achieve numerous functions, including identifying anatomical abnormalities and predicting long-term disease processes. Unlike many AI models, Merlin was not built with one specific task in mind, but with many. Researchers believe that with a little fine-tuning, the model’s capabilities could be expanded even further.
“With Merlin, you could potentially go beyond traditional radiology and jump straight from imaging to a possible diagnosis. And that’s just one potential use,” co-first author Louis Blankemeier, PhD—who conducted this work while a graduate student at Stanford University and is now CEO of Cognita, acquired by Radiology Partners last year—said in a statement from the National Institutes of Health.
The foundational model was trained on a clinical dataset of over 15,000 paired CT scans comprising more than 6 million images; over 1.8 million diagnosis codes and tens of thousands of radiology reports also were included in its training data. This, the researchers detail in Nature, represents the largest collection of abdominal CT data to date.
“Rich datasets like this are necessary to push the limits of what artificial intelligence models can accomplish in medicine,” Bruce Tromberg, PhD, director of NIH’s National Institute of Biomedical Imaging and Bioengineering (NIBIB), which is supporting the study, said in a statement. “This work exemplifies how meticulously crafted training data can enable remarkable insights that significantly streamline workflows and assist in clinical decision-making.”
Merlin was tested on over 50,000 abdominal CT images from four different hospitals. Experts challenged the model to a myriad of tasks, including predicting diagnosis codes, identifying and outlining specific anatomy, drafting radiology reports, predicting disease onset and more.
The model performed respectably at all tasks, though some (drafting radiology reports) required additional training. It was able to predict which scans were likely to be associated with specific diagnostic codes over 80% of the time, yielding 90% accuracy with a specific subset. It also could predict which patients were at higher risk of developing a particular disease in the next five years 75% of the time—outperforming other models designed for that specific task. In fact, it performed in line with or better than several AI models that have been developed with dedicated functions in mind.
Those involved in Merlin’s development are optimistic about its future and have set their sights on obtaining regulatory approval.
“Our model and the data will provide the community a robust backbone to build upon,” said senior author Akshay Chaudhari, PhD, a professor of radiology and biomedical data science at Stanford University and co-founder of Cognita. “From here, the sky’s the limit.”
Read more here.
In addition to her background in journalism, Hannah also has patient-facing experience in clinical settings, having spent more than 12 years working as a registered rad tech. She began covering the medical imaging industry for Innovate Healthcare in 2021.