New training tool shows promise in improving radiologist performance
A training program being used in Australia and New Zealand has been shown to help radiologists and radiology trainees interpret mammograms, according to a new study published in Academic Radiology.
The Breast Screen Reader Assessment Strategy (BREAST) program was established by specialists from the University of Sydney and BreastScreen New South Wales. Users interpret test sets of mammograms, submit their answers and then receive immediate feedback through a web-based application that shows how they did.
To test the program’s impact on reader performance, the study’s authors tracked how it affected 40 radiologists from BreastScreen Australia and 10 radiology registrars. Each specialist completed at least three different BREAST test sets between 2011 and 2017. The median age of the radiologists was 50 years old. The median age of the registrars was 34 years old.
“Readers were informed that the test sets had been cancer-enriched relative compared to a typical screening population, but the specific cancer frequency was not revealed,” wrote Phuong Dung (Yun) Trieu, PhD, MDR, of the Faculty of Health Sciences at the University of Sydney, and colleagues. “For each case, participants were asked to indicate whether they would recall or not recall the case, and give a score based on the RANZCR Breast Imaging Reporting and Data System lexicon definition.”
Overall, “significant improvements” of 21 to 31 percent were seen in radiologists’ lesion sensitivity after they completed a later test set compared to when they completed their first test set. Eighty-three percent of radiologists had higher lesion sensitivity after completing their first test set.
While 60 percent of registrars saw their lesion sensitivity increase on their second test set, 100 percent of registrars saw an increase on their third test set. Registrars also showed an increase in their Jackknife Free-Response Receiver Operating Characteristic (JAFROC), which takes lesion sensitivity, specificity and confidence into account.
“Overall, results show that later readings of BREAST showed higher performance scores particularly for lesion sensitivity among radiologists and registrars compared to the first test set reading and this improvement was recorded in up to 83 percent of radiologist fellows and 100 percent of radiology trainees,” the authors wrote.
The researchers also noted that progress for registrars seemed to be “happening at a slower pace as the improvement in scores were not found until they had completed the third test set, which might reflect the lower level of interpretive skills of the trainees.”
The authors also wrote that they could not say for certain that all changes in reader performance were directly related to the BREAST program. “For example, at the time a test was taken, a reader might have been involved in interpreting more mammograms in clinical practice compared to the time this reader completed the first BREAST test set,” they wrote.
Still, the team concluded, there is enough evidence to conclude that asking radiologists and trainees to train using the BREAST program can at least partly contribute to improved performance.
Michael has more than 18 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.