Why the ABR’s shift to standardized testing is bad for radiology
The American Board of Radiology (ABR) made significant changes to its certification program for diagnostic radiologists in recent years, removing the oral component of the exam and relying on a standardized format with multiple-choice questions. This shift toward psychometric testing has made the certification program less reliable, according to a new analysis published by the Journal of the American College of Radiology, and the ABR should consider working with various stakeholders and education experts to change it.
“A core component of the mission of the ABR is that it ‘serves patients, the public, and the medical profession by certifying that its diplomates have acquired, demonstrated, and maintained a requisite standard of knowledge, skill, understanding, and performance,’” wrote lead author Lincoln L. Berland, MD, University of Alabama at Birmingham, and colleagues. “Yet, there is weak evidence that the public’s concerns have been adequately understood, suggesting that it is reasonable to integrate public representatives into the re-evaluation process, possibly from among one or more of numerous existing patient advocacy groups.”
The ABR and other American Board of Medical Specialties (ABMS) member boards say they support psychometric testing because it is “established science” and “authentic,” the authors explained. But is this truly the case?
Strikes against psychometric testing
Berland et al. cited previous research on multiple-choice testing that found it to rely too heavily on an individual’s ability to take tests as opposed to measuring the individual’s actual ability. Prior research also found such testing to be “remarkably insensitive to differences in instruction.”
In addition, the authors added, this style of testing results in residents spending less time focused on clinical assignments and more time studying just to correctly answer these specific multiple-choice questions. It interferes “with mastering other important professional skills.”
“The residents reasonably conclude that studying for the ‘high-stakes’ test should be their priority over clinical experience,” the authors wrote. “This is reinforced by residency programs, which often condone this behavior.”
Berland and colleagues also noted that, thanks to this more standardized approach of testing, radiology trainees are now more likely to focus on studying “recalled” test questions—questions prior test takers remember after the fact and share with future test takers—than they were in the past.
Also, when trainees are taking the test, they are “constantly under surveillance (even in the restroom)” and they are put into an environment that hardly resembles an actual radiologist’s working environment.
An eye on reform
“Ironically, we believe that while purporting to be creating the ‘examination of the future,’ the ABMS and ABR have been mired in testing concepts based on decades-old thinking that do not apply to modern radiology and medicine,” the authors wrote.
So, what do they think could be done to improve this situation?
Berland et al. concluded that the ABR’s certification program could be updated “by including a deep commitment to involving stakeholders and education experts who study modern, evolving principles of learning.” Such reform, they believe, could help specialists feel more confident in the ABR’s ability to help radiologists provide the best patient care possible.
Michael has more than 18 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.