Comparative-effectiveness Research and Imaging: Insights and Ambitions

In addition to extending coverage to an estimated 31 million US residents, the recently passed HR 3590, the Patient Protection and Affordable Care Act, has ensured a future for comparative-effectiveness research (CER) by legislating funding, at $500 million per year, for the indefinite future. The stimulus bill launched the CER initiative with seed funding and required the Institute of Medicine (IOM) to identify a list of high-priority CER topics. Most recently, an article by VanLare et al¹ described a five-step process for determining how to translate those 100 priorities into a portfolio of specific research projects (see Additional Reading). ImagingBiz spoke with one of the authors, Harold C. Sox, MD, cochair of the IOM committee to set national priorities for CER, about its role in decision making, its potential use in answering imaging questions, and the recent controversial mammography recommendations of the US Preventive Services Task Force (USPSTF), which he chaired in the early 1990s.

“Anybody who is making coverage decisions, as well as physicians and patients, is going to benefit from CER because it’s going to help to make better decisions. Better evidence should mean better decisions.”

— Harold C. Sox, MD, chair of the IOM committee to set national priorities for comparative-effectiveness research

Sox spent his career preparing for his current position as cochair of the IOM committee. His many years of practicing medicine and studying medical decision making began in Northern California, where he served as chief of general internal medicine and director of ambulatory care at Palo Alto VA Medical Center, after which he chaired the department of medicine at Dartmouth Medical School from 1988 through 2001. Most recently, he completed eight years as the editor-in-chief of Annals of Internal Medicine. He also has served on numerous public-health and coverage-advisory committees, including chairing the USPSTF from 1990 through 1995 and the Medicare Coverage Advisory Committee for CMS from 1999 through 2003. Sox is understandably cautious, choosing his words with deliberation and carefully clarifying questions. The subject of comparative-effectiveness research has proven to be a lightning rod for health-reform dissenters, who characterized the proposed CER board as a death panel during the debate. Sox is not reserved, however, about the potential of CER to add clarity to medical decision making. “The health legislation, now the law of the land, states that there shall be a national CER institute, which will be public–private in governance,” he says. “It will be funded by a per–insured-head tax on Medicare and on private insurance companies. This tax will provide a source of continuing year-to-year funding that won’t be dependent on congressional appropriations, and that’s really a big deal. It means there will be a substantial CER effort going on for the indefinite future.” A common misconception about the CER board is that it will make coverage decisions, even though the language in the Senate bill states that the information generated would be used as part of the body of evidence that would include research not funded by CER money. “There was a lot of concern in the media that CER results would be directly linked to coverage decisions and would deny people care that they really need,” Sox notes. “In truth, CER is going to generate more evidence for organizations that do make coverage decisions, but that evidence will be part of a larger body of evidence, not the sole determinant of coverage policy.” In fact, as the CER effort moves forward, Sox predicts that the evidence will permit a more nuanced and personalized approach to whether a patient gets a particular treatment or test. “Currently, with coverage decisions, either you get it or you don’t, but in fact, as we look at things more closely, we are going to find that it is not that simple,” Sox says. “Patients with certain characteristics may do better with a test that, for the average person, would not be the preferred choice. I can see some more subtle coverage decision making happening as we learn better how to choose the best test or treatment for a particular patient with a particular profile of clinical findings.” Imaging Applications and USPTFWhen asked how CER might be applied to the problem of inappropriate imaging, Sox begins by explaining that in the clinical setting, he thinks of testing as changing probabilities. For instance, when Sox sees a patient, he takes a history and makes a guess as to how likely it is that the patient has the disease about which he or she is worried. If there is a high probability of disease, he might forego the test because even a negative result will not change the probability enough to change the treatment. Likewise, with a low probability, even if the result of the test is positive, the probability would not be high enough to start treatment. He expects that CER will provide better information about predicting the probability that a patient has a disease and, possibly, how well a test performs in practice. “This is a much more clinically realistic way to think about tests than simply that this test has a higher sensitivity or specificity than another test, which is the usual way that you compare diagnostic tests,” Sox says. “It’s using the information that you get from measuring the performance of a test—sensitivity and specificity—but using it in a way that individualizes the care of the patient by factoring in the pretest probability that the patient has the target condition.” Rather than limiting the indications for diagnostic imaging, Sox suggested that CER could result in broadening coverage for it. “It’s hard to predict how coverage decisions will adapt to having this more fine-grained information, but it may well be that it will broaden coverage,” Sox suggests. “It will mean more tests could be covered because, at least in some patients, they will have the potential to be the best test. Again, it’s hard to predict, but I would say that CER could get us to a more thoughtful and subtle way of deciding when the test should be done and when it shouldn’t, and therefore, whether it should be covered by insurance.” When asked to comment on the recent controversial mammography recommendations from the USPSTF, Sox was quick to emphasize that the task force is a guideline committee, and is not engaged in CER. “The first thing to realize is that they made what is called a C recommendation in 2002, which is what you make when the balance of benefits and harms is really close, so that it’s a toss-up decision,” he explains. “According to the task force’s rules, in 2002, a C recommendation meant that physicians and patients who used the screening should continue to do it. For people who weren’t getting screening, it would be perfectly reasonable not to do it. In other words, the evidence wasn’t strong enough for people to switch from screening to nonscreening, or vice versa.” Between 2002 and 2009, the evidence about mammography did not change. The task force changed its rules, however, so that it would recommend against routinely doing a preventive service when the evidence was at the C level. Therefore, in 2009, it recommended against routinely doing screening mammography in women aged 40 to 49 years. “It changed its own rules,” Sox notes. “In some sense, it painted itself into a corner with its new rules for what to recommend with C-type evidence when, in fact, the evidence hadn’t changed at all.” CER BasicsIn order for CER to advance medical decision making, a study needs three elements, Sox says. The first requirement is for the study to compare interventions that are known to be effective, where the only question is which one is more effective. For instance, CER is not the place for placebo-controlled clinical trials. Second, the type of patients that CER will enroll should be typical of daily practice, unlike those enrolled in many randomized trials that eliminate or exclude patients from the study if they have life-threatening conditions. “CER addresses this issue,” Sox notes. “It says the study populations should be representative of typical clinical practice, which means that there should be relatively few patients excluded from the study. The result of this requirement is that physicians and patients can be more confident that the evidence applies to their cases.” The third big piece that makes a study a CER study is the focus on obtaining information that helps make patient-specific decisions, Sox explains. For example, in a randomized comparison of two treatments, the researchers would gather a fairly detailed, standard body of clinical information about all of the patients in the study and use that information to try to predict a patient’s response to each treatment. “That’s the third big element that is unique about CER: It’s focused on patient-centered or individualized decision making, and it attempts to get the evidence that will make that possible,” Sox says. Sox is optimistic about the potential for CER to improve the evidence on which organizations like the USPSTF and the American College of Physicians base their recommendations. “CER has the potential to help individualize decision making about a test and to make more thoughtful and individualized decisions possible about which test to do, or whether to do the test at all,” Sox believes. “Physicians and patients should welcome CER.” Additional Reading—Five Steps to a CER Project PortfolioCheryl Proval is editorial director of ImagingBiz.com and vice president, publishing, of ImagingBiz, Tustin, California.