Comparative Effectiveness Research: No Longer Stuck in Neutral

Where has the American health research initiative been taking us all these years? At the risk of sounding like the perennial brat who whines, "Are we there yet?" let me explain. With the urgency and frustration of Humpty Dumpty's followers trying to reconstruct the severely cracked soft-boiled egg that represents much of the nation's health and health care delivery system, we have witnessed the birth of the Public Health Service which begat the NIH to initiate and fund research in the health-related sciences, which begat the OAM (Office of Alternative Medicine) to bring this research into clinical studies relating to nontraditional types of health care, which begat NCCAM (National Center of Complementary and Alternative Medicine) to crank up the OAM into high gear, which begat the WHCCAMP (White House Commission on Complementary and Alternative Medicine) to get our researchers to be able to actually work together and to put a hopefully waking-up call before Congress in the form of (what else?) a report.

Oh yes, and then we saw the creation of the AHCPR (Agency for Health Policy Research), which was designed to get doctors to actually change their practice patterns so they conformed with health policy guidelines that were presumably constructed out of the most robust research to date. Add to this the IHI (Institute for Healthcare Improvement), designed to change practice patterns incrementally at the institutional end. Finally, we saw the IOM (Institute of Medicine) jump into the fray by publishing such sobering treatises as "Crossing the Quality Chasm" to tell us how incredibly broken down and glacial our health care system really is.¹

Keep in mind that all these efforts did not necessarily move in a uniformly forward direction. Because they were considered too controversial for offering their opinions on the effectiveness of health care, the AHCPR,² Congressional Office of Technology Assessment and National Center for Healthcare Technology all had to yield to pressure by opponents of their work. The Office of Technology Assessment, which was created in 1972, disappeared in 1995 after the Republicans took over the majority of the House of Representatives in the 1994 elections.³

That gumbo represents the institutional side of things. On the actual research side, it's pretty much the same story. We've witnessed the birth of the randomized clinical trial,⁴ its ascendancy to the top of the hierarchy of clinical research,⁵ the introduction of systematic reviews and meta-analyses to corral and rank the clinical research to date,⁶ the creation of such bodies as the Cochrane Collaboration⁷ and Yale Prevention Research Center⁸ to archive this research in orderly fashion, the unleashing of criticisms of the RCT telling us how it sometimes misses the boat,⁹ the subsequent "greening" of RCTs in response with such concepts as pragmatic clinical trials,¹⁰ practice-based research¹¹ and whole systems research,^12,13 and finally, attacks upon evidence-based medicine itself^14,15 and rolling out the notion that the traditional pyramid ranking clinical research evidence is actually lacking and should really be reconstructed into something more resembling an actual house - one that could finally admit such key players as basic research, epidemiological studies, and health systems research as integral parts of advancing our health care knowledge.¹⁶

Understandably, then, this would put us precisely at the point at which we'd hope for some restoration of order and perspective. With the clarion call of Gabriel's trumpet, a superb paper has emerged from the Urban Institute which, in my opinion, has emphatically achieved that objective.³ It speaks of comparative effectiveness research (CER), which is simply defined by the Institute of Medicine (IOM) as "the comparison of one diagnostic or treatment option to one or more others."¹⁷ The American College of Physicians takes this definition one step further to include comparisons in safety and cost. Taking the definition yet further, the IOM later deemed that CER encompasses the medical, economic, social, and economic implications of the application and diffusion of an intervention used to promote health. Finally, the IOM extended the definition to include alternative approaches to health care delivery, and that CER is intended to assist consumers, clinicians, purchasers, and policy-makers alike to make informed decisions to improve health care at both the individual and population levels.¹⁸

What is being said here is that not just more research is needed, but better research. Among the recommendations of the Urban Institute are to do the following:

• Involve patients, clinicians, payers and other decision-makers in key phases of CER study development and implementation.
• Develop a range (italics mine) of research methods grounded in empirical data to replace (italics mine) the traditional hierarchies of evidence, in keeping with Jonas' apt "evidence house" mentioned earlier.¹⁶

This means being able to admit such items as more basic research; indirect costs involving time lost from work, retraining, and home assistance; and patient values and expectations, which have been shown to skew the results of randomized controlled trials;^19,20 Essentially, CER answers concerns that much of our current research either is not designed or not understood to affect practical questions of risks or benefits that are of most concern to patients, physicians, and other individuals involved in decision-making. It emulates what both the AHCPR and the late FCER have been banging their head against the wall for decades in attempting to accomplish one primary goal: Translate research into practice.

The need for such action has never been greater. It has been found, for instance, that it takes an average of 17 years to incorporate the discovery of more effective means of treatment into routine patient care.²¹ Witness, for instance, how hand washing was found as early as the 1840s to reduce infections and deaths in hospitals, yet compliance with hand-washing standards in hospitals still stands at only 30 percent to 50 percent.²² Add to this the fact that there is still a lack of effectiveness research on the street; i.e., research conducted under average conditions in diverse populations and clinical practice settings, as opposed to the artificial protocols often imposed in traditional clinical trials.²³

Unlike the situation in Canada and European countries, which assume a major role of government in financing and delivering health care services, the United States lacks the infrastructure to utilize or even implement CER. Such issues as comparative effectiveness and costs and return to work seem to have escaped most private insurers in their objective assessment of what is truly the evidence in evidence-based medicine.

Until a serious effort is made to crank up our efforts at CER, the United States will remain very much Third World in its attempts to deliver efficient and equitable health care. Historically, for instance, less than 0.1 percent of what is more than $2 trillion in annual U.S. health care expenditures had been allocated to work on CER. One sign of encouragement has been the American Recovery and Reinvestment Act (ARRA), which has infused an additional $1.1 billion for new CER to be overseen by the HHS, NIH and AHRQ. Hopefully, that trend will continue and expand in the months and years to come, spearheaded by the very astute and timely paper from the Urban Institute.³

Let it also be understood that CER implies that a far more serious effort be made to compare nonpharmacologic health care interventions to pharmacologic, the latter having taken the lion's share of research funding and publication - with the tendency to skew publications toward positive results and use inferior controls.²⁴ Under such circumstances in which no less a body than the House of Commons Health Committee in the United Kingdom concluded that "pharmaceutical companies will inevitably continue to be the dominant influence in deciding what research is undertaken,"²⁵ research addressing nonpharmacological interventions will inevitably be crushed and obscured by the sheer weight of drug-related research. CER represents a viable attempt to at last level the playing field, allowing nonpharmacologic interventions to emerge from the shadows and receive a greater share of funding, public attention, and reimbursements from third-party payers.

The CER report from the Urban Institute³ provides a call to arms for us to redouble our efforts to support and get involved in research addressing chiropractic, nutrition, applied kinesiology and many other areas of practice with which our readership is most familiar. In forthcoming columns, I am looking forward to discussing this very topic in greater detail.

References

1. Committee on Health Care Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academy Press, 2001.
2. Gray BH, Gusmano MK, Collins SR. AHCPR and the changing politics of health services research. Health Affairs (Web exclusive), June 25, 2003.
3. Docteur E, Berenson R. How Will Comparative Effectiveness Research Affect the Quality of Health Care: Timely Analysis of Immediate Health Policy Issues. Washington, D.C.: The Urban Institute, February 2010:1-15.
4. Fisher RA. The Design of Experiments. Edinburgh, SCOTLAND: Oliver & Boyd, 1935.
5. Fisher CG, Wood KB. Introduction and techniques of evidence-based medicine. Spine, 2007;32(19S): S66-S72.
6. Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Evaluations in the Health Professions, 2002;25(1):12-37.
7. Clarke E. The Cochrane Collaboration and the Cochrane Library. Otolaryngology, Head and Neck Surgery, 2007;137(4 Suppl): S52-S54.
8. Katz DL, Williams A-I, Giard C, Goodman J, Comerford B, Behrman A, Bracken MB. The evidence base for complementary and alternative medicine: methods of evidence mapping with applications to CAM. Alternative Therapies in Health and Medicine, 2003;9(4):22-30.
9. Rosen L, Manor O, Engelhard D, Zucker D. In defense of the randomized controlled trial for health promotion research. American Journal of Public Health, 2006;96(7):1181-1186.
10. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association, 2003;290(12):1624-1632.
11. Nyiendo J, Lloyd M, Haas M. Practice-based research: the Oregon experience. Journal of Manipulative and Physiological Therapeutics, 2001;24(1):25-34.
12. Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A. Complementary and alternative whole systems research: beyond identification of inadequacies of the RCT. Complementary Therapies in Medicine, 2005;13:206-212.
13. Hawk C, Khorsan R, Lisi AJ, Ferrance RJ, Evans MW. Chiropractic care for nonmusculoskeletal conditions: a systematic review with implications for whole systems research. Journal of Alternative and Complementary Medicine, 2007;13(5): 491-512.
14. Holmes D, Murray SJ, Perron A, Rail G. Deconstructing the evidence-based discourse in health sciences. Truth, power, and fascism. International Journal of Evidence-Based Healthcare, 2006;4:180-186.
15. Henry SG, Zaner RM, Dittus RS. Viewpoint: moving beyond evidence-based medicine. Academic Medicine, 2007;82(3):292-297.
16. Jonas WB. The evidence house: how to build an inclusive base for complementary medicine. British Medical Journal, 2001;75:79-80.
17. Institute of Medicine Roundtable on Evidence-Based Medicine: Learning What Works Best: The Nation's Need for Comparative Effectiveness in Healthcare. Washington, D.C.: Institute of Medicine, 2007.
18. Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. Report Brief. Washington, D.C.: Institute of Medicine, 2009.
19. Weinstein JD, Tosteson TD, Lurie JD, Tosteson ANA, Hanscom B, Skinner JS, Abdu WA, Hillrand AS, Boden SD, Deyo RA. Surgical vs nonoperative treatment for disk herniation: the Spine Outcomes Research Trial (SPORT): a randomized trial. Journal of the American Medical Association, 2006:296(20):2441-2450.
20. Weinstein JD, Lurie JD, Tosteson TD, Skinner JS, Hanscom B, Tosteson ANA, Herkowtiz H, Fischgrund J, Camissa FP, Albert T, Deyo RA. Surgical vs nonoperative treatment for disk herniation: the Spine Outcomes Research Trial (SPORT): observational cohort. Journal of the American Medical Association, 2006;296(20):2451-2459.
21. Balas EA, Boren SA. Managing Clinical Knowledge for Healthcare Improvement. In: Yearbook of Medical Informatics. Bethesda, MD: National Library of Medicine, 2000.
22. McGuckin M, Waterman R, Govednik J. Hand hygiene compliance rates in the United States: a one year multicenter collaboration using product/volume usage measurement and feedback. American Journal of Medical Quality, 2009;24(3):205-213.
23. Rich EC. The policy debate over public investment in comparative effectiveness research. Journal of General Internal Medicine, published online April 21,2009.
24. Djulbegovic B, Lacevic M, Cantor A, Fields K, Bennett CL, Adams JR, Kuderer NM, Lyman GH. The uncertainty principle and industry-sponsored research. Lancet, 2000;356:635-638.
25. House of Commons Health Committee. The Influence of the Pharmaceutical Industry. Fourth Report of the Session 2004-2005. Volume I. Ordered by the House of Commons to be printed 22 March 2005.