Research Information: The Straight Poop

In the hours and days following the 136th running of the Belmont Stakes on June 5, I was informed that the neighborhood around the racetrack in Elmont, New York, was not a pleasant place to be. The prohibitive favorite, Smarty Jones, who had departed the starting gate at 2-5 odds, had just been whipped by a 15-1 nag - a relative long shot in the racing business. Thus, we're reminded of that classic axiom of railbirds and other racing aficionados: There is no such phenomenon as a sure thing.

And so it is with research, chiropractic or otherwise. A quotation I read at a recently dedicated art museum in Vienna sums it up perfectly: "Proof of the existence of information does, in fact, not exist in actuality, but is based on probability."¹

This is by no means to discredit or discount the value of research. Indeed, I have recently outlined, in some detail, the accomplishments and importance of research as it pertains to the practitioner.² But as I have done so many times before, I need to comment upon how research needs to be monitored and interpreted judiciously to avoid the plight of Rick, who, in the movie "Casablanca" informed the authorities as to why he had incredulously chosen to come take the waters at a locale in the middle of the desert: "I was misinformed."

A recent clinical investigation that gives us pause recently appeared in JAMA, featuring an author (Gotzsche) who has long been a compelling critic of the foibles of clinical research undermining its validity.^3,4 This most provocative study compared what is published in clinical studies with what the authors originally submitted for ethical approval, so that the research could be carried out in the first place. The disparities between the original designs and what appeared in the end could only be described as unnerving:

1. Some 71% of trials measuring an efficacy outcome had at least one unreported outcome.
2. In particular, 60% of the trials reporting a harm outcome had at least one unreported outcome.
3. When questionnaires were submitted to the investigators asking for details of these unreported outcomes, only 48% responded, and 86% of these initially denied the existence of missing outcomes until they were given an actual accounting of what was omitted.
4. Outcomes that were not statistically significant were more than twice as likely to be unreported.⁵

What this study is telling us is that such outcome reporting bias increased the prevalence of spurious results. In other words, improperly executed and poorly monitored research creates the real possibility of spreading misinformation. In even blunter terms, the unfortunate reader of many of these studies would have been hoodwinked, had Gotzsche and his co-workers not come to the rescue. To rectify this disturbing scenario, journal editors and reviewers need to demand from the authors the original protocols, together with the manuscripts submitted for publication. The authors also need to be more forthright in reporting any deviations from the outcomes listed from the original trial protocols.⁵ In terms of potential abuses and abuses of clinical research in general, I have published a more comprehensive assessment elsewhere.⁶

If clinical research is to be conducted properly, without these hobgoblins, where does FCER fit in? Simply through its ongoing sponsorship of thoroughly peer-reviewed research, an example of which (tension headache study) was rated among the highest in quality by four independently conducted published systematic literature reviews.^7-10 Also through its vigorous interpretations and rebuttals of spurious conclusions found in the scientific literature over the past decade,^11-17 more recently pertaining to cost-effectiveness misinterpretations in the literature,^18,19 FCER has taken a highly proactive position in enabling the chiropractic physician to interpret and articulate the current literature to both patients and prospective payors alike. Such is to simply state that FCER remains a primary source of reference and interpretive material pertaining to chiropractic research.

References

1. Kozlov C. "Information, No Theory." Museum Moderner Kunst Ludwig: Vienna, Austria, 2004.
2. Rosner A. Research in chiropractic: deliverance of the chiropractor. Chiropractic Economics 2004; In press.
3. Hrobjartsson A, Gotzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. The New England Journal of Medicine 2001; 344(21):1594-1562.
4. Johanssen HK, Gotzsche PC. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. Journal of the American Medical Association 1999;282(18):1752-1759.
5. Chan A-W, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Journal of the American Medical Association 2004;291(20):2457-2465.
6. Rosner A. Fables of foibles: inherent problems with RCTs. Journal of Manipulative and Physiological Therapeutics 2003;26(7):460-467.
7. Hurwitz EL, Aker PD, Adams AH, Meeker WC, Shekelle PG. Manipulation and mobilization of the cervical spine: a systematic review of the literature. Spine 1996;21(15): 1746-1760.
8. Kjellman GV, Skagren EI, Oberg BE. A critical analysis of randomised clinical trials on neck pain and treatment efficacy: a review of the literature. Scandinavian Journal of Rehabilitative Medicine 1999;31:139-152.
9. Bronfort G, Assendelft WJJ, Evans R, Haas M, Bouter L. Efficacy of spinal manipulation for chronic headaches: a systematic review. Journal of Manipulative and Physiological Therapeutics 2001;24(7):457-466.
10. Vernon H. The effectiveness of chiropractic manipulation in the treatment of headache: an exploration of the literature. Journal of Manipulative and Physiological Therapeutics 1995;18(9):611-617.
11. Rosner A. Unconventional medicine [letter to the editor]. The New England Journal of Medicine 1993;329(16):1201,1203-1204.
12. Rosner AL. Comparing the costs between provider types of episodes of back pain care [letter to the editor]. Spine 1995;20(23):2595-2598.
13. Rosner AL. Chiropractic: more good than harm or vice versa?" Journal of Manipulative and Physiological Therapeutics 1999;22(4):250-253.
14. Rosner AL. Evidence-based clinical guidelines for the management of acute low back pain: response to the guidelines prepared for the Australian Medical Health and Research Council. Journal of Manipulative and Physiological Therapeutics 2001;24(3):214-220.
15. Morley J, Rosner AL, Redwood D. A case study of misrepresentation of the scientific literature: recent reviews of chiropractic. Journal of Alternative and Complementary Medicine 2001;7(1):65-78, 79-82.
16. Rosner A. Chiropractic manipulation and stroke [letter to the editor]. Stroke 2001;32(9):2207-2208.
17. Rosner A. Stroke and spinal manipulation: basic questions. Journal of Quality Health Care 2004;3(2):12-15.
18. Rosner A. WorkmenÕs compensation issues and chiropractic care: Response to WorkersÕ Compensation Research Institute Data. Lexington, S.C.: Congress of Chiropractic State Associations, 2003.
19. Rosner A. WorkersÕ compensation costs and chiropractic: taking a position on center stage. Journal of the International Association of Industrial Accidents, Boards and Commissions 2004;41(1):22-49.

Anthony L. Rosner, PhD,
Director of Research and Education,
Foundation for Chiropractic Education and Research