Scientific Rigor: Rock of Gibraltar or Sand Castle?

Nearly 200 years ago, the French novelist Honore de Balzac wrote, "For the journalist, anything probable is gospel truth." When it comes to sorting out fact from fiction in some of the medical literature, that phrase often comes to mind when you review how published findings become a part of conventional practice. While we are blessed with the likes of systematic reviews and meta-analyses in the rush to establish what we call "evidence" in the new postmodern evidence-based world, we need (as my chemistry professor liked to argue) to look at the kinetics as well as the robustness of the incorporation process of the evidence, in order to gain more insight into what truly is going on.

At one end of the scale is one of the most rigorous experiments ever performed by Barry Marshall, who as I discussed in this column space previously,¹ essentially subjected himself to an entire sequence of Koch's Postulates to nail down his demonstration in the early 1980s that a strain of the bacterium Heliobacter was the causative agent of peptic ulcers. Previously, the prevailing belief had been that ulcers were the consequence of living in families with dominant mothers and passive fathers.³ It was not until 2005 that Marshall finally was awarded the Nobel Prize for this elegant investigation.

At the other end of the spectrum is the infamous testimony of the Canadian neurologist John Norris,⁴ who insisted that dozens of cerebrovascular dissections were the result of chiropractic manipulation, until he contradicted his own testimony under cross-examination in an inquest, with his survey instrument found to be poorly constructed.⁵ This occurred amid a flurry of other papers of similar dubious construction, attesting to cervical manipulation as the probable cause of what likely is an overestimated number of cerebrovascular accidents.^6-12 The unhappy truth here is that this second body of literature seems to have gained immediate acceptance and media attention, whereas the first of almost unassailable logic remained in limbo for years until the prevailing beliefs about ulcers could be dislodged. Put in more pictorial terms, the Rock of Gibraltar lost. The sand castle won, with its presentation unfortunately capturing the attention of the public and that of several prospective insurers practically overnight.

So, if one were to decide how or whether to incorporate evidence into the pantheon of what is considered to be the most authoritative source of evidence, where would they turn? In cases such as this, I easily can relate to the massive frustrations of the renowned Italian film-maker Federico Fellini, who never seemed to be able to capture the proper attention of the priest to whom he periodically took confession in his hometown of Rimini. Either the priest would slap him upside the head profusely for what he considered to be grievous transgressions or he would immediately nod off to sleep within seconds, oblivious to Fellini pouring his heart out in great detail.

On Nov. 18, 2006, I was fortunate enough to attend a conference titled "Practicing Within Mainstream Health Care," presented by Integrated Health Care Practice Resources in Waltham, Mass. A multidisciplinary panel, bent on discussing the rationale and logistics of an integrated health care delivery model including chiropractic, wrestled with the concept of legitimacy, as demonstrated by the use of evidence appearing in the scientific literature. While the likes of David Sackett and others have developed models of evidence based upon the blending of experimental data and clinical judgement,13 it's all too easy to lose sight of the fact that flawed literature might be admitted into the body of what is regarded as sound experimental evidence, while deserving studies (such as Marshall's experimentation with ulcers) might be neglected for years. Once again, the sand castle prevails. Sand Castle 2, Rock of Gibraltar 0.

It gets worse. In the chiropractic literature, let us consider three glaring examples in which there is deliberate misrepresentation of the therapist held to be responsible for cerebrovascular accidents. The earliest, published over a decade ago by Alan Terrett, represents a careful retrospective of numerous studies in which the therapist involved was in fact not a chiropractor, but instead anything from a blind masseur, an Indian barber, a helipraktiker, to a kung-fu or lay practitioner. It was only after direct questioning from Terrett that the authors of the original articles recanted and admitted the true identities of the practitioners involved. Terrett rightfully concluded:

"The words chiropractic and chiropractor have been incorrectly used in numerous publications dealing with SMT [spinal manipulation therapy] injury by medical authors, respected medical journals and medical organizations. In many cases this is not accidental: the authors had access to original reports that identified the practitioner involved as a nonchiropractor. The true incidence of such reporting cannot be determined. Such reporting adversely affects the reader's opinion of chiropractic and chiropractors."¹⁴

But, of course, this act of misrepresentation was unfortunately followed 11 years later by a study from Germany that reviewed data from 13 academic neurology centers in which vertebral artery dissections were associated with "chiropractic" neck manipulation. The only problem is that in Germany, there is essentially a homeopathic dilution of chiropractors, with 60 or so practitioners swamped in a sea of some 10,000 helipraktikers who perform spinal manipulation and, for all intents and purposes, confuse the public as to who truly is qualified to deliver a spinal manipulation.¹⁵ This comes at a price, for it's clear in this second study that just 11 percent of the actual dissections occurred after visits to the offices of true chiropractors, with the remaining 89 percent occurring at the hands of orthopedic surgeons, physiotherapists or general practitioners.¹¹ Clearly, one can draw conclusions as to the relative safety of each of these professions performing the cervical manipulations. Basically, this portrayal of chiropractic is both a serious and unfounded misrepresentation of the chiropractic profession, and needs to be rectified immediately.

As if this weren't serious enough, we have one final retrospective, located in a PubMed search, of 24 different cases in Europe. In each case, the author implicated that a chiropractor delivered the care prior to reporting symptoms suggestive of a traumatic cervical injury. Once again, the authors relented in most cases under direct questioning and admitted that the term chiropractor had been inappropriately used to brand the therapists involved who, in fact, did not receive the formal training associated with chiropractic. In this instance, letters to the editor with this correction were happily accepted and published by all four journals involved, with one actually publishing a correction.¹⁶ Once again, the flaky science got more exposure than it ever deserved. Taking all these three studies together, you can draw up a final scorecard of: Sand Castle 3, Rock of Gibraltar 0.

Sadly, these three papers are but the tip of the iceberg. This particular category of studies occupies such a low echelon as to have drawn a well-deserved raspberry (known in some parts as a Bronx cheer) from Adrian Upton, the head of the Division of Neurology at the McMaster University School of Medicine: "Attacks against chiropractic concerning perceived risks of cervical manipulation are currently based upon 'coincidence, anecdotal reports and junk science.'"¹⁷

Even with the abundance of systematic literature reviews and meta-analyses which are supposed to represent more objective poolings of the refereed clinical trials, more rigorous filters are going to have to be put into place by a team of experts representing all relevant professions if we are ever going to admit the truly deserving evidence with more efficiency and transparency. In this manner, the Rock of Gibraltar finally will have had the opportunity to prove its mettle against the sand castle, which, in a more natural setting, would be simply and rightfully swept away by the next incoming tide.

References

1. Rosner AL. Toward a new epistemology through an accessible language. Dynamic Chiropractic, Feb. 12, 2005. www.chiroweb.com/archives/23/04/17.html.
2. Marshall BJ. Virulence and pathogenicity of Heliobacter pylori. Journal of Gastroenterology and Hepatology 1991;6(2):121-124.
3. Susser M. Causes of peptic ulcer: a selective epidemiological review. Journal of Chronic Diseases 1967;20:435-456.
4. Norris JW, Beletsky V, Nadareishville ZG. Sudden neck movement and cervical artery dissection: The Canadian Stroke Consortium. Canadian Medical Association Journal 2000;163(1):38-40.
5. Haneline MT, Lewkowich GN. An analysis of the etiology of cervical artery dissections: 1994 to 2003. Journal of Manipulative and Physiological Therapeutics 2005;28(8):617-622.
6. Lee KP, Carlini WG, McCormick GF, Albers GW. Neurologic complications following chiropractic manipulation: a survey of California neurologists. Neurology 1995;45(6):1213-1215.
7. Saeed AB, Shuaib A, Al-Sulaiti G, Emery D. Vertebral artery dissection: warning symptoms, clinical features and prognosis in 26 patients. Canadian Journal of Neurological Sciences 2000;27(4):292-296.
8. Hufnagel A, Hammers A, Schonle PW, Bohm KD, Leonhardt G. Stroke following chiropractic manipulation of the cervical spine. Journal of Neurology 1999;246(8):683-688.
9. Rothwell DM, Bondy SJ, Williams JI. Chiropractic manipulation and stroke: a population-based case-control study. Stroke 2001;32(5):1054-1060.
10. Smith WS, Johnston SC, Skalabrin EJ et al. Spinal manipulative therapy is an independent risk factor for vertebral artery dissection. Neurology 2003;60(9):1424-1428.
11. Reuter U, Hamling M, Kavuk I, Einhaupf EK, Schlelke E. Vertebral artery dissections after chiropractic neck manipulation in Germany over three years. Journal of Neurology 2006;253(6):724-730.
12. Cagnie B, Barbaix E, Vinck E. D'Herde K, Cambier D. Atherosclerosis in the vertebral artery: an intrinsic risk factor in the use of spinal manipulation? Surgical and Radiologic Anatomy 2006;28(2):129-134.
13. Sackett DL. "Evidence-Based Medicine" [Editorial]. Spine 1998;23(10):1085-1086.
14. Terrett AGJ. Misuse of the literature by medical authors in discussing spinal manipulative therapy injury. Journal of Manipulative and Physiological Therapeutics 1995;18(4):203-210.
15. "Info-Tell fur den Patienten." Deutsche Gesellscheft fur Manuelle Medizin, 2003, p. 2.
16. Wenban AB. Inappropriate use of the title chiropractor and term chiropractic manipulation in the peer-reviewed biomedical literature. Chiropractic & Osteopathy 14:16.
17. Upton A. Chiropractic therapy as seen by a neurologist. Lecture at 80^th Annual Spring Convention of the British Chiropractic Association. Bournemouth, United Kingdom: April 22, 2005. Quoted in: Chapman-Smith D. The Chiropractic Report May 2005;19(3):4.