Two Types of Unproven Claims

When I graduated from chiropractic college in 1961, my colleagues and I had never heard the term "unsubstantiated claim." We were well-aware of organized medicine's branding of chiropractic as quackery, and we had learned not to claim that we could cure any condition or disease. After all, Innate or Mother Nature did the healing, and we were there to help out by adjusting. "Randomized, controlled clinical trials" (RCTs) was also not a part of our lexicon. Our sense of science was the content of scientific knowledge; we were not trained in research methods, and most of us could not have distinguished between a hypothesis test and a literature review. Chiropractic was a science, we learned, since our theories and methods were consistent with the basic sciences, our principles were true, and because the usefulness of adjusting was proved every day in chiropractors' private offices. What's more, we were taught that it was essential to develop a strong belief in chiropractic in order to provide the maximum benefit to patients. The doctor's confidence was a strong ingredient in the success of chiropractic care.

Times have changed, and we've come a long way in the past few decades. Chiropractors now routinely publish in scholarly and scientific journals, and we are a presence at various international scientific conferences. I've enjoyed the honor of being named a senior member of the International Society for the Study of the Lumbar Spine (ISSLS); I'm in good company, as another senior member is one of our profession's best scientists, Dr. Scott Haldeman. A few of our colleges in the United States have received million-dollar research grants from the National Institutes of Health, and evidence-based federal guidelines for the care of back pain patients specifically endorse our method of care: spinal manipulation. Overseas, we now find more chiropractic colleges than here in the U.S., and most of these are state-university-based or state-funded institutions.

Yet, for all the progress we've made of late, many of us have still not learned to distinguish between our speculations (valuable as hypotheses) and those methods of healing which have gained validation through rigorous scientific studies. Our century-long tendency to claim more than we have scientifically demonstrated continues without pause, thereby limiting our credibility among health care policy-makers and the wider scientific community. I must confess that my own "crap-detector" is still more likely to go off when I hear a "negative" claim for chiropractic care than for a positive outcome, but the harm to our science is just as serious in either direction.

My friends in research tell me they take pains to avoid two kinds of mistakes when testing chiropractic hypotheses. Type 1 errors in research occur when we falsely conclude that some form of intervention (e.g., adjusting) was effective, when in fact it was not. Of equal concern are type 2 errors, which take place when the results of a clinical experiment lead the investigator(s) to falsely conclude that some treatment was not effective, although it actually was. Clinical researchers calculate various sophisticated statistics in order to put numbers on the probabilities of type 1 vs. type 2 errors in their investigations.

I'd like to suggest that this two-category classification for mistakes in scientific experiments can be extended to describe two kinds of unproven claims made to the public and in various professional forums. Type 1 unproven claims are those in which we insist that some part of the chiropractic art is useful, although we lack the controlled outcome studies (RCTs) to substantiate the claim. Type 2 unproven claims occur when we insist that some method of chiropractic care is not useful, although we lack the hard data to validate the notion. In days gone by, we might merely have distinguished between positive vs. negative claims for chiropractic (or pro-chiropractic vs. anti-chiropractic), but this language is rather unsophisticated and misleading. (In fact, we ought to be just as pleased when research leads us to the knowledge that something is or is not helpful to our patients, whether it violates our pet theories or not.) But here, we're talking about type 1 vs. type 2 unproven claims, not misinterpretation of research (type 1 vs. type 2 errors). The defining characteristic of an unproven claim is that we lack adequate information to draw a firm conclusion, but we draw one, nonetheless. When we ought to be reserving judgment, we go ahead and insist that something does or doesn't work, even though we don't really know, in any scientific sense.

This distinction between type 1 and type 2 unproven claims came to my attention recently through an e-mail exchange, in which it was asserted that the effect of the Activator instrument was no better than placebo. As you might imagine, I was annoyed. Admittedly, the relevant clinical outcomes literature currently consists of only three randomized comparisons of instrument vs. manual adjusting, and these three have small sample sizes. Nonetheless, these comparative trials suggest that instrument adjusting produces a benefit comparable to manual adjusting.^1-3 There were also sufficient data to earn favorable ratings for instrument adjusting in two clinical practice guidelines.^4,5 Nonetheless, there are folks who are making type 2 unproven claims for instrument adjusting based on little or no knowledge of the current literature.

Of course, the Activator is not the only victim of type 2 unproven claims. There are folks who acknowledge that there is benefit from manipulation for musculoskeletal problems, but also insist that manipulation does not benefit visceral disorders or behavioral problems. How do they know this? What justification do they have for rejecting the possibly broad value of our healing art? They have only their opinions, which are offered with a finality that defies the scientific method. And although once upon a time, we heard these type 2 unproven claims only from our antagonists in political medicine, nowadays, this nonsense originates within our own ranks!

Researchers must engage in sophisticated statistical analyses to determine (and try to avoid) type 1 and type 2 errors in their investigations. But it takes nothing more than resolve to avoid type 1 and type 2 unproven claims. We could, as C.O. Watkins, DC, suggested long ago, choose to be bold in what we hypothesize, but cautious and humble in what we claim. We could decide to leave the door open to clinical possibilities as yet untested or even unimagined, and proceed, based upon the best available evidence and our clinical experience, to meet our patients' unique and individual needs. We could acknowledge the areas of weakness in our knowledge base, and resolve to apply the best scientific methods to further develop the art of chiropractic. Some might even suggest that recognizing our areas of ignorance is the first step toward greater clinical wisdom.

There are two types of unproven claims made in chiropractic, and we cannot afford either of them. It is time, I suggest, for us to grow beyond our humble origins and to develop as a legitimate, first-class clinical art and science.

References

1. Gemmell HA, Jacobson BH. The immediate effect of Activator vs. Meric adjustment on acute low back pain: a randomized controlled trial. J Manipulative Physiol Ther 1995;18(7):453-6.
2. Wood TG, Colloca CJ, Matthews R. A pilot randomized clinical trial on the relative effect of instrumental (MFMA) versus manual (HVLA) thrust manipulation in the treatment of cervical spine dysfunction. J Manipulative Physiol Ther 2001;24(4):260-71.
3. Yurkiw D, Mior S. Comparison of two chiropractic techniques on pain and lateral flexion in neck pain patients: a pilot study. Chiropr Technique 1996;8(4):155-62.
4. Haldeman S, Chapman-Smith D, Petersen DM (Eds.): Guidelines for Chiropractic Quality Assurance and Practice Parameters: Proceedings of the Mercy Center Consensus Conference. Gaithersburg MD: Aspen, 1993, pp. 108-9.
5. Henderson D, Chapman-Smith D, Mior S, Vernon H (Eds.): Clinical Practice Guidelines for Chiropractic Practice in Canada. Proceedings of a consensus conference commissioned by the Canadian Chiropractic Association; held at the Glenerin Inn, Mississauga, Ontario, Canada, April 3-7, 1993. Toronto: Canadian Chiropractic Association, 1994, p. 110.