Response to Evidence Review of Back Pain Therapies Published by Cherkin, et. al., in the Annals of Internal Medicine

Published back-to-back in the same issue of the Annals of Internal Medicine with a flawed meta-analysis by Assendelft, et al.,¹ is a "review of the evidence" by Cherkin, et al., for the management of back pain by a number of physical interventions, including acupuncture, massage therapy and spinal manipulation.² From the point of view of a reviewer basing his or her perspective on the literature pertaining to spinal manipulation, this overview of treatment alternatives is, unfortunately, highly problematic. In striving to "provide clinicians, patients and health plans with a clear and balanced understanding of the current evidence" pertaining to the use of these CAM (complementary and alternative medical) therapies, this paper, regrettably, offers precisely the opposite.

Most perplexing is its arbitrary inclusion and exclusion of studies to arrive at its conclusions. Although the authors' original intent was to cite systematic reviews and original articles describing the results of random controlled trials (RCTs) published since the reviews were conducted, there are too many examples, including:

1. qualifying RCTs that were excluded;
2. RCTs of dubious quality that were included;
3. descriptions of pertinent RCTs with no further indication why or how they were either included or excluded; and
4. the omission of earlier systematic reviews or meta-analyses with inadequate explanations of their shortcomings or flaws - or even how the current study built on these earlier investigations.

It is almost as if one were witnessing revisionist history.

More specifically, some of the shortcomings in Cherkin's paper can be summarized as follows:

1. Failure to resolve conflicting previous reviews. In an effort to filter out low-quality studies, rating systems of trial quality have abounded - attempts to assure that the edifice of evidence used to warrant a therapeutic approach is more than a house of cards. These form the cornerstone of systematic literature reviews and meta-analyses. Literature reviews are defined as comprehensive and rigorous reviews of the peer-reviewed scientific literature, requiring a predetermined threshold of graded quality for inclusion. In meta-analyses, on the other hand, actual effect sizes are calculated from pooled results of different clinical trials, using a variety of statistical procedures and taking into account the size of each study.

A meta-analysis by one of the authors (Shekelle) of the current study¹ retrieved 58 articles representing 25 trials, and supported the short-term benefit of spinal manipulation in some patients, particularly those with uncomplicated, acute low back pain. Data regarding chronic low back pain at the time of this publication were judged insufficient to evaluate the efficacy of spinal manipulation in managing this particular condition.³

Several years later, this qualification no longer stood. The rise in the stature of evidence supporting the use of spinal manipulation in managing chronic low-back pain could be described as meteoric. In a systematic review of 16 randomized controlled trials involving manipulation, van Tulder and his colleagues singled out two that were high quality.^4,5 Van Tulder's review explicitly stated that evidence supporting manipulation for chronic low back pain is found to be actually stronger than that for acute conditions: "There is limited evidence that manipulation is more effective than a placebo treatment for acute LBP [level 3]. There is no evidence that manipulation is more effective than [other] physiotherapeutic applications...or drug therapy...There is strong evidence that manipulation is more effective than a placebo treatment for chronic LBP [level 1]. There is moderate evidence that manipulation is more effective for chronic LBP than usual care by the general practitioner, bed-rest, analgesics, and massage [level 2]."⁶

A somewhat differing interpretation was reached in Bronfort's systematic review.⁷ In this case, the evidence supporting spinal manipulation for managing either acute or chronic low-back pain was judged to be "moderate," while that for a mix of chronic and acute low-back pain was considered "inconclusive." Furthermore, all but one of the back pain studies considered to be of sufficient validity were eliminated by the criteria invoked by van Tulder, while the latter study included one trial⁵ that had been rejected by Bronfort.

Yet another systematic review of randomized clinical trials cites adequate follow-up periods, avoidance of cointerventions, and avoidance of dropouts as frequent strengths. Recurrent weaknesses, however, include randomization procedures, sample sizes, and blinded assessments of outcomes that are virtually impossible to perform in a trial involving manual therapy.⁸ Finally, a meta-analysis of 51 literature reviews of spinal manipulative therapy suggests that, although the overall methodologic quality was low, nine of the 10 methodologically best reviews reached positive conclusions regarding spinal adjustments.⁹

The fact that systematic reviews may conflict in both their conclusions and as to which studies to accept is admittedly troublesome. None of these inconsistencies, conflicts or resolutions was ever brought to light by the current authors to clarify why their most recent review should be accepted as more definitive than past similar efforts.

2. Exclusions of previous studies. In terms of outcomes, there is no apparent reason why the highly positive findings of van Tulder⁶ or Giles¹⁰ were not integrated into the body of the authors' discussion concerning spinal manipulation. To further illustrate the inconsistency and arbitrary nature of this review, one need only wonder why the Giles study, in particular, was mentioned only in context of the acupuncture literature - in which neither back dysfunction nor pain significantly improved. In the context of spinal manipulation, in which significant improvements in both criteria were noted, there is no mention of any kind of this study in Cherkin's paper.

In reviewing cost-effectiveness, the authors argue that they only chose "the few effectiveness RCTs that measured cost" to minimize bias that might have been experienced with observational data. Why, then, did they entirely overlook a trial concerned with lumbar disc herniation (a subject the authors introduced only in connection with cauda equina syndrome)?

This particular orphaned investigation examined 40 patients with unremitting sciatica diagnosed as "due to lumbar disc herniation" with no clinical indication for surgical intervention. Subjects were randomized into two treatments:

1. chemonucleolysis (chymopapain injection under general anesthesia); and
2. manipulation (15-minute treatments over 12 weeks, including soft-tissue stretching, low-amplitude passive maneuvers of the lumbar spine and the judicious use of side-posture manipulations).

Back pain and disability were appreciably lower in the manipulated group at two and six weeks, with no improvement or deterioration in the chemonucleolytic group. By 12 months, there were improvements in both groups, with a tendency toward superiority in the manipulated cohort. Costs of treatment in the manipulated group were less than 30 percent of that encountered by the injected patients; furthermore, the latter group averaged expenditures of 300 pounds [approx. $472] for treatment failures, with no such costs experienced by the manipulated population.¹¹

3. Inclusion of study of questionable quality. In discussing cost-effectiveness and spinal manipulation in the context of RCT, Cherkin cites only his own study in which any advantages of spinal manipulation in terms of outcomes or cost-effectiveness, in comparison to physical therapy (McKenzie treatments), or the use of an educational booklet, were not evident.¹² The shortcomings of this study, including its misrepresentation of therapies and the overgeneralization of results, are so extensive that they have been presented elsewhere in no fewer than four separate venues.^13-16 In fact, the deficiencies of its design are so severe that they merited a special review by the Royal College of General Practitioners - which concluded that this particular RCT neither adds nor detracts from the evidence base regarding appropriate interventions for low back pain.¹⁷ Since this review can only be as robust as the literature on which it rests, one would have to scrutinize closely a review in which the lead author has chosen his own RCT, whose inadequacies may have neutered its clinical utility.

Concluding Remarks

From the perspectives of some of the recent literature addressing spinal manipulation, this review of outcomes and cost-effectiveness contains several glaring inconsistencies and omissions as to seriously undermine its credibility. It easily misses the mark in providing the "balanced" perspective originally sought by the authors and demanded of reviews of this nature. To summarize, Cherkin's review falls short of achieving its objectives due to:

• its failure to resolve conflicts in previous literature reviews;
• its inexplicable omissions of previous and highly pertinent studies; and
• the author's citation of only one of his own, widely discredited papers as the sole source of data regarding cost-effectiveness and spinal manipulation.

By appearing in an academic journal adjacent to another deeply flawed study¹ whose deficiencies have been addressed elsewhere,¹⁸ this paper creates a dangerously misleading impression to the effect that spinal manipulation is of limited or no value.

References

1. Assendelft WJJ, Morton SC, Yu EI, Suttorp MJ, Shekelle PG. Spinal manipulative therapy for low back pain: A meta-analysis of effectiveness relative to other therapies. Annals of Internal Medicine 2003;138:871-881.
2. Cherkin DL, Sherman SJ, Deyo RA, Shekelle PG. A review of the evidence for the effectiveness, safety, and cost of acupuncture, massage therapy, and spinal manipulation for back pain. Annals of Internal Medicine 2003;138:898-906.
3. Shekelle PG, Adams AH, Chassin MR, Hurwitz EL, Brook RH. Spinal manipulation for low-back pain. Annals of Internal Medicine 1992;117:590-599.
4. MacDonald RS, Bell CM. An open controlled assessment of osteopathic manipulation in nonspecific low-back pain. Spine 1990;15:364-370.
5. Sanders GE, Reinert O, Tepe R, Maloney P. Chiropractic adjustive manipulation on subjects with acute low-back pain: Visual analog scores and plasma beta-endorphin levels. Journal of Manipulative and Physiological Therapeutics 1990;13:391-395.
6. van Tulder MW, Koes BW, Bouter LM. Conservative treatment of acute and chronic nonspecific low back pain: A systematic review of randomized controlled trials of the most common interventions. Spine 1997;22(18):2128-2156.
7. Bronfort G: Efficacy of Manual Therapies of the Spine. Amsterdam, 1997, Thesis Publishers.
8. Koes BW, Bouter LM, van der Jeijden GJMG. Methodological quality of randomized clinical trials on treatment efficacy in low back pain. Spine 1995;20:228-235.
9. Assendelft WJJ, Koes BW, Knipschild PG, Bouter LM. The relationship between methodological quality and conclusions in reviews of spinal manipulation. Journal of the American Medical Association 1995;274(24):1942-1948.
10. Giles LGF, Muller R. Chronic spinal pain syndrome: A clinical pilot trial comparing acupuncture, a non-steroidal anti-inflammatory drug, and spinal manipulation. Journal of Manipulative and Physiological Therapeutics 1999; 22(6):376-381.
11. Burton AK, Tillotson KM, Cleary J. Single-blind randomised controlled trial of chemonucleolysis and manipulation in the treatment of symptomatic lumbar disc herniation. European Spine Journal 2000;9:202-207, 2000.
12. Cherkin DC, Deyo RA, Battie M, Street J, Barlow W. A comparison of physical therapy, chiropractic manipulation, and provision of an educational booklet for the treatment of patients with low back pain. New England Journal of Medicine 1998;339(15):1021-1029.
13. Rosner AL. Evidence-based clinical guidelines for the management of acute low back pain: Response to the guidelines prepared for the Australian Medical Health and Research Council. Journal of Manipulative and Physiological Therapies 2001; 24(3):214-220.
14. Freeman MD, Rossignol AM. A critical evaluation of the methodology of a low-back pain clinical trial. Journal of Manipulative and Physiological Therapies 2000;23(5):363-364.
15. Chapman-Smith D. Back pain, science, politics and money. The Chiropractic Report November 1998;12(6).
16. Rosner A. Fables of foibles: Inherent problems with RCTs. Journal of Manipulative and Physiological Theapeutics 2003;26(7):In press.
17. Royal College of General Practitioners. Unpublished update of CSAG guidelines (reference 12); 1999.
18. Rosner A. Response to meta-analysis by Assendelft et al. www.FCER.org, posted 07/09/03. Also in Dynamic Chiropractic Sept. 13, 2003;21(19):34-6, 38, 57.

Anthony Rosner, PhD
Brookline, Massachusetts
rosnerfcer@aol.com