Rebuttal to Daniel Cherkin
Dear Editor:
There is no doubt that Daniel Cherkin's study published in the October 8th New England Journal of Medicine has accomplished the researcher's objective of creating a dialogue. In his recent response to criticisms of that study (Dynamic Chiropractic, December 14, 1998), the lead author has done one better with further statements which only appear to compound the faults that seemed to plague the study in the first place. They cannot afford to remain unanswered.
Out of the starting blocks in his response, Dr. Cherkin already stumbles onto thin ice by suggesting that both clinicians and therapists often overlook the fact that "low-back pain patients improve substantially over time, even if they receive little or no treatment" and that "the passage of time may be the most potent 'therapy.'"
This particular statement flies in the face of the recent data published by Peter Croft in the British Medical Journal which clearly indicates that, contrary to the widely held belief (virtually an urban legend) that the vast majority of all episodes of low back pain resolve within one month, no less than 60% of low-back pain patients initially presenting with pain and disability still complain of pain and disability 12 months later. In Croft's revealing study, the entire notion of acute low back pain is challenged with the conclusion that, rather than being a self-serving condition, it is "a chronic problem with an untidy pattern of grumbling symptoms."1
In the very next sentence, Cherkin suggests that studies of back pain treatments lacking comparison groups "invariably report high success rates for treatments under study." If this is so, I am mystified by his findings of recurrence rates of 50% and 70% in years one and two in each group. This seems to be a rather inauspicious way to begin a response to what I believe are viable and serious criticisms of the study, which I will address in the order of his points of reply.
- The study is an inaccurate and unfortunate representation of the patients who normally seek chiropractic care for low back pain. If the patients in this study were "not meant" to represent patients normally seeking chiropractic care for low-back pain, one wonders why this particular experimental design was initiated if, in fact, Dr. Cherkin wanted to establish continuity with the previous literature cited and address its deficiencies. If these patients were, in fact, only a subset of typical back pain patients, it then seems somewhat disingenuous to me for the paper to a) provide an indexed title which gives no indication of this fact but rather suggests more general treatment comparisons; b) open the discussion section with the deceptively broad statement "for patients with low-back pain"; and c) not even point out in the generalizability section of the discussion section the fact that patients were limited as Cherkin asserts.
- In this trial, only one high-velocity technique (side-posture) was applied. This entire argument comes down to representing chiropractic properly. In none of the locations in the paper which are most likely to be digested and quoted by the pubic, the press, or even other colleagues (title, abstract, results or discussion) was "side-posture" even identified. To an unsuspecting public, the danger with this type of workmanship is no less than if (for example) the use of steroid injections was found to be ineffective for a given condition, then referral to a medical physician could not be recommended.
- Doctors of chiropractic were limited in the types of treatment they could provide, while McKenzie practitioners were allowed full scope of therapy options and specially trained for the purpose of the project by Robin McKenzie. The criticism that was originally posed was whether ancillary procedures and patient counseling were properly applied in conjunction with whatever technique was identified as "chiropractic." There is no doubt that the efficacy of the individual maneuvers which are commonly practiced by chiropractors need to be researched; however, as pointed out immediately above, the study should have taken pains to represent these as merely components of what is available to the protocol of treatment known (and represented by the article) as chiropractic.
- Baseline values regarding severity among the three groups tested appeared to create a bias in the outcomes. I am still searching for what Dr. Cherkin reports as "less severe symptoms" in the chiropractic group as compared to the McKenzie and booklet cohorts. Instead, I find: a) the chiropractic group more than the other groups reports at east one day of bed rest due to LBP in the preceding week (35% vs. 24% and 22%); b) at least one day of work lost due to LBP in the preceding week (39% vs. 41% and 30%); and c) at least one day of restricted activity due to LBP in the preceding week (72% vs. 65% and 52%).
- The initial bothersome (symptoms) and Roland-Morris disability scores of 4 and 7-8 are substantially below the respective values of 6-7 and 10 which are more frequently observed in trials involving significant back pain. I did indeed erroneously read the initial values in the charts as baseline rather than one-week values. However, I am then puzzled why the plots began at one week rather than at baseline as is done customarily. Furthermore, the actual baseline values for the bothersome index (see sections 5-6 in Table 1 of the Cherkin NEJM article) still appear somewhat low.
- Clinical significance of observed treatment effects. There is no doubt that all three experiment groups displayed substantial improvement over baseline values in both the Roland disability and the bothersome scores. Whether the observed differences between groups are clinically significant is indeed open to question, although there was a consistent tendency to lower values at both four weeks and 12 weeks in the chiropractic group.
What remains unacceptable are the cost comparison issues raised in my earlier critique. The previous literature has suggested that payments to chiropractors for their own services constitute more than 80% of the costs per episode, while those to medical doctors per episode are only about 23% of the total costs.2,3 Dr. Cherkin's HMO "surcharge" of 50% above and beyond chiropractic costs, in addition to exceeding values seen in the literature, remains unexplained. Furthermore, as mentioned in my earlier critique, the actual distribution of costs about the mean need to be represented in order to identify skewing and possible statistical outliers.
- Treatment was given to a heterogeneous population without regard to standard assessment. As questioned previously and still unanswered, what were the criteria for assessing sciatica and rejecting affected patients from the study? It would be useful also to have considered (even separately) those patients filing for workman's compensation -- as these have been informed of the percentages or numbers of patients rejected due to their participation in workman's compensation.
In addition to providing refutations to each of the points raised in Dr. Cherkin's response, I feel obligated to enumerate those points in my original critique which remain unanswered:
- What were the precise characteristics of the medical intervention? What kind of attention, if any, was given to the patient in addition the literature, and what was the actual content of the booklet?
- What details are available regarding how patients were polled with respect to their expectations of treatment? Was the polling instrument validated?
- Once patients were eligible to participate, how many refused to participate and for what reasons?
- Exactly how was patient compliance assessed? Why is there an apparent contradiction between the percentage which physicians assume represents compliance (55%) and those reported by patients regarding their performance of the recommended exercises (78%), the use of lumbar rolls (71%), and recommended sitting postures (83%)?
- Are the practice patterns and patient attitudes of Washington state reasonable facsimiles of the national norms? If not, why isn't the state of Washington identified in the title and abstract of the study?
Until these questions and the points of refutation are adequately addressed, I am forced to conclude that the recent Cherkin study published in the NEJM is deeply flawed and not generalizable to a useful segment of the population of patients with low-back pain. My deepest thanks go out to Dynamic Chiropractic for allowing this debate to continue.
References - Croft PR, McFarlane J, Papageorgious AC, Thomas E, Silman AJ. Outcome of low back pain in general practice: a prospective study. BMJ 1998;316:1356-1359.
- Manga P. Enhanced Chiropractic Coverage under OHOP as a Means of Reducing Health Care Costs, Attaining Better Outcomes, and Achieving Equitable Access to Health Services. Report to the Ontario Ministry of Health, Ottawa, Ontario, February 1998.
- Manga P, Angus D, Papadopoulos C, Swan W. The Effectiveness and Cost-Effectiveness of Chiropractic Management of Low-Back Pain. Kenilworth, Richmond Hill, Ontario, 1993.
Anthony L. Rosner, PhD
Brookline, Massachusetts
