Patricia Brennan's dysmenorrhea study, a full-scale clinical trial begun in 1992, has resulted in a paper accepted for publication in April 1999 by the journal Pain.1 It was preceded by a pilot study addressed to the same issue which was published in the Journal of Manipulative and Physiological Therapeutics in 1992.2 Unfortunately, the recent publication in Pain suffers from a number of design flaws, omissions of data, and unexplained contradictions with the earlier pilot study -- all of which significantly compromise its validity and leave more questions unanswered than resolved. One wishes that greater efforts were made by the authors to resolve what had appeared to be a positive result in the earlier investigation with the negative findings obtained in the more recent publication, above and beyond what sometimes might be expected with the differences of population types and numbers between the two investigations.
The purpose in both of these studies was to assess the comparative effects of a high-force spinal manipulative therapy (SMT) and low force maneuver (LFM) in affecting the pain and prostaglandin levels experienced in women with primary dysmenorrhea. The full-scale trial employed patient groups of 65-70, approximately three times larger than those employed in the pilot study. In addition, it observed patients over a total of four menstrual cycles: one as a baseline control after which subjects were randomized into the two intervention groups, and three over the course of treatments applied.
In both studies, three outcome measures were evaluated: (1) pain scores as determined by the visual analog scale (VAS); (2) prostaglandin levels measured as blood plasma concentrations of the metabolite KDPGF2a taken from venipuncture 60 minutes after treatment; and (3) scores obtained on the Moos Menstrual Distress Questionnaire (MDQ). The purpose of monitoring additional menstrual cycles in the full-scale study was to allow the researchers to observe the damping of possible nonspecific effects of the interventions over time (called washout).
These were the outcomes obtained:
- Pain scores. In both the pilot and full study, decreases were observed from pre- to post-treatment. However, in the full study, the difference between the sham and treatment groups was not statistically significant, whereas in the pilot study it was.
- KDPGF2a scores. In both the pilot and full study, decreases were observed from pre- to post-treatment. In the pilot study, standard error ranges suggested that the decreases observed in the SMT group were statistically greater than those observed in the sham group; however, an analysis of covariance suggested that the sham and SMT groups were not significantly different from one another. In the full study, no significant differences between the treatment arms could be detected; in fact, it appeared that the magnitude of the metabolite decrease was less in the SMT group as opposed to the sham population -- opposite from what was seen in the pilot study.
- Menstrual distress questionnaire. The pilot study showed a statistically significant difference between groups, the decrease in the SMT group being greater. No such difference was reported in the full study, although the specific raw data could not be found in the paper for proper assessment.
The risk, of course, is that the take-home message from the full-scale study will be that chiropractic is ineffective for managing primary dysmenorrhea. What is disappointing in this publication are a number of design flaws that were not evident in the original research plan, summarized in the following paragraphs. A second problem is that there are a number of inconsistencies between the pilot and full-scale trials.
The inconsistencies can be best appreciated from the raw data pooled from the two studies:
| 1. Pain: VAS (cm): | | | | |
| | SMT | | LFM | |
| | Full* | Pilot | Full | Pilot |
| Pre- | 4.28 | 5.87 | 3.80 | 6.00 |
| Post- | 3.27 | 3.78 | 3.00 | 5.19 |
| 2. KDPGF2a (pg/ml): | | | | |
| | SMT | | LFM | |
| | Full* | Pilot | Full* | Pilot |
| Pre- | 128.6 | 133.9 | 130.2 | 142.8 |
| Post- | 124.3 | 116.1 | 119.2 | 126.3 |
| 3. Distress: MDQ (%): | | | | |
| | SMT | | LFM | |
| | Full* | Pilot | Full* | Pilot |
| Pre- | ? | 44.2 | ? | 47.9 |
| Post- | ? | 25.2 | ? | 37.6 |
| | | | | |
- Indicates outcome at end of menstrual cycle #2, the closest approximation to the single menstrual cycle evaluated in the pilot study.
Other than a graph which displays similar post-MDQ scores of 36-37% for either the SMT or LFM interventions, there is virtually no data offered in the manuscript (other than a statistical statement) to support the authors' contention that there is no significant difference between the two intervention groups with regard to this particular outcome measure.
From the data presented above, it is apparent that:
- Pain scores at baseline are 27-37% lower in the full-scale trial as compared to the pilot. This essentially means that patients in the full-scale trial were in less pain and therefore had less of a range with which to assess clinical improvement.
- With regards to the LFM, pain scores are significantly (73%) higher at post-intervention for the pilot study as compared to the full-scale trial. This could be a reflection of the greater presenting pain (higher baseline values) of the patients in the pilot study as discussed immediately above (#1). It could also indicate that the sham procedure in the full-scale trial was considerably more effective than in the pilot study. Finally, a combination of the two preceding phenomena could be at play.
- With regards to the SMT, post-intervention pain scores appear to be similar in the pilot and full-scale studies.
- KDPGF2a scores appear to be somewhat more erratic, being sharply elevated for the baseline value for the LFM group and depressed at post-intervention in the SMT cohort in the pilot study only. Both the pilot and full-scale studies are similar in that they indicate decreases in the values from pre- to post-intervention in both groups, which are probably not statistically significant from each other.
- MDQ scores, while more difficult to fathom, appear to have decreased only half as much from pre- to post-intervention in the SMT cohort in the full-scale trial as compared to the pilot study. This may reflect a difference in the initial pain of participating patients (as suggested in #1 above) and/or a deviation in the type of adjustment delivered or other undetected interaction with the patients.
These inconsistencies point the way to several design flaws evident in the full-scale trial:
- Preconditioning of patients. In the full-scale study only, there are indications that the patients admitted into the study were (1) administered a "superficial effleurage" (light body massage) prior to administering either LFM or SMT, and (2) told to refrain from the use of NSAIDs or other analgesics with prostaglandin synthetase-inhibiting activity the week before the expected onset of each menstrual period, which may not be a sufficient period to allow complete washout of the medication. Either of these effects would diminish the severity of symptoms at the outset of the trial (suggested by much of the data presented above), diminish any clinical improvements obtained at post-treatment, and mask possible differences between the two treatment groups.
- Nature of the sham adjustment. Overlap of the mimic and SMT effects has been consistently referred to in both the pilot and full-scale studies. Even though sham procedures were not to have exceeded 400 newtons while SMT procedures typically involved thrusts exceeding 750 newtons, a growing body of evidence suggests that nonspecific effects observed in the sham procedure, as well as devising a technique which stands clearly apart from chiropractic intervention, both require far more attention than previously shown. Perhaps the fairest single statement in Dr. Brennan's entire paper is the final sentence, which indicates that the results of this trial "... are strong evidence that either the low force mimic maneuver was an insufficient placebo treatment or, in fact, that manual therapy does not relieve the pain in women with primary dysmenorrhea." (italics added)
- Lack of waiting control group. Despite the fact that the blinding itself was effective in masking the patients' perceptions of sham and actual treatments, no group was included which delayed or withheld physical contact with the patient. It would seem rather disingenuous for the authors to have suggested in the paper that the level of available funding precluded including this valuable control, as if such funding agencies as the Foundation for Chiropractic Education and Research (FCER) and NCMIC Group, Inc. (NCMIC) might somehow be at fault. In the grant application stages to FCER and NCMIC, waiting list controls were in fact originally stipulated by the authors but were unfortunately dropped during the course of the study.
- Effects of exercise. Since exercise may have a relieving, aggravating, or no effect upon menstrual pain in different women, it was proscribed for 24 hours prior to treatment in the pilot study. No such provision could be found in the full-scale trial publication. It is possible that exercise within a 24-hour period of treatment in an unknown number of participants confounded the results in the later study.
To summarize, the recent full-scale clinical trial fails to distinguish between two possible alternatives: (1) a sham procedure that is an inadequate placebo treatment; and (2) the possibility that the side-posture adjustments administered in this investigation were not effective for relieving pain or lowering prostaglandin levels in women with primary dysmenorrhea. Our hope is that further research will help in distinguishing between these alternatives in the very near future.
References:
- Hondras MA, Long CR, Brennan PC. Spinal manipulative therapy vs. a low force mimic maneuver for women with primary dysmenorrhea: a randomized, observer-blinded, clinical trial. Pain 1999, in press.
- Kokjohn K, Schmid DM, Triano JJ, Brennan PC. The effect of spinal manipulation on pain and prostaglandin levels in women with primary dysmenorrhea. Journal of Manipulative and Physiological Therapeutics 1992;15(5):279-285.
