On the eve of Thanksgiving (November 19, 1999), one would assume that another Halloween season is now safely behind us. Then again, maybe not, considering what some of the most recent medical journals have to say about that holy grail of evidence-based medicine: the randomized clinical trial. Visions of ghosts, hobgoblins and skeletons that were supposed to have retired for another year are upon us once again. This is because the revelations in these journals tell us that (I) humankind's interpretation of the results of clinical trials remains far from a complete science, and (II) outcome measures are clearly susceptible to the vagaries of human nature; not only from the perspective of the human subjects tested in clinical trials, but especially from the behavior of the trial investigators as well. Some of these images border on outright skullduggery.
The first of these ghost stories appeared very recently in the Journal of the American Medical Association, a publication that I have found to be one of the most conscientious vehicles of the critique of medical practice and experimental design. In this particular study, Juni prepared a statistical review of the results of clinical trials (called a meta-analysis) that compared low molecular weight heparin (LMWH) to standard heparin preparations in their respective abilities to prevent postoperative thrombosis. He used no fewer than 25 previously used scales to distinguish between high-quality and low-quality trials. Depending upon which scales he used and whether he was assessing high or low-quality trials, he could demonstrate that LMWH either was or was not superior to standard heparin solutions.1
Therefore, depending upon which scale one uses, it is possible to obtain diametrically opposing results!
The second hair-raising tale, also appearing in JAMA, is also devoted to bizarre events that happen en route to building a meta-analysis. This paper concerns the use of pharmaceutical agents for patients with cancer complicated by neutropenia. My reaction to this reading is easy to describe if you have seen "The Blair Witch Project." At the very least, I felt as though I had just witnessed a major miscarriage of scientific integrity. Specifically, comparisons were made in this report between fluconazole (a new antifungal agent manufactured by Pfizer, Inc.) and amphotericin B (an older pharmaceutical manufactured by Bristol-Myers Squibb to eliminate fungi from the GI tract). A cursory examination of the trials was found to yield a distinct advantage to fluconazole. However, the picture began to rapidly unravel once you took a closer look at the vexing details in this work, in which we have been told so many times the devil must surely reside:
The third creepy saga also appears in JAMA and tells us of yet another obstacle to the proper preparation of meta-analyses, which are supposed to systematically review clinical trials which have been published regarding the treatment of a given clinical malady. Here we are introduced to the fine art of creating "sausage publications," which is what occurs when an author or group of authors publish virtually duplicate or highly overlapping (read: redundant) papers in separate places. What happens, understandably, is that the multiple appearances of the same or highly similar results in separate articles leads to their overrepresentation in meta-analyses.
Worse, authors who publish clones of their works often fail to inform people of their serial productions unless it is in their own best interest (as in the preparation of their CVs). They do this by omitting citations of their works and may even permute the order of the authors in order to cover their tracks. Unless the astute reviewer is aware of these fine points, he or she will cite and rate these papers as if they were independent studies, giving them more weight than is warranted.3 Specific examples of these shenanigans follow:
In truth, sample sizes may have been too small to unequivocally demonstrate an active treatment effect, known as a type II error.8 The way this situation should have been approached is best exemplified by the integrity-filled efforts of Nils Nilsson, who revisited what he suspected had been a type II error in his earlier study of the effects of manipulation in the management of cervicogenic headache.9 It was only by increasing the sample size in his later study that Nilsson was able to obtain a statistically significant improvement obtained by high-velocity manipulations.10
The fifth scary legend is what happens when the quality of clinical trials is scored by slavishly attaching arbitrary scores (themselves rarely validated!) to the numerous features of a published RCT without thinking through their consequences. Thus, numerous methodological scores such as those provided by Koes11 and others create a misleading profile of high- and low-quality studies if they place too much emphasis upon sham procedures, which we already know will seriously compromise controlled studies involving physical methods such as spinal manipulation7 if they are not true placebos. Clinical irrelevance remains a fatal flaw in the Cherkin low-back pain study published in The New England Journal of Medicine12 and which I13,14 and others15 have extensively criticized elsewhere. In other instances, the mere utterance of such terms as "blinded" or "randomized" in the title of a paper may be sufficient to glean points in the rating of clinical trials, even though such terms are never defined or qualified. The proper remedy here is to demote the trial ratings if such terms are inappropriately used.1
Other RCT fables abound. I have already presented a few with political overtones in this column previously16 and, in the interest of space, will spare those gory details for now. Suffice it to say that while the randomized clinical trial remains an important piece of the mosaic of evidence that needs to be assembled to substantiate a clinical procedure, it is not the only piece. One needs to remain vigilant in questioning the integrity of that particular piece. Armed with the stakes and garlic of insight and patience, the astute clinical researcher will hopefully be able to keep the vampires and spirits emerging from the misinterpretation of randomized clinical trials at bay. It gives a whole new dimension to the term "stakeholder"!
References
When sports chiropractors first appeared at the Olympic Games in the 1980s, it was alongside individual athletes who had experienced the benefits of chiropractic care in their training and recovery processes at home. Fast forward to Paris 2024, where chiropractic care was available in the polyclinic for all athletes, and the attitude has now evolved to recognize that “every athlete deserves access to sports chiropractic."
It is estimated that 61% of patients with CTS avoid taking surgical options due to postoperative complications and costly surgical procedures. Chiropractic care offers a comprehensive and effective treatment for carpal tunnel syndrome, addressing the condition from multiple angles. Recent studies also have unveiled a game-changing adjunct to chiropractic treatments for CTS: nerve flossing.
Some doctors thrive in a personality-based clinic and have a loyal following no matter what services or equipment they offer, but for most chiropractic offices who are trying to grow and expand, new equipment purchases help us stay relevant and continue to service our client base in the best, most up-to-date manner possible. So, regarding equipment purchasing: should you lease, get a bank loan, or pay cash?
