Straw Men on Parade: When Research Findings Get Misrepresented

The simplest way to explain a straw man argument is that it is a fallacy based upon the misrepresentation of an opponent's position.¹ Like a monster drawn from Greek mythology, it can assume many forms. It may oversimplify the opponent's position or quote it out of context, stacking the deck so one's argument may run over the opponent with the crushing totality of a Panzer tank division. In its worst incarnation, the straw man misrepresentation is deliberate and only the weakest target is chosen- that target is then held to represent the totality of the opponent's being.²

Historically, the term seems to have been taken from using human practice dummies made of straw, such as used in military training and thus a cinch to knock over. It has also been referred to as a traditional fairground game in the U.K., calling it an Aunt Sally. An even more unusual origin suggests it was used to designate men who stood outside the courthouse with straw in their shoes indicating their willingness to be false witnesses; but that notion itself seems to be an urban legend, if not a straw man itself.^3-4

What is obvious and disturbing is that straw men are not just marketing ploys that we've become jaded to seeing in television commercials; they are found even in the highest echelons of peer-reviewed journals in medical literature. Straw men seem to have been particularly dominant in publications critical of alternative medical approaches, even appearing in at least one instance in a pharmaceutical company's attempt to denigrate another company.

Straw Men: Six Examples

1. In a strongly condemning chapter, Robert Houston eloquently suggests how no less prestigious a body than the National Cancer Institute at the NIH took away an anticancer approach [called antineoplaston therapy] developed by Stanislaw Burzynski, with rates of complete remission ranging from 10 percent to 40 percent, and redesigned protocols that Burzynski knew would not work to be run at the Mayo Clinic and Sloan-Kettering Institute. The NCI disowned Burzynski and did not allow him to participate in future studies. Predictably, the rate of complete remission delivered by the NCI-backed studies was a paltry 0.16 percent or just 0.4 percent the rates produced by Brzynski's original design. As Houston put it: "They want to stamp out the innovator, crucify the discoverer, while running away with his discovery to bring it out under the respectable auspices of the National Cancer Institute."⁵ This was not only a straw man, but one burned at the stake.

2. A comparison of new antifungal agents [fluconazole, developed by Pfizer, and amphotericin B, a product of Merck] was conducted over a series of clinical trials. Nearly 80 percent of the patients randomized to the amphotericin B cohort received the drug orally when it was known that the drug normally has to be administered intravenously due to its poor absorption. Quite obviously, the Pfizer product prevailed. Turns out that nearly all the authors and patients involved in these studies were funded by Pfizer. Understandably, when the authors of the study were probed about details, they ran for cover, claimed ignorance, and otherwise refused to cooperate.⁶ So, this one's not only a straw man, but one with the legs cut off.

3. A widely publicized clinical trial appearing in the The New England Journal of Medicine, considered by many the toughest journal in which to gain the publication of one's research, compared chiropractic to two other approaches: distribution of an educational booklet and application of the McKenzie method. All were found to be equally effective, such that the authors concluded in a blatant editorial comment (which far exceeded what was shown by the data): "Given the limited benefits and high costs, it seems unwise to refer all patients with low back pain for chiropractic or McKenzie therapy."⁷

In addition to stepping out of bounds with a statement such as this, the authors gave the misleading impression that the single side-posture manipulation used in this study represented all of chiropractic. In addition, the proportions of patients who had previous experience with chiropractic and who were experiencing more severe back pain seemed decidedly stacked against the chiropractic cohort, despite the fact that the statistics run on a decidedly small sample failed to reach significance. Type II errors with undersized populations are known to mask effects such as this. In this instance, we not only have another straw man, but one wearing a grass skirt.

4. In the very same journal - in fact, appearing adjacent to the Cherkin study - was a second paper that discredited chiropractic in that it was judged to be ineffective in managing childhood asthma. The only problem here was that the placebo treatment chiropractic was compared to involved physical applications to the cranial, gluteal, and scapular areas of the patients; in other words, anything but an inactive sham treatment. Understandably, both sham and chiropractic treatments delivered measurable effects; however, the authors chose to base their conclusions upon the (lack of) difference between sham and active treatments.⁸ Elsewhere, the sham treatment was shown in a massage study of proper design to significantly improve childhood asthma symptom, attitude, and lung function scores.⁹ I guess this one would have to go down as a straw man with asthma.

5. A litany of studies have criticized the nutritional component of applied kinesiology (AK) by failing to find positive results involving a variety of substances affecting muscle testing results. The problem here is that all have avoided having the patient insert any substance at all in the mouth, thus bypassing the gustatory response. Instead, patients in all these studies have simply held bottles of sugar,^10-12 sand,^10-12 sucrose,^13,14 or the toxins hydroxylamine hydrochloride,¹⁵ or even snake venom.¹⁶ You can't convince me that holding is the same as tasting.

6. Elsewhere, a randomized controlled trial which debunks AK and challenge procedures employed 68 naive volunteers from a chiropractic college. The provocative vertebral challenge applied was a standardized 4-5 kg force delivered with a pressure algometer to the lateral aspects of the T3-T12 spinous processes. The therapeutic intervention was a manual high-velocity low-amplitude adjustment; a switched-off Activator instrument was used as a sham. Reactivity of the piriformis muscle following a vertebral challenge was assessed, as was responsiveness following spinal manipulation. Because the percent of reactivity to the vertebral challenge was only 16 percent with 0 percent responsiveness to spinal manipulation, it was judged that the muscle response seemed to be only a random occurrence which was unrelated to the manipulable subluxation. The authors concluded that manual muscle testing was of questionable value for spinal screening and post-adjustive evaluation.¹⁷

The problem with this trial was that the majority of subjects investigated (60 percent) lacked pain - with only half having stiffness in the thoracic region. Challenging the normally functioning vertebra in most of these cases would necessarily lead to a negative result, such that positive tests performed in this experiment would not be expected. Furthermore, the specific vector to challenge must match the specific subluxation of the vertebra if changes in manual muscle testing are to be found and the challenge procedure to be of any use in guiding subsequent therapeutic interventions. So, with the deck stacked as it turned out to be in this experiment, one would not expect positive results. (I'm still trying to imagine how a subluxated straw man would look.)

Don't Be Fooled

Drawing from the classic computer mantra GIGO (Garbage In, Garbage Out), we need to be increasingly vigilant as to how false representations will lead to foregone, erroneous and misleading conclusions. The vetting of false information is not merely something we need to be mindful of when contemplating blogs, Web sites, or supermarket tabloids, but also when consulting some of the most trusted tomes in the literature. Straw men may never quite measure up to the horrifying visage of Freddy Kruger, but they are certainly enough to disturb one's neural circuitry.

References

1. Pirie M. How to Win Every Argument: The Use and Abuse of Logic. UK: Continuum International Publishing Group, 2007.
2. "The Straw Man Fallacy." (www.fallacyfiles.org/strawman.html).
3. "Idioms Around the World." Referenced at http://en.wikipedia.org/wiki/Straw_man.
4. Brewer EC. Dictionary of Phrase and Fable. Published 1898.
5. Houston R. In Hess D. Evaluating Alternative Cancer Therapies: A Guide to the Science and Politics of an Emerging Medical Field. New Brunswick, NJ: Rutgers University Press, 1999:134-5.
6. Johansen HK, Gotzsche PC. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. JAMA, 1999;282(18):1752-1759.
7. Cherkin DC, Deyo RA, Battie M, Street J, Barlow W. A comparison of physical therapy, chiropractic manipulation, and provision of an educational booklet for the treatment of patients with low back pain. New Eng J of Med, 1998;339(15):1021-9.
8. Balon J, Akder PD, Crowther ER, Danielson C et al. A comparison of active and simulated chiropractic manipulation as adjunctive treatment for childhood asthma. New Eng J of Med, 1998;339(15):1013-20.
9. Field T, Henteleff T, Hernandez M, Martinez E et al. Children with asthma have improved pulmonary functions after massage therapy. Journal of Pediatrics, 1998;32(5):854-8.
10. Radin DI. A possible proximity effect on human grip strength. Percept Mot Skills, 1984;58(3):887-8.
11. Quintanar AF, Hill TV. Sugar proximity and human grip strength. Percept Mot Skills, 1988;67(3):853-4.
12. Brand WG. A possible proximity effect on human grip strength: an attempted replication. Percep Mot Skills, 1989;68(1):157-8.
13. Arnett MG, Friedenberg J, Kendler BS. Double-blind study of possible proximity effect of sucrose on skeletal muscle strength. Percept Mot Skills, 1999;89(3 Pt. 1):966-8.
14. Kendler BS, Keating TM. Re-evaluation of a possible proximity effect on muscular strength. Percept Mot Skills, 2003;97(2):371-4.
15. Schwartz SA, Utts J, Spottiswoode JP, Shade CW, Tully L, Morris WF, Nachman G. A study to assess the validity of applied kinesiology (AK) as a diagnostic tool and as a nonlocal proximity effect. JAMA, 2009. Submitted for publication.
16. Ludtke R, Kunz B, Seeber N, Ring J. Test-retest reliability and validity of the kinesiology muscle test. Complementary Therapies in Medicine, 2001;9(3):141-5.
17. Haas M, Peterson D, Hoyer D, Ross G. Muscle testing response to vertebral challenge and spinal manipulation: a randomized controlled trial of construct validity. JMPT, 1994;17(3):141-8.