Intellectual Honesty

We read with interest the recent article by Scott Cuthbert, DC, in Dynamic Chiropractic.¹ At first glance, one might get the impression that the applied kinesiology (AK) movement had produced some original research that validates their unique version of manual muscle testing. Dr. Cuthbert starts off his article with the statement that he is reviewing "a landmark study presenting the basic science and clinical research evidence for the reliability and validity of the manual muscle test."

It's interesting that the "landmark study" to which he refers actually is a literature review he co-authored and published in the BioMed Central journal Chiropractic and Osteopathy.² It's rather presumptuous to write an article for DC reviewing and praising his own work as a "landmark study." Calling a literature review a "study" is an exaggeration. History, not an author himself, labels a publication as a "landmark." One cannot even imagine Newton labeling his Principia as a landmark. Providing such aggrandizing labels does little more than deceive the casual reader and seems to us to lack intellectual honesty.

There are numerous methodological flaws in the literature review by Cuthbert and Goodheart, and we were pleased to see that a commentary by Haas, et al., already has been published in the same journal.³ We would refer the comprehensive reader to the full critique article published by Haas, et al., regarding the serious problems with the poorly implemented literature review published by Cuthbert and Goodheart. Most of the BioMed Central electronic journals are "Open Access," meaning they are immediately and permanently available online without charge. To view the full text articles by Cuthbert and Goodheart, and the response article by Hass, Cooperstein and Peterson, visit www.chiroandosteo.com.

We are writing this article to briefly outline the major misinterpretations of the literature and unsubstantiated statements made by Cuthbert in his Dynamic Chiropractic article.¹ Cuthbert and Goodheart² did not perform any original research. It appears they simply reviewed the existing literature, looking for research articles that supported their notion that manual muscle testing was reliable and valid. While a review of the literature is an important and necessary step in advancing our understanding of the research literature, authors must ensure that their conclusions are appropriate and unbiased and based upon all the existing literature. Cuthbert and Goodheart did none of this. Trying to label a literature review as a study appears to be intellectually dishonest, unless Cuthbert is just ignorant of the proper terminology.

There is a huge difference between the appropriate uses of the standard manual muscle test (SMMT) and the uses of the applied kinesiology manual muscle test (AKMMT). Cuthbert tries to interchange the uses of these two types of muscle testing as if they are equivalent procedures, but this is intellectually dishonest. He cites studies that show excellent inter-examiner reliability for the SMMT, generally for specific neurological conditions resulting in muscle weakness. The SMMT, a standard component of an orthopedic/neurological examination, is measured on a five-point scale originally developed by Kendall, et al.⁴ This scale has been shown to be a reliable method to document the amount of muscle weakness, atrophy or paralysis in patients with serious neuromuscular diseases such as polio, stroke, cerebral palsy, multiple sclerosis and spinal-cord injuries.

The use of manual muscle testing for applied kinesiology (AK) does not follow this five-point scale protocol, and instead uses a simple two-point scale: "weak" or "strong." More importantly, AK muscle testing is not used to determine atrophy or serious muscle weakness in neuromuscular diseases. AK practitioners use their two-point version of muscle-strength testing to determine organic disease, "functional" disorders, subluxation, etc. Substantial scientific literature does not exist to support the use of the AK version of manual muscle testing for these "functional" conditions.⁵

It's scientifically disingenuous to use the literature that supports SMMT to claim it validates AKMMT. This is akin to saying that since a thermometer immersed in water can reliably and with great validity measure the water temperature, that one can reliably use the same thermometer to measure the temperature and sugar concentration of that water without immersion of the thermometer in the water. Both measurements are done with thermometers but with vastly different methods and interpretations. A review of all the research currently available on manual muscle testing says absolutely nothing about the ability of an AK practitioner to diagnose any condition other than what those using the muscle testing in the studies reviewed were measuring: muscle strength.

Cuthbert¹ makes the statement that "to provide the strongest evidence for the use of chiropractic MMT techniques, more randomized controlled clinical trials (RCTs) and systematic reviews like this one ... will be essential." It appears he does not understand that the RCT design is primarily for testing treatment effectiveness and is not necessary for examining the question of reliability or validity. The strongest evidence to support the use of AKMMT would be systematic reviews of level 1 evidence on validity. Cohort studies are one type of level 1 evidence of diagnostic validity.⁶

Simply, a validity study examines whether a diagnostic test actually measures what it purports to measure by comparison to a gold standard. Sometimes, prior to conducting validity studies (and when there is a lack of a gold standard), studies of reliability are conducted. Reliability studies determine if the diagnostic test gives the same answer when applied repeatedly. A diagnostic test that is not reliable cannot typically be considered valid.

Several inter-examiner reliability studies have been performed on the AKMMT, and they show poor results. In their critique article, Haas, et al.,³ have cited numerous references that show poor inter-examiner reliability for the AKMMT, and again, we refer the serious reader to their paper for those references.

In addition, Cuthbert and Goodheart² did not perform a systematic review of the literature.¹ In fact, their literature review was not "systematic" because of their failure to adequately and comprehensively search, then rate and critically review the quality of the articles they retrieved. Haas, et al,³ point out in their critique article that Cuthbert and Goodheart missed a number of important citations in their flawed literature search, including a number of chiropractic studies by several authors, including LeBoeuf, Triano and Haas, that showed poor inter-examiner reliability for the AKMMT.

Although it seems impressive that 100-plus studies exist on manual muscle testing, it appears that Cuthbert and Goodheart simply went online and found papers with the term muscle testing in the title and saved those studies which looked interesting to them. They admit to only saving articles that showed high inter-examiner reliability by virtue of having a Kappa value of >.50. This is circuitous logic. By omitting those studies which showed poor or low inter-examiner reliability, they were left only with studies showing good or high reliability. It is therefore no surprise that their cherry-picking of the literature resulted in only high reliability studies! We question the intellectual honesty of writing that a diagnostic method is reliable when one systematically ignores every paper that refutes that assertion. Imagine a jury never being allowed to see exculpatory evidence, even if the preponderance of the evidence was in the accused favor. Would it be a surprise that the jury found the accused guilty?

In conclusion, it's disturbing to see such a misleading title that suggests Cuthbert and Goodheart have "risen to the challenge of validating their work." This is simply not true and, in our opinion, is intellectually dishonest. In order to validate AK, they actually would have to produce their own original research, such as inter-examiner reliability studies that use applied kinesiology muscle-testing procedures on patients with the functional disorders they purport to be capable of detecting. They need to provide the evidence that the unique version of manual muscle testing used with their system of applied kinesiology is reliable, and then proceed to show that this reliable testing procedure also is clinically valid. In order to show validity, a reliable diagnostic testing procedure must be compared to some other sort of gold-standard test.

We perceive cherry-picking the physical therapy and rehabilitation literature on standard, manual muscle testing and inappropriately trying to pawn this off as proof of the reliability and validity of the applied kinesiology version of manual muscle testing as completely intellectually dishonest. We look forward to the day that we can read the results of some well-designed inter-examiner reliability and validity studies produced by the applied kinesiology community, rather than an intellectually dishonest attempt to claim to have validated AK though a poorly done literature review.

References

1. Cuthbert SC. Chiropractic muscle testers rise to the challenge of validating their work. Dynamic Chiropractic, 2007;25(18).
2. Cuthbert SC, Goodheart GJ. On the reliability and validity of manual muscle testing: a literature review. Chiropr Osteopat, March 6, 2007;15:4.
3. Haas M, Cooperstein R, Peterson D. Disentangling manual muscle testing and Applied Kinesiology: critique and reinterpretation of a literature review. Chiropr Osteopat, Aug. 23, 2007;15(1):11.
4. Kendall FP, McCreary E, Provance PG. Muscles: Testing and Function. Baltimore: Williams & Wilkins, 1993.
5. Cooperstein R, Gleberzon B. Technique Systems in Chiropractic. Edinburgh: Churchill-Livingston, 2004.
6. Centre for Evidence Based Medicine. Levels of Evidence. www.cebm.net/index.aspx?o=1025.

Dr. Michael Schneider has been in private chiropractic practice for 25 years and currently is a PhD candidate in rehabilitation science at the University of Pittsburgh.

Dr. Jeffrey Hebert is a PhD student at the University of Utah. Additionally, he is a practicing chiropractor and diplomate of the American Chiropractic Board of Sports Physicians.

Dr. John Ventura has been in private practice for 23 years, is clinical instructor in family medicine at University of Rochester School of Medicine and assistant clinical professor at NYCC.

Dr. Warren Jahn has been in private practice for 30 years, attained his master's degree concentrating in biomechanical trauma and is on the editorial board for the Journal of Chiropractic Medicine.