Levels of Evidence: I Heard It Through the Grapevine

Whenever we refer to the "grapevine" as a source of information, we customarily associate it with the most tenuous stream of information, perhaps just steps away from the urban legend. Funny, because "grapevine" also refers to one of the most ironclad, choking holds found in wrestling. So, which is it? Levels of evidence have long held sway with the clinical community in health care. The traditional pyramidal hierarchy of evidence-based medicine, topped off by systematic reviews and meta-analyses, has commonly been referred to as the gold standard, but recently has come under fire for omissions and flaws.¹

Regardless of its drawbacks, the apex of the pyramid often has become the basis for writing clinical guidelines that one would suspect – in a perfect world – to be followed religiously. Yet the medical literature abounds with reports in which guidelines for such varied conditions as hypertension,² multiple sclerosis,³ bipolar disorder,⁴ pregnancy weight gain,⁵ heart failure,⁶ cervical⁷ and prostatic⁸ cancer screening, and even lower back pain⁹ are likely to be ignored in daily practice.

But guidelines get treated even more badly; they get hammered, in fact. You probably remember some of the lurid details. The celebrated AHCPR (Agency for Health Care Policy and Research) guidelines on low back pain, released in 1994 and, among other things, so supportive of chiropractic care¹⁰ were so vilified that both the Senate and House Budget Committees zeroed out the allocation for the entire health care agency, and Representative Sam Johnson of Texas introduced a floor amendment to kill the appropriation for the AHCPR.¹¹ It was only by dint of shifting the mission of the agency, removing the word policy from the agency's name, and rebuilding a coalition supporting what was subsequently called the AHRQ (Agency for Health Research and Quality) that the agency was saved.^12-13

Let us return to the base of the pyramid for a moment. Descending beneath the so-called lowly levels of the case study and anecdote, one is tempted to add, in order of decreasing rank: urban legends, old wives' tales and rumors. Think, for a moment, how such urban legends as (1) the demise of Paul McCartney; (2) the residence of alligators in sewers; (3) the placing of poodles in microwave ovens; (4) fatal hairdos; (5) the Neiman-Marcus cookie recipe; and (6) flesh-eating bananas have taken hold of the public imagination.

On the health care side, we have experienced such will-'o-wisps as the orgone box and arguably colonics and laetrile, as well as the popular conception that ulcers were the product of dominant mothers and passive fathers.¹⁴ And diets? Try Scarsdale, South Beach, Atkins, Gerson, Hollywood, Beverly Hills, or zone for starters. Where's the evidence?

One solution to at least some conditions – if not the ones expressed immediately above – may lie in the cornucopia of case studies, some of which are actually elegantly conducted and serve as excellent templates from which further studies of this nature may be designed.¹⁵ Of greater impact is the inescapable fact that a number of medical milestones larger than one might imagine are solely based upon case studies. These would include (1) the psychoanalytic methods of Freud;¹⁶ (2) the use of anesthesia;¹⁷ (3) the identifications of Addison's disease,¹⁸ Paget's disease,¹⁹ and Cushing's disease;²⁰ (4) the treatment of empyema with intrapeural fibronolytics;²¹ (5) the practice of heart transplants;²² (6) and the identification of acquired immunodeficiency syndrome/HIV from an outbreak of Kaposi's sarcoma in a public bathhouse in San Francisco.²³

Important to remember is the fact that, over the past 20 years, the quality of observational (i.e., cohort and case) studies has been found to improve such that their predictive value could now be compared to that seen in the more fastidious randomized, controlled trials.^24-25

These are all reasons why such compelling case series in the treatment of otitis media by chiropractic, authored by Froehle²⁶ and Fallon²⁷ in the face of excessive and often absurd application of antibiotics²⁸ and tympanostomy tubes²⁹ in more conventional health care venues for this same condition, have to be regarded with more common sense. This line of reasoning also sets the stage for considering more seriously how alternative management approaches as applied kinesiology may have some value in managing even such recalcitrant conditions as asthma.³⁰

This virtual inversion of the pyramid as to what practices emanate from which level of evidence has led epidemiologists and others to run for cover and reform and redesign the schematic as to what constitutes a more realistic portrayal of the types of approaches which offer the greatest validation of clinical practice. Not only has the traditional EBM pyramid been transfigured into the so-called "evidence house,"³¹ but one of the authors (Wayne Jonas) has derived a concept derived from the social sciences that acknowledges the multiplicity of methods available in clinical research, recognizing the strengths and weaknesses of each and offering optimal methods for answering specific questions. Thus, with this presentation, we have now can say that we've witnessed the progression of schematics in EBM from a pyramid to a house to a circle, as shown in the image.³²

Speaking of circular concepts, we can now return to the topic of grapevines and suggest that, since the lower forms of evidence discussed here now seem to have been increasingly accepted and even embraced by academic entities, perhaps the distance between the ivy-covered institutions and grapevines isn't so great after all.

References

1. Rosner A. Evidence-based medicine: revisiting the pyramid of priorities. Journal of Bodywork and Movement Therapies, 2012;16(1):42-49.
2. Okechukwu I, Mahmud A, Bennett K, Feely J. Choice of first antihypertensive - are existing guidelines ignored? British Journal of Pharmacology, 2007;64(6):722-726.
3. Hawkes N. NICE guidelines on multiple sclerosis are being ignored. British Medical Journal, 2011;343:d6525.
4. McAllister-Williams RH, Watson S. Bipolar disorder ignored by the Mental Health National Service framework but not forgotten by the British Association for Psychopharmacology. Journal of Psychopharmacology, 2003;14(4 Suppl):7-10.
5. Cockey CD. Pregnancy weight gain guidelines ignored. AWHONN Lifelines, 2005;9(3):211-213.
6. "Heart Failure Treatment Guidelines Often Ignored. But Medical Centers With Highest Adherence Had Shorter Lengths of Stay and Lower Mortality Rates." Health News, 2005;11(12):8-9.
7. Roland KB, Soman A, Benard VB, Seraiya M. Human papilloma virus and Papanicolaou tests screening individual recommendations in the United States. Journal of American Obstetrics and Gynecology, 2011;205(5):447:e1-e8.
8. Prasad SM, Drazer MW, Huo D, Ju JC, Eggener SE. 2008 US Preventive Services Task Force recommendations and prostatic cancer screening rules. Journal of the American Medical Association, 2012;307(16):1692-1694.
9. "Guidelines for Lower Back Pain Treatment Ignored. For Patients With Low Back Pain, the Result Is Higher Risk for Adverse Results and Higher Costs." Duke Medical Health News, 2010;16(5):3.
10. Bigos S, Bowyer O, Braen G, et al. Acute Low Back Pain in Adults. Clinical Practice Guideline No. 14. AHCPR Publication No. 95-0642. Rockville, MD: Agency for Health Care Policy and Research, Public Health Service, U.S. Department of Health and Human Services. December 1994.
11. Cook-Deegan R. Boosting health services research. Science, 2011;333(6048):1384-5.
12. National Research Council and Institute of Medicine. Enhancing the Vitality of the National Institutes of Health: Organizational Change to Meet New Challenges. Washington, DC: National Archive Press, 2003.
13. Centers for Medicare and Medical Services. National Health Expenditures by Type of Service and Source of Funds, CY 1960-2009. Washington, DC: Department of Health and Human Services, 2011.
14. Susser M. Causes of peptic ulcer: a selective epidemiological review. Journal of Chronic Diseases, 1967;20:435-456.
15. Liebl NA, Butler LM. A chiropractic approach to the treatment of dysmenorrheal. Journal of Manipulative and Physiological Therapeutics, 1990;13(2):101-106.
16. Pletsch C. Freud's case studies. Partisan Review, 1982;49(1):101-118.
17. Squire W. On the introduction of ether inhalation as an anaesthetic in London. The Lancet, 1888;132:1220-1221.
18. Addison T. On the Constitutional and Local Effects of Disease of the Supra-Renal Capsules. London, UK: Samuel Highley, 1855.
19. Paget J. On a form of chronic inflammation of bones [ostetis deformans]. Transactions of the Medico-Chirugical Society, 1877;60;235-256.
20. Cushing H. The Pituitary Body and Its Disorders. Philadelphia, PA: JB Lippincott Company, 1912.
21. Tillett WS, Sherry S. The effect in patients of streptococcal fibrinolysin [streptokinase] and streptococcal desoxyribonuclease on fibrinous, purulent, and sanguinous pleural exudations. Journal of Clinical Investigation, 1949;28:173-190.
22. Barnard CN. A human cardiac transplant: an interim report of a successful operation performed at Groote Schuur Hospital in Cape Town. South African Medical Journal, 1967;41:1271-1274.
23. Kaposi's sarcoma and Pneumocystitis pneumonia among homosexual men: New York City and California. Morbidity and Mortality Weekly Report, 1981;30(25):305-308.
24. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. New England Journal of Medicine, 2000;342(25):1878-1886.
25. Concato J, Nirav-Shah, Horwitz RI. Randomized, controlled trials, observational studies and the hierarchy of research designs. New England Journal of Medicine, 2000;342(25):1887-1892.
26. Froehle RM. Ear infection: A retrospective study examining improvement from chiropractic care and analyzing for influencing factors. Journal of Manipulative and Physiological Therapeutics, 1996;19(3):169-177.
27. Fallon JM. The role of the chiropractic adjustment in the care and treatment of 332 children with otitis media. Journal of Clinical Chiropractic Pediatrics, 1997;2(2):167-183.
28. Mangione-Smith R, McGlynn EA, Elliott MN, Krogstad P, Brook RH. The relationship between perceived parental expectations and pediatrician antimicrobial prescribing behavior. Pediatrics, 1999;103(4):711-718.
29. Keyhani S, Kleinman LC, Rothschild M, Bernstein JM, Anderson R, Chassin M. Overuse of tympanostomy tubes in New York metropolitan area: evidence from five hospital cohort. British Medical Journal, 2008;337:a1607.
30. Cuthbert SC. A multi-modal chiropractic treatment approach for asthma: a 10-patient retrospective case series. Chiropractic Journal of Australia, 2008;38:17-27.
31. Jonas W. The evidence house: How to build an inclusive base for complementary medicine. Western Journal of Medicine, 2001;175;79-80.
32. Walach H, Falkenburg T, Funnebo V, Lewith G, Jonas W. Circular instead of hierarchical: methodological principles for the evaluation of complex interactions. BMC Medical Research Methodology, 2006;6(1):29.