Special Interest and Research: Daubert or Dilbert?

A dozen years ago, the U.S. Supreme Court handed down a ruling that worries the scientific purists in our ranks more than just a bit. Daubert v Merrell Dow Pharmaceuticals, Inc.¹ directed federal judges to be the gatekeepers of evidence, deciding whether to permit expert evidence to be presented to a jury. This meant judges were expected to review the scientific method underlying expert evidence and to then admit what was scientifically reliable and relevant to the issue at hand.

While it may be easy for virtually any individual to debunk concepts that suggest the world is flat or phlogiston is the means by which heat travels, most scientific principles are extremely complex, with many conflicting lines of evidence. Witness, for example, the recent and highly visible announcement that even the most fastidious and orthodox of randomized clinical trials are prone to yielding contradictory results over time.² Put in simpler terms: What justice would ever be able to deliver a clear and definitive answer as to whether hormone replacement therapy or mammograms are universally indicated for women's health? Given that our Supreme Court recently decided our current president by dividing along the lines of their respective political party affiliations, the implications of the Daubert ruling seem more like the tribulations of Dilbert every day.

Daubert v Merrell Dow Pharmaceuticals, Inc., addressed the question of whether Bendectin could cause birth defects in humans. Despite the facts that (1) expert testimony, based upon both live animal and in vitro studies existed; (2) pharmacological similarities between Bendectin and other substances known to cause birth defects existed; and (3) negative but unpublished epidemiological studies of Bendectin were reported, none of this evidence appeared to directly address causality. Therefore, it was deemed inadmissable.³ Lacking guidelines as to how to clearly establish scientific validity, judges were clearly left to their own values and preconceptions to make decisions,⁴ which now has become a legal precedent.

Does this create an opportunity for special interest groups to gain access to our system of scientific inquiry? Absolutely. Consider a study published in the Lancet five years ago, which revealed a clear trend of research results tied to the source of funding: If the investigation was supported by a for-profit source, as opposed to nonprofit, the percentage of trials that favored a new treatment increased from 47% to 74%, while the corresponding figure of trials that revealed inferior controls rose from 21% to 60%.⁵ That finding was echoed three years later by an observational study of some 370 randomized drug trials, which indicated that the experimental drug was recommended in 16% of trials funded by nonprofit sources and 51% of trials supported by for-profit organizations.⁶ Not a pretty picture at all.

This controlling of results reached the level of outright fudging (some would say vaudeville) in the comparisons of fluconazole and amphotericin B trials that were conducted mostly under the funding of Pfizer, as I have described previously.⁷ Briefly, amphotericin B was administered inappropriately in most clinical trials, so that its capacity to be effective in comparisons with Pfizer's own product (fluconazole) was fatally compromised from the get-go; yet all of these results got past peer reviewers and landed in the indexed literature. The punch line of this research is that the results of 92% of all patients were generated by Pfizer funding.⁸

If this sort of hijinx can exist in the pharmaceutical industry, what about public safety? The recent problems with Vioxx and Celebrex, and their tenuous positions on the market, should provide a sobering answer.⁹ These excesses have gone so far as to drive the House of Commons Health Committee in the United Kingdom to produce a report concerning the overall pervasiveness of the pharmaceutical industry. Among other items, the report (1) called for the implementation of an effective regulatory regime; (2) described the corruptions of clinical trial design, which were intended to show a drug to its best advantage and suppress results concerning risks; and (3) lay the blame at the feet of regulators, prescribers and industry. But the wheels appeared to come spinning off the wagon with this incredibly revealing statement from the report:

"Pharmaceutical companies will inevitably continue to be the dominant influence in deciding what research is undertaken and conducting that research, publishing it and providing information to prescribers."¹⁰

Sez who? Is this the fox guarding the chickens, or what? To paraphrase that Beach Boys classic, Daddy has just handed the T-bird keys back to the joyriders and they will without a doubt have fun, fun, fun. Until this premise is challenged and overturned, very little progress will be evident and this entire report will have lost much of its effect, to say nothing of its credibility.

Problems of declining research integrity are nothing new. In mid-August 2000, the NIH sponsored a special conference of some 700 to discuss financial conflicts of interest in research. Said Greg Koski, director of the Office of Human Protection: "Conflicts of interest are very real, very serious and a threat to our entire endeavor. During the last five years, they may have gotten out of control. Public trust has been eroded."¹¹

Well, exactly. In the United States, more than 60,000 human experiments have been reported by a Boston organization called CenterWatch, costing over $7 billion. Even more troubling is the report from CenterWatch, which reveals that in 1991, some 80% of studies were conducted at nonprofit academic medical centers. Nine years later, that figure has been cut in half, with the balance shifting to for-profit companies. The opportunities for abuse have never been higher. And here is the coup de grace - the cherry on the ice cream sundae: A more recent piece of dubious legislation, The Data Quality Act of 2000, has provided yet another means for special interest groups to challenge the value of scientific information used by federal agencies for making regulatory decisions.¹²

Fortunately, some of these violations can be brought to light if a detailed comparison is made of an original research plan with what is ultimately reported in the journals. In an exhaustive and groundbreaking study, Chan and co-authors indicated that deficiencies in outcome reporting pose a threat to the reliability of the entire body of literature pertaining to randomized trials. Their recommendations are simple, to the point, and fortunately, appear to be in the process of being adopted on a broad scale:¹³

1. Protocols should be made publicly available to identify both unreported outcomes and post hoc amendments.
2. Deviations from trial protocols should be described in articles published.
3. Journal editors should demand both original protocols and any amendments to be submitted for review together with manuscripts.

Under these rules, there is some hope that the ethics of clinical research can find their way back to the rails and adhere more closely to the tenets of scientific inquiry and debate as we once envisioned them. Together with what we have attempted to provide as a research program at FCER, we expect that most, if not all of these problems will not become hallmarks of the chiropractic research that we support.

References

1. Daubert v Merrell Dow Pharmaceuticals, Inc., 509 US 579 [1993].
2. Ioaniidis JPA, Schmid CH. Summing up evidence: one answer is not enough. Lancet 1998;351:123-127.
3. Melnick RL. A Daubert motion: legal strategy to exclude essential scientific evidence in toxic tort litigation. American Journal of Public Health 2005;95(S1):S30-S34.
4. Cecil JS. Ten years of political gatekeeping under Daubert. American Journal of Public Health 2005;95(S1):S74-S80.
5. Djulbegovic B, Lacevic M, Cantor A, et al. The uncertainty principle and industry-sponsored research. Lancet 2000;356:635-638.
6. Nielson B-A, Chen W, Gluud C, Kjargard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment or adverse events? Journal of the American Medical Association 2003;290(7):921-928.
7. Rosner A. Fables of foibles: inherent problems with RCTs. Journal of Manipulative and Physiological Therapeutics 2003;26(7):460-467.
8. Johansen HK, Gotzsche PC. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. Journal of the American Medical Association 1999;282(18):1752-1759.
9. Couzin J. Withdrawal of Vioxx casts a shadow over COX-2 inhibitors. Science 2004;306(5695):384-385.
10. House of Commons Health Committee. The Influence of the Pharmaceutical Industry. Fourth Report of the Season 2004-2005. Volume I. Ordered by the House of Commons to be printed 22 March 2005.
11. Dembner A. Research integrity declines. Boston Globe, Aug. 22, 2000: p. E1.
12. Wagner W. The perils of relying on interested parties to evaluate scientific quality. American Journal of Public Health 2005;95(S1):S99-S106.
13. Chan A-W, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Journal of the American Medical Association 2004;291(20):2457-2465.

Anthony Rosner, PhD
Brookline, Massachusetts
rosnerfcer@aol.com