Stephen R. Seater
Americans crave news of clinical research that they believe will guide them to live healthier lifestyles or to pursue more effective medical treatments. But how reliable are many of the studies being reported in the media? And, for that matter, is the media doing a good job reporting these results? Health conscious Americans are increasingly finding themselves confused by all sorts of contradictory research findings, say Marcia Angell, MD, and Jerome Kassier, MD, in a recent article in the New England Journal of Medicine.
It isn't hard to see why confusion reigns among the public. First they were told to substitute margarine for butter, only to hear that margarine may actually be worse for their arteries. Vitamin E and beta carotene, long said to prevent cancer, were recently found to be not better and perhaps worse than placebo.
Just a few weeks ago, two new studies were published on calcium channel blockers. One said that these drugs were harmless and should continue to be used; the other said the drugs may cause cancer. A recent meta-analysis of clinical trials done on the effectiveness of garlic in reducing blood levels of cholesterol showed positive effects for both garlic powder and raw garlic. However, a new study completed at Oxford University challenged these findings, and the researchers said that the other garlic trials were methodologically flawed. And so it goes.
Some writers have blamed the media for this state of affairs, saying that research results are often not reported accurately. Other experts point out it is the fault of both the media and the public who are scientific imbeciles, making the point that only rarely can a single study be definitive. This is true because most, if not all studies, contain some biases in the methodology. According to a paper by Williamson et al., only about one percent of all the articles in medical journals are scientifically sound. If true, this is an incredible indictment of so-called medical research.
According to Douglas G. Altman of the Medical Statistics Laboratory, Imperial Cancer Research Fund in London: "We need less research, better research, research done for the right reasons. Huge sums of money are spent annually on research that is seriously flawed." The reason for this, Altman argues, is that people in medicine and medical educators feel compelled to publish papers. "The effects of the 'publish or perish' mentality can be most easily seen in the increase in scientific fraud. Moreover, many medical researchers are ignorant of statistics and experimental design." Even though, "the poor quality of much medical research is widely acknowledged, yet disturbingly the leaders of the medical profession seem only minimally concerned about the problem."
It appears the fault lies with the scientists as well as the public and the media. We cannot expect the public at large to become much more sophisticated about clinical research, especially something as esoteric as epidemiology. At best, it might be possible through improved science education in our schools to educate the public enough about the scientific method to know that there are few breakthroughs in clinical research. In general, it might be possible to educate the public enough so that only after many studies have reached the same or very similar conclusions should anyone embrace a particular finding as valid.
Scientists may actually be more to blame for the present state of confusion than either the media or the public. If so, little of what is published in the medical literature is scientifically sound as Dr. Williamson et al. If publishing for the sake of publishing is the main reason medical scientists submit papers to journals, as Dr. Altman says, then no wonder there is so much confusion and uncertainty. Perhaps it is the medical researchers who need to be better trained in statistics and experimental design before attempting to conduct research. Perhaps the journals themselves need to be much more rigorous in reviewing the manuscripts being submitted.
There is no question that well designed clinical research is very important to both medicine and chiropractic. FCER has made a big commitment to funding clinical research, and has also instituted a very tough review system. For one thing, there are no preconceived biases built into the system. In other words, FCER is not committed to funding only research carried out by chiropractors or research proposals coming from chiropractic colleges. Our commitment is to fund the best research that we believe will help the chiropractic profession. It so happens that much of that research does come from the chiropractic colleges, but not all of it by any means.
Every study that is proposed to FCER for funding must first be reviewed by Dr. Tony Rosner. As FCER's research director, he sends the proposals to a committee of practicing chiropractors for a determination of the proposed research's relevance to clinical practice. Normally to be considered further, proposed studies must have a strong relationship to practice. Dr. Rosner also sends proposals to a scientific review committee, made up of outstanding researchers from all over the world. These people include well known biostatisticians, epidemiologists, and other highly specialized individuals. To be funded, each proposed study must receive at least a grade of 6 points out of a possible 7 points. Many studies which show promise, but lack scientific rigor, are sent back for revisions before they can be funded.
After reviewing FCER's approach to funding proposed research, Harvard's David Eisenberg, MD, told Dr. Rosner that he had never seen such an exacting and demanding process, including that employed by the National Institutes of Health. Proof that FCER funds only the best research came in the form of an article recently published in Spine. The authors ranked an FCER funded headache trial as tops in quality out of a large field of similar studies done on cervical manipulation.
For more information about FCER's research program, write Dr. Anthony Rosner, director of research, FCER, 1701 Clarendon Blvd., Arlington, VA, 22209 or call (703) 276-7445.
Stephen Seater, CAE
Arlington, Virginia
