A study published in the journal PLoS ONE suggests that since the launch of the trials registry clinicaltrials.gov, there has been an overall decrease in reported positive trial outcomes. The law requiring development of the registry was passed in 1997, which made it mandatory for researchers from 2000 onward to report their methods and outcome measures prior to collecting trial data.
The researchers reported that of 55 large clinical trials testing heart-disease treatments, 57 percent of trials published prior to 2000 recorded positive effects from the treatments tested. After 2000, when researchers were required to report trial information on the government site, the percentage of trials reporting positive results fell to just 8 percent. According to Veronica Irvin, the study’s author and a health scientist at Oregon State University in Corvallis, these results suggest that mandatory registration of clinical trials is leading to more rigorous research.
The study focused on trials that were funded by the US National Heart, Lung and Blood Institute (NHLBI), and emphasized human randomized controlled trials. Irvin and co-author Robert Kaplan, chief science officer at the Agency for Healthcare Research and Quality in Rockville Maryland, explained that the trend could not be accounted for by changes in clinical trial methodology, or by fluctuations in industry sponsorship. They concluded that the decrease in reported positive results was due to the registration of trials with clinicaltrials.gov.
“It’s more difficult for investigators to selectively report some outcomes and exclude others,” Irvin says of the requirement for study coordinators to report their methods before commencing the trial. Neurologist Steven Novella of Yale University in New Haven Connecticut comments on the study via his NeuroLogica Blog, “Loose scientific methods are leading to a massive false positive bias in the literature.”
This study has garnered widespread attention in the science world, including the attention of a psychologist at the University of Virginia in Charlottesville and the executive director of the Center for Open Science, Brian Nosek. He expressed interest in follow-up studies and commented on that topic in an interview, “Have they all held up in subsequent research, or are they showing signs of low reproducibility?”
Though the researchers believe that the changes in reported results are due to registration of the trials, they do admit that there are other possible explanations for the reduction in reported positive effects. One such explanation is that it may be tricky to identify benefits of novel treatments, in an age of improved cardiovascular health care. The director of NHLBI’s division of cardiovascular sciences, David Gordon said “In some ways these trials may have been the proverbial low-hanging fruit. It has become increasingly difficult to improve upon these therapies.”
According to Irvin, even registration of clinical trials cannot prevent all bias and warns that even trials showing positive results should be considered with “healthy skepticism.” She commented further by saying, “Too often, the audience only reads the headline and the abstract. It is only when you take a close look at the study details — such as effect sizes and response rates — that you can judge whether a result is likely to be clinically meaningful.”
Sources:
- Registered clinical trials make positive findings vanish – http://www.nature.com/news/registered-clinical-trials-make-positive-findings-vanish-1.18181
- Registering Studies Reduces Positive Outcomes – http://theness.com/neurologicablog/index.php/registering-studies-reduces-positive-outcomes
- Kaplan, R., and Irvin, V. (2015). Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLoS ONE.
Join or login to leave a comment
JOIN LOGIN