Beware the Lure of Early Clinical Trials Results: More Than a Third May be Exaggerated

It’s true that no matter how promising a new drug appears to be in early clinical trials, a large percentage will never make it past the all-important Phase III. Now, researchers at the Mayo Clinic have given healthcare professionals another reason to take reports of early clinical trials results with a grain of salt; they found that positive results in over one in three studies were exaggerated.

“This phenomenon of exaggerated early results was present in a whopping 37 percent of the studies we reviewed,” said Dr. Fares Alahdab, lead author of the study published in Mayo Clinic Proceedings and a research fellow in Mayo Clinic’s Evidence-Based Practice Center. “Physicians and patients should be cautious about new or early clinical trial evidence. Exaggerated results could lead to false hope as well as possibly harmful effects.”

To come to this conclusion, Alahdab and his colleagues sifted through thousands of research articles published in the top 10 medical journals with the highest impact factors. These impact factors take into account the number of times articles published in that journal are cited in other works; those with the highest impact factors have the largest potential reach when it comes to disseminating information and informing future research.

Included in their work were 70 meta-analysis articles which encompassed results from over 900 different clinical trials. The articles were published between 2007 and 2015.

The research team focused on analyzing study results from trials of chronic disease treatments for conditions such as stroke, heart disease, diabetes and cancer. They found that results published from preliminary trials were 2.67 times larger than those from subsequent trials.

According to Dr. M. Hassan Murad, lead researcher and the director of the Mayo Clinic’s Evidence-Based Practice Center, exaggerating trial results – whether intentional or not – can have significant negative effects on patients, particularly those with comorbidities.

“Often, patients are living with more than one chronic condition, and they and their doctors watch for research about new treatments,” said Murad. “They need to be aware that the effect seen in earlier trials may not bear out over time and may be much more modest.”

Murad, Alahdab and colleagues hope their findings will encourage future researchers to practice cautious optimism when writing about and publishing results from early-stage clinical trials.

“Some people may think this is an anti-innovation message,” said Murad. “To the contrary, we welcome new treatments. We just want people to know that the benefit seen in real practice, when treatments are given to people with various comorbidities and in different settings, may be smaller than what was seen in the earliest clinical trials.”