An alternative statistical method could improve clinical trials


Credit: public domain CC0

An alternative statistical method perfected and advanced by Cornell researchers can make clinical trials more reliable and trustworthy while helping to address what has been called a “replicability crisis” in the scientific community.

In a new article published this month in the Proceedings of the National Academy of Sciences, Cornell researchers developed the “Fragility Index,” a method that is gaining traction in the medical community as an adjunct to p-value, a measure of probability applied across science since the 1920s and cited, sometimes recklessly, as proof of solid results.

“Clinicians are confident that the procedures and protocols they apply are informed by sound clinical trials. Anything less makes surgeons nervous, and rightly so,” said Martin Wells, Charles A. Alexander Professor of Statistical Sciences and co-author of the paper. “We find that many of these back-to-back trials that have shown promising results and have been published in top journals are flimsy. It’s a disconcerting surprise that has come out of this research.”

The article, written by statisticians from Cornell and physicians from Weill Cornell Medicine and the University of Toronto, offers a new statistical toolkit using the frailty index as an alternative method to help researchers better determine whether Their test results are, in fact, solid and reliable. or simply a product of chance.

“When you tell the world that a treatment should or should not be used, you want that decision to be based on reliable results, not results that can swing one way or the other depending on the results. one or two patients,” said Benjamin. Baer, ​​Ph.D. ’21, co-author of a paper and currently a postdoctoral researcher at the University of Rochester. “Such results can be considered fragile.”

Randomized clinical trials to test effectiveness are essential for surgical procedures and medical treatments. To interpret the statistical significance of trial results, researchers have for decades relied on an often misunderstood measure, the p-value, to determine whether results have merit or are simply the result of chance.

But skepticism surrounding the reliability of the p-value, when used alone and without supporting methods, has grown over the past 15 years, especially since earlier trial results initially thought to be robust have failed. be replicated in follow-up trials. In a 2014 study using the Frailty Index, researchers analyzed 400 randomized clinical trials and found that 1 in 4 trials with “statistically significant” p-values ​​actually had alarming frailty scores, indicating lower outcomes. reliable.

“You can see why there’s a replication crisis in science. Researchers find good results, but they don’t hold up,” Wells said. “These are large, serious trials that investigate cutting-edge questions, with results published in top journals. And yet, some of these large trials have low indices of fragility, which raises the question of the reliability of the results.”

With their latest research, Cornell researchers offer a solution by refining the frailty index, which studies the number of patient outcomes that could tip a trial over whether it was successful or not. The lower the fragility number, the more fragile and unreliable the results. For example, a trial with 1,000 participants that is statistically significant or insignificant based on the results of a few patient outcomes has an extremely low frailty index.

Since its appearance in the 1990s, the frailty index has been criticized for its rigidity – it is only applicable for data with two study groups, treatment and control, and a binary outcome, event or not. This latest research proposes a more flexible frailty index that can be applied to any type of outcome and any number of explanatory variables.

The team’s method also gives researchers from all scientific fields the ability to calculate the frailty index based on the likelihood of particular outcomes.

“The traditional framework of statistical significance in terms of yes-no is overly simplistic, and the issues we study are not,” said Dr. Mary Charlson, William Foley Professor Emeritus of Medicine at Weill Cornell Medical College and a co-authored article. “With each clinical situation, you are dealing with different contexts. This method allows us to test hypotheses and examine the implications of a much narrower range of results.”

New research raises concerns about bias in clinical trials due to undisclosed censorship

More information:
Benjamin R. Baer et al, Fragility Indices Only for Sufficiently Probable Changes, Proceedings of the National Academy of Sciences (2021). DOI: 10.1073/pnas.2105254118

Michael Walsh et al, The statistical significance of the results of randomized controlled trials is often fragile: a case for a frailty index, Journal of Clinical Epidemiology (2014). DOI: 10.1016/j.jclinepi.2013.10.019

Provided by Cornell University

Quote: Alternative Statistical Method Could Improve Clinical Trials (2021, December 21) Retrieved January 28, 2022 from

This document is subject to copyright. Except for fair use for purposes of private study or research, no part may be reproduced without written permission. The content is provided for information only.


About Author

Comments are closed.