Analysis Of Non-inferiority Trials Research Articles

Non-inferiority Clinical Trials: Treating Margin as Missing Information

Non-inferiority (NI) clinical trials’ goal is to demonstrate that a new treatment is not worse than a standard of care by a certain amount called margin. The choice of non-inferiority margin is not straightforward as it depends on historical data, and clinical experts’ opinion. Knowing the “true”, objective clinical margin would be helpful for design and analysis of non-inferiority trials, but it is not possible in practice. We propose to treat non-inferiority margin as missing information. In order to recover an objective margin, we believe it is essential to conduct a survey among a group of representative clinical experts. We introduce a novel framework, where data obtained from a survey are combined with NI trial data, so that both an estimated clinically acceptable margin and its uncertainty are accounted for when claiming non-inferiority. Through simulations, we compare several methods for implementing this framework. We believe the proposed framework would lead to better informed decisions regarding new potentially non-inferior treatments and could help resolve current practical issues related to the choice of the margin.

Noninferiority Clinical Trials With Binary Outcome: Statistical Methods Used in Practice

The use of noninferiority (NI) clinical trials continues to increase. At the same time, this study design remains complex with numerous issues related to the planning, implementation, and analysis of such trials. In this systematic review, we expand previous reviews, and evaluate reporting of the statistical methods used for design and analysis of NI trials with binomial proportions. We assessed 70 NI clinical trials, published between June 2017 and May 2018. Based on these trials, we outline the gaps in statistical reporting in the published articles, and provide recommendations to minimize the gaps. The recommendations, and the awareness they raise are a helpful addition to the efforts the research community is facing today in combating reproducibility and replicability issues.

Statistical considerations in the design and analysis of non-inferiority trials with binary endpoints in the presence of non-adherence: a simulation study.

Protocol non-adherence is common and poses unique challenges in the interpretation of trial outcomes, especially in non-inferiority trials. We performed simulations of a non-inferiority trial with a time-fixed treatment and a binary endpoint in order to: i) explore the impact of various patterns of non-adherence and analysis methods on treatment effect estimates; ii) quantify the probability of claiming non-inferiority when the experimental treatment effect is actually inferior; and iii) evaluate alternative methods such as inverse probability weighting and instrumental variable estimation. We found that the probability of concluding non-inferiority when the experimental treatment is actually inferior depends on whether non-adherence is due to confounding or non-confounding factors, and the actual treatments received by the non-adherent participants. With non-adherence, intention-to-treat analysis has a higher tendency to conclude non-inferiority when the experimental treatment is actually inferior under most patterns of non-adherence. This probability of concluding non-inferiority can be increased to as high as 0.1 from 0.025 when the adherence is relatively high at 90%. The direction of bias for the per-protocol analysis depends on the directions of influence the confounders have on adherence and probability of outcome. The inverse probability weighting approach can reduce bias but will only eliminate it if all confounders can be measured without error and are appropriately adjusted for. Instrumental variable estimation overcomes this limitation and gives unbiased estimates even when confounders are not known, but typically requires large sample sizes to achieve acceptable power. Investigators need to consider patterns of non-adherence and potential confounders in trial designs. Adjusted analysis of the per-protocol population with sensitivity analyses on confounders and other approaches, such as instrumental variable estimation, should be considered when non-compliance is anticipated. We provide an online power calculator allowing for various patterns of non-adherence using the above methods.

Statistical considerations in the design and analysis of non-inferiority trials with binary endpoints in the presence of non-adherence: a simulation study.

Protocol non-adherence is common and poses unique challenges in the interpretation of trial outcomes, especially in non-inferiority trials. We performed simulations of a non-inferiority trial with a time-fixed treatment and a binary endpoint in order to: i) explore the impact of various patterns of non-adherence and analysis methods on treatment effect estimates; ii) quantify the probability of claiming non-inferiority when the experimental treatment effect is actually inferior; and iii) evaluate alternative methods such as inverse probability weighting and instrumental variable estimation. We found that the probability of concluding non-inferiority when the experimental treatment is actually inferior depends on whether non-adherence is due to confounding or non-confounding factors, and the actual treatments received by the non-adherent participants. With non-adherence, intention-to-treat analysis has a higher tendency to conclude non-inferiority when the experimental treatment is actually inferior under most patterns of non-adherence. This probability of concluding non-inferiority can be increased to as high as 0.1 from 0.025 when the adherence is relatively high at 90%. The direction of bias for the per-protocol analysis depends on the directions of influence the confounders have on adherence and probability of outcome. The inverse probability weighting approach can reduce bias but will only eliminate it if all confounders can be measured without error and are appropriately adjusted for. Instrumental variable estimation overcomes this limitation and gives unbiased estimates even when confounders are not known, but typically requires large sample sizes to achieve acceptable power. Investigators need to consider patterns of non-adherence and potential confounders in trial designs. Adjusted analysis of the per-protocol population with sensitivity analyses on confounders and other approaches, such as instrumental variable estimation, should be considered when non-compliance is anticipated. We provide an online power calculator allowing for various patterns of non-adherence using the above methods.

Statistical considerations in the design and analysis of non-inferiority trials with binary endpoints in the presence of non-adherence: a simulation study (Supplementary material)

Statistical considerations in the design and analysis of non-inferiority trials with binary endpoints in the presence of non-adherence: a simulation study (Supplementary material)

A Conservative Approach for Analysis of Noninferiority Trials With Missing Data and Subject Noncompliance

Noninferiority clinical trials aim to show an experimental treatment is therapeutically no worse than standard of care, particularly if the new treatment is preferred for reasons such as cost, convenience, safety, and so on. Noninferiority trials are by nature less conservative than superiority studies: protocol violations may increase bias toward the alternative hypothesis of noninferiority. Our objective was to compare multiple imputation, a linear mixed model, and other methods for analyzing a longitudinal trial with missing data in intention-to-treat and per-protocol populations. We simulated trials with missing data and noncompliance due to treatment inefficacy under varying trial conditions (e.g., trajectory of treatment effects, correlation between repeated measures, and missing data mechanism), assessing each approach by estimating bias, Type I error, and power. We found that multiple imputation using auxiliary data on noncompliance in the imputation model performed best. A hybrid intention-to-treat/per-protocol multiple imputation approach with a missing not at random imputation model produced low Type I error, was unbiased and maintained reasonable power to detect noninferiority. We conclude that the anti-conservatism of noninferiority trial estimands conforming with the intention-to-treat principle may be offset by imputation models that include variables on intercurrent events. Supplementary materials for this article are available online.

Design of non-inferiority randomized trials using the difference in restricted mean survival times.

Background/aims Non-inferiority trials with time-to-event outcomes are becoming increasingly common. Designing non-inferiority trials is challenging, in particular, they require very large sample sizes. We hypothesized that the difference in restricted mean survival time, an alternative to the hazard ratio, could lead to smaller required sample sizes. Methods We show how to convert a margin for the hazard ratio into a margin for the difference in restricted mean survival time and how to calculate the required sample size under a Weibull survival distribution. We systematically selected non-inferiority trials published between 2013 and 2016 in seven major journals. Based on the protocol and article of each trial, we determined the clinically relevant time horizon of interest. We reconstructed individual patient data for the primary outcome and fit a Weibull distribution to the comparator arm. We converted the margin for the hazard ratio into the margin for the difference in restricted mean survival time. We tested for non-inferiority using the difference in restricted mean survival time and hazard ratio. We determined the required sample size based on both measures, using the type I error risk and power from the original trial design. Results We included 35 trials. We found evidence of non-proportional hazards in five (14%) trials. The hazard ratio and the difference in restricted mean survival time were consistent regarding non-inferiority testing, except in one trial where the difference in restricted mean survival time led to evidence of non-inferiority while the hazard ratio did not. The median hazard ratio margin was 1.43 (Q1-Q3, 1.29-1.75). The median of the corresponding margins for the difference in restricted mean survival time was -21 days (Q1-Q3, -36 to -8) for a median time horizon of 2.0 years (Q1-Q3, 1-3 years). The required sample size according to the difference in restricted mean survival time was smaller in 71% of trials, with a median relative decrease of 8.5% (Q1-Q3, 0.4%-38.0%). Across all 35 trials, about 25,000 participants would have been spared from enrollment using the difference in restricted mean survival time compared to hazard ratio for trial design. Conclusion The margins for the hazard ratio may seem large but translate to relatively small differences in restricted mean survival time. The difference in restricted mean survival time offers meaningful interpretation and can result in considerable reductions in sample size. Restricted mean survival time-based measures should be considered more widely in the design and analysis of non-inferiority trials with time-to-event outcomes.

Bayesian methods for the design and analysis of noninferiority trials

ABSTRACTThe gold standard for evaluating treatment efficacy of a medical product is a placebo-controlled trial. However, when the use of placebo is considered to be unethical or impractical, a viable alternative for evaluating treatment efficacy is through a noninferiority (NI) study where a test treatment is compared to an active control treatment. The minimal objective of such a study is to determine whether the test treatment is superior to placebo. An assumption is made that if the active control treatment remains efficacious, as was observed when it was compared against placebo, then a test treatment that has comparable efficacy with the active control, within a certain range, must also be superior to placebo. Because of this assumption, the design, implementation, and analysis of NI trials present challenges for sponsors and regulators. In designing and analyzing NI trials, substantial historical data are often required on the active control treatment and placebo. Bayesian approaches provide a natural framework for synthesizing the historical data in the form of prior distributions that can effectively be used in design and analysis of a NI clinical trial. Despite a flurry of recent research activities in the area of Bayesian approaches in medical product development, there are still substantial gaps in recognition and acceptance of Bayesian approaches in NI trial design and analysis. The Bayesian Scientific Working Group of the Drug Information Association provides a coordinated effort to target the education and implementation issues on Bayesian approaches for NI trials. In this article, we provide a review of both frequentist and Bayesian approaches in NI trials, and elaborate on the implementation for two common Bayesian methods including hierarchical prior method and meta-analytic-predictive approach. Simulations are conducted to investigate the properties of the Bayesian methods, and some real clinical trial examples are presented for illustration.

Noninferiority Trials in Regulatory Guidance and Marketing Authorization Applications: Huge Advances Over the Last 20 Years but Problems Still to be Solved

The analysis of noninferiority trials in marketing authorization applications has advanced greatly over the last 20 years, moving from an approach where failing to detect a statistically significant difference led to concluding that there was no difference to the current confidence-interval-based methods. This article follows the changes with reference to regulatory guidance issued across this period while touching on some of the regularly recurring debates surrounding analysis populations and coverage probabilities. The complex issue of how to define the noninferiority margin is addressed, including discussion of whether a margin exists in all situations and the implications this has for the entire noninferiority framework.

Design and Analysis of Non-inferiority Trials by Mark D. Rothmann, Brian L. Wiens and Ivan S. F. Chan

Design and Analysis of Non-inferiority Trials by Mark D. Rothmann, Brian L. Wiens and Ivan S. F. Chan

Challenges in the design and analysis of non-inferiority trials: a case study

The literature on the design, conduct, and analysis of non-inferiority trials is continuously evolving. Several design issues continue to be researched including the choice of active control, choice of non-inferiority margin, and optimal analytic approaches. To date, there has been relatively little in the literature documenting actual experiences with implementing available methodology for non-inferiority trials. This article serves as a case study and highlights some of the challenges encountered in the design of a Phase III non-inferiority trial in status epilepticus that is being conducted under a Food and Drug Administration Investigational New Drug Application (IND). The IND application was put on clinical hold by the Food Drug and Administration due to concerns with the design. Specifically, support for the active control, non-inferiority margin, and overall interpretability of trial results were questioned, and a recommendation was made to consider a superiority design. The authors describe their interactions with the Food Drug and Administration and their application of available methods and approaches to address these concerns. The investigators' response to the clinical hold provided detailed information to support the conduct of a non-inferiority trial. The study team received Food Drug and Administration approval to initiate the trial in October 2008. The trial enrollment began in June 2009 and is being conducted by roughly 800 paramedic units in over 40 Emergency Medicine Service systems across the United States. There is still a great deal of methodological research needed to fully understand the application and impact of the non-inferiority trial design. It is evident that non-inferiority trials have an important place in clinical trial design and analysis. These trials may be the only way and only opportunity to answer certain questions; so, they must be designed and conducted with rigor. This case study is an attempt to share our experiences in implementation.

Assessing noninferiority in a three‐arm trial using the Bayesian approach

Non-inferiority trials, which aim to demonstrate that a test product is not worse than a competitor by more than a pre-specified small amount, are of great importance to the pharmaceutical community. As a result, methodology for designing and analyzing such trials is required, and developing new methods for such analysis is an important area of statistical research. The three-arm trial consists of a placebo, a reference and an experimental treatment, and simultaneously tests the superiority of the reference over the placebo along with comparing this reference to an experimental treatment. In this paper, we consider the analysis of non-inferiority trials using Bayesian methods which incorporate both parametric as well as semi-parametric models. The resulting testing approach is both flexible and robust. The benefit of the proposed Bayesian methods is assessed via simulation, based on a study examining home-based blood pressure interventions.

Executive summary: workshop on issues in the design and conduct of clinical trials of antibacterial drugs in the treatment of community-acquired pneumonia.

Executive summary: workshop on issues in the design and conduct of clinical trials of antibacterial drugs in the treatment of community-acquired pneumonia.

Alternatives for Discounting in the Analysis of Noninferiority Trials

Determining the efficacy of an experimental therapy relative to placebo on the basis of an active-control noninferiority trial requires reference to historical placebo-controlled trials. The validity of the resulting comparison depends on two key assumptions: assay sensitivity and constancy. Since the truth of these assumptions cannot be verified, it seems logical to raise the standard of evidence required to declare efficacy; this concept is referred to as discounting. It is not often recognized that two common design and analysis approaches, setting a noninferiority margin and requiring preservation of a fraction of the standard therapy's effect, are forms of discounting. The noninferiority margin is a particularly poor approach, since its degree of discounting depends on an irrelevant factor. Preservation of effect is more reasonable, but it addresses only the constancy assumption, not the issue of assay sensitivity. Gaining consensus on the most appropriate approach to the design and analysis of noninferiority trials will require a common understanding of the concept of discounting.