Investigations on non‐inferiority—the Food and Drug Administration draft guidance on treatments for nosocomial pneumonia as a case for exact tests for binomial proportions

Abstract

This paper addresses statistical issues in non-inferiority trials where the primary outcome is a fatal event. The investigations are inspired by a recent Food and Drug Administration (FDA) draft guideline on treatments for nosocomial pneumonia. The non-inferiority margin suggested in this guideline for the endpoint all-cause mortality is defined on different distance measures (rate difference and odds ratio) and is discontinuous. Furthermore, the margin enables considerable power for the statistical proof of non-inferiority at alternatives that might be regarded as clinically unacceptable, that is, even if the experimental treatment is harmful as compared with the control. We investigated the appropriateness of the proposed non-inferiority margin as well as the performance of possible test statistics to be used for the analysis. A continuous variant of the margin proposed in the FDA guideline together with the unconditional exact test according to Barnard showed favorable characteristics with respect to type I error rate control and power. To prevent harmful new treatments from being declared as non-inferior, we propose to add a 'second hurdle'. We discuss examples and explore power characteristics when requiring both statistical significance and overcoming the second hurdle.