The issue of multiplicity in noninferiority studies

Abstract

In a clinical study to evaluate noninferiority (NI) of an experimental drug relative to an established therapy, it is common to further test superiority of the experimental drug after NI is established. It has been shown that no multiplicity issue exists between NI and superiority tests of the primary endpoint. However, when there is an additional, or secondary, endpoint that will be tested for superiority in a hierarchical fashion to the NI testing of the primary endpoint, it is not clear whether the overall type I error rate is strictly controlled among all the tests. In this article, our goal is to evaluate if the family-wise type I error rate is strictly controlled in this setting. We also evaluate a multiplicity adjustment procedure based on the Hochberg procedure. We use the closed testing principle to evaluate the family-wise type I error rate. Some simulations are performed to appreciate the magnitude of the potential inflation of the type I error rate. It is demonstrated that the family-wise type I error rate is not controlled and an appropriate multiplicity adjustment procedure must take into account the NI and superiority tests of the two endpoints. When the test statistic used for superiority testing of the primary endpoint is the same as that for the NI testing, a multiplicity adjustment method using the popular Hochberg procedure is shown to be potentially conservative. The assessment is based on a simplified set-up where there is only one secondary endpoint tested for superiority in addition to the primary endpoint. It is necessary to evaluate the issue of multiplicity in non-inferiority studies to assure strict control of the family-wise type I error rate.