Intersection Hypotheses Research Articles

A general consonance principle for closure tests based on -values.

The closure principle is a powerful approach to constructing efficient testing procedures controlling the familywise error rate in the strong sense. For small numbers of hypotheses and the setting of independent elementary -values we consider closed tests where each intersection hypothesis is tested with a -value combination test. Examples of such combination tests are the Fisher combination test, the Stouffer test, the Omnibus test, the truncated test, or the Wilson test. Some of these tests, such as the Fisher combination, the Stouffer, or the Omnibus test, are not consonant and rejection of the global null hypothesis does not always lead to rejection of at least one elementary null hypothesis. We develop a general principle to uniformly improve closed tests based on -value combination tests by modifying the rejection regions such that the new procedure becomes consonant. For the Fisher combination test and the Stouffer test, we show by simulations that this improvement can lead to a substantial increase in power.

Blinded sample size recalculation in multiple composite population designs with normal data and baseline adjustments.

The increasing interest in subpopulation analysis has led to the development of various new trial designs and analysis methods in the fields of personalized medicine and targeted therapies. In this paper, subpopulations are defined in terms of an accumulation of disjoint population subsets and will therefore be called composite populations. The proposed trial design is applicable to any set of composite populations, considering normally distributed endpoints and random baseline covariates. Treatment effects for composite populations are tested by combining p-values, calculated on the subset levels, using the inverse normal combination function to generate test statistics for those composite populations while the closed testing procedure accounts for multiple testing. Critical boundaries for intersection hypothesis tests are derived using multivariate normal distributions, reflecting the joint distribution of composite population test statistics given no treatment effect exists. For sample size calculation and sample size, recalculation multivariate normal distributions are derived which describe the joint distribution of composite population test statistics under an assumed alternative hypothesis. Simulations demonstrate the absence of any practical relevant inflation of the type I error rate. The target power after sample size recalculation is typically met or close to beingmet.

Spectral rigidity for spherically symmetric manifolds with boundary

We prove a trace formula for three-dimensional spherically symmetric Riemannian manifolds with boundary which satisfy the Herglotz condition: Under a “clean intersection hypothesis” and assuming an injectivity hypothesis associated to the length spectrum, the wave trace is singular at the lengths of periodic broken rays. In particular, the Neumann spectrum of the Laplace–Beltrami operator uniquely determines the length spectrum. The trace formula also applies for the toroidal modes of the free oscillations in the earth. Under this hypothesis and the Herglotz condition, we then prove that the length spectrum is rigid: Deformations preserving the length spectrum and spherical symmetry are necessarily trivial in any dimension, provided the Herglotz condition and a geometrical condition are satisfied. Combining the two results shows that the Neumann spectrum of the Laplace–Beltrami operator is rigid in this class of manifolds with boundary.

Optimizing Graphical Procedures for Multiplicity Control in a Confirmatory Clinical Trial via Deep Learning

In confirmatory clinical trials, it has been proposed to use a simple iterative graphical approach to construct and perform intersection hypotheses tests with a weighted Bonferroni-type procedure to control Type I errors in the strong sense. Given Phase II study results or other prior knowledge, it is usually of main interest to find the optimal graph that maximizes a certain objective function in a future Phase III study. In this article, we evaluate the performance of two existing derivative-free constrained methods, and further propose a deep learning enhanced optimization framework. Our method numerically approximates the objective function via feedforward neural networks (FNNs) and then performs optimization with available gradient information. It can be constrained so that some features of the testing procedure are held fixed while optimizing over other features. Simulation studies show that our FNN-based approach has a better balance between robustness and time efficiency than some existing derivative-free constrained optimization algorithms. Compared to the traditional stochastic search method, our optimizer has moderate multiplicity adjusted power gain when the number of hypotheses is relatively large. We further apply it to a case study to illustrate how to optimize a multiple testing procedure with respect to a specific study objective.

An improved closed procedure for testing multiple hypotheses.

Clinical trials routinely involve multiple hypothesis testing. The closed testing procedure (CTP) is a fundamental principle in testing multiple hypotheses. This article presents an improved CTP in which intersection hypotheses can be tested at a level greater than such that the control of the familywise error rate at level remains. Consequently, our method uniformly improves the power of discovering false hypotheses over the original CTP. We illustrate that an improvement by our method exists for many commonly used tests. An empirical study on the effectiveness of a glucose-lowering drug is provided.

Comparisons of global tests on intersection hypotheses and their application in matched parallel gatekeeping procedures

ABSTRACT A clinical trial often has primary and secondary endpoints and comparisons of high and low doses of a study drug to a control. Multiplicity is not only caused by the multiple comparisons of study drugs versus the control, but also from the hierarchical structure of the hypotheses. Closed test procedures were proposed as general methods to address multiplicity. Two commonly used tests for intersection hypotheses in closed test procedures are the Simes test and the average method. When the treatment effect of a less efficacious dose is not much smaller than the treatment effect of a more efficacious dose for a specific endpoint, the average method has better power than the Simes test for the comparison of two doses versus control. Accordingly, for inferences for primary and secondary endpoints, the matched parallel gatekeeping procedure based on the Simes test for testing intersection hypotheses is extended here to allow the average method for such testing. This procedure is further extended to clinical trials with more than two endpoints as well as to clinical trials with more than two active doses and a control.

Symmetric graphs for equally weighted tests, with application to the Hochberg procedure.

The graphical approach to multiple testing provides a convenient tool for designing, visualizing, and performing multiplicity adjustments in confirmatory clinical trials while controlling the familywise error rate. It assigns a set of weights to each intersection null hypothesis within the closed test framework. These weights form the basis for intersection tests using weighted individual p-values, such as the weighted Bonferroni test. In this paper, we extend the graphical approach to intersection tests that assume equal weights for the elementary null hypotheses associated with any intersection hypothesis, including the Hochberg procedure as well as omnibus tests such as Fisher's combination, O'Brien's, and F tests. More specifically, we introduce symmetric graphs that generate sets of equal weights so that the aforementioned tests can be applied with the graphical approach. In addition, we visualize the Hochberg and the truncated Hochberg procedures in serial and parallel gatekeeping settings using symmetric component graphs. We illustrate the method with two clinical trial examples.

Closed testing using surrogate hypotheses with restricted alternatives.

IntroductionThe closed testing principle provides strong control of the type I error probabilities of tests of a set of hypotheses that are closed under intersection such that a given hypothesis H can only be tested and rejected at level α if all intersection hypotheses containing that hypothesis are also tested and rejected at level α. For the higher order hypotheses, multivariate tests (> 1df) are generally employed. However, such tests are directed to an omnibus alternative hypothesis of a difference in any direction for any component that may be less meaningful than a test directed against a restricted alternative hypothesis of interest.MethodsHerein we describe applications of this principle using an α-level test of a surrogate hypothesis such that the type I error probability is preserved if such that rejection of implies rejection of H. Applications include the analysis of multiple event times in a Wei-Lachin test against a one-directional alternative, a test of the treatment group difference in the means of K repeated measures using a 1 df test of the difference in the longitudinal LSMEANS, and analyses within subgroups when a test of treatment by subgroup interaction is significant. In such cases the successive higher order surrogate tests can be aimed at detecting parameter values that fall within a more desirable restricted subspace of the global alternative hypothesis parameter space.ConclusionClosed testing using α-level tests of surrogate hypotheses will protect the type I error probability and detect specific alternatives of interest, as opposed to the global alternative hypothesis of any difference in any direction.

Optimal exact tests for multiple binary endpoints

In confirmatory clinical trials with small sample sizes, hypothesis tests based on asymptotic distributions are often not valid and exact non-parametric procedures are applied instead. However, the latter are based on discrete test statistics and can become very conservative, even more so, if adjustments for multiple testing as the Bonferroni correction are applied. Improved exact multiple testing procedures are proposed for the setting where two parallel groups are compared in multiple binary endpoints. Based on the joint conditional distribution of test statistics of Fisher’s exact tests, optimal rejection regions for intersection hypothesis tests are constructed utilizing different objective functions. Depending on the optimization objective, the optimal test yields maximal power under a specific alternative, maximal exhaustion of the nominal type I error rate, or the largest possible rejection region controlling the type I error rate. To efficiently search the large space of possible rejection regions, an optimization algorithm based on constrained optimization and integer linear programming is proposed. Applying the closed testing principle, optimized multiple testing procedures with strong familywise error rate control are constructed. Furthermore, a computationally efficient greedy algorithm for nearly optimal tests is proposed. The unconditional power of the optimized procedures is numerically compared to the power of alternative approaches and the optimal tests are illustrated with a clinical trial example in a rare disease. The described methods are implemented in the R package multfisher.

Accurate p-values for adaptive designs with binary endpoints.

Adaptive designs encompass all trials allowing various types of design modifications over the course of the trial. A key requirement for confirmatory adaptive designs to be accepted by regulators is the strong control of the family-wise error rate. This can be achieved by combining the p-values for each arm and stage to account for adaptations (including but not limited to treatment selection), sample size adaptation and multiple stages. While the theory for this is novel and well-established, in practice, these methods can perform poorly, especially for unbalanced designs and for small to moderate sample sizes. The problem is that standard stagewise tests have inflated type I error rate, especially but not only when the baseline success rate is close to the boundary and this is carried over to the adaptive tests, seriously inflating the family-wise error rate. We propose to fix this problem by feeding the adaptive test with second-order accurate p-values, in particular bootstrap p-values. Secondly, an adjusted version of the Simes procedure for testing intersection hypotheses that reduces the built-in conservatism is suggested. Numerical work and simulations show that unlike their standard counterparts the new approach preserves the overall error rate, at or below the nominal level across the board, irrespective of the baseline rate, stagewise sample sizes or allocation ratio. Copyright © 2017 John Wiley & Sons, Ltd.

A unified framework for weighted parametric multiple test procedures

We describe a general framework for weighted parametric multiple test procedures based on the closure principle. We utilize general weighting strategies that can reflect complex study objectives and include many procedures in the literature as special cases. The proposed weighted parametric tests bridge the gap between rejection rules using either adjusted significance levels or adjusted p-values. This connection is made by allowing intersection hypotheses of the underlying closed test procedure to be tested at level smaller than α. This may be also necessary to take certain study situations into account. For such cases we introduce a subclass of exact α-level parametric tests that satisfy the consonance property. When the correlation is known only for certain subsets of the test statistics, a new procedure is proposed to fully utilize this knowledge within each subset. We illustrate the proposed weighted parametric tests using a clinical trial example and conduct a simulation study to investigate its operating characteristics.

Mathematical modelling for variations of inbreeding populations fitness with single and polygenic traits

BackgroundInbreeding mating has been widely accepted as the key mechanism to enhance homozygosity which normally will decrease the fitness of the population. Although this result has been validated by a large amount of biological data from the natural populations, a mathematical proof of these experimental discoveries is still not complete. A related question is whether we can extend the well-established result regarding the mean fitness from a randomly mating population to inbreeding populations. A confirmative answer may provide insights into the frequent occurrence of self-fertilization populations.ResultsThis work presents a theoretic proof of the result that, for a large inbreeding population with directional relative genotype fitness, the mean fitness of population increases monotonically. However, it cannot be extended to the case with over-dominant genotype fitness. In addition, by employing multiplicative intersection hypothesis, we prove that inbreeding mating does decrease the mean fitness of polygenic population in general, but does not decrease the mean fitness with mixed dominant-recessive genotypes. We also prove a novel result that inbreeding depression depends on not only the mating pattern but also genetic structure of population.ConclusionsFor natural inbreeding populations without serious inbreeding depression, our theoretical analysis suggests the majority of its genotypes should be additive or dominant-recessive genotypes. This result gives a reason to explain why many hermaphroditism populations do not show severe inbreeding depression. In addition, the calculated purging rate shows that inbreeding mating purges the deleterious mutants more efficiently than randomly mating does.

Weighted false discovery rate controlling procedures for clinical trials.

SummaryHaving identified that the lack of replicability of results in earlier phases of clinical medical research stems largely from unattended selective inference, we offer a new hierarchical weighted false discovery rate controlling testing procedure alongside the single-level weighted procedure. These address the special structure of clinical research, where the comparisons of treatments involve both primary and secondary endpoints, by assigning weights that reflect the relative importance of the endpoints in the error being controlled. In the hierarchical method, the primary endpoints and a properly weighted intersection hypothesis that represents all secondary endpoints are tested. Should the intersection hypothesis be among the rejected, individual secondary endpoints are tested. We identify configurations where each of the two procedures has the advantage. Both offer higher power than competing hierarchical (gatekeeper) familywise error-rate controlling procedures being used for drug approval. By their design, the advantage of the proposed methods is the increased power to discover effects on secondary endpoints, without giving up the rigor of addressing their multiplicity.

Graphical approaches using a Bonferroni mixture of weighted Simes tests.

Graphical approaches to multiple testing procedures are very flexible and easy to communicate with non-statisticians. The availability of the R package gMCP further propelled the application of graphical approaches in randomized clinical trials. Bretz et al. (Biometrical Journal 2011; 53:894-913) introduced a class of nonparametric testing procedures based on a Bonferroni mixture of weighted Simes tests for intersection hypotheses. Such approaches are extremely useful when the conditions for the Simes test are known to hold for hypotheses within certain subsets but may not hold for hypotheses across subsets. We describe the calculation of adjusted p-values for such approaches, which is currently not available in the gMCP package. We also optimize the generation of the weights for each intersection hypothesis in the closure of a graph-based multiple testing procedure, which can dramatically reduce the computing time for simulation-based power calculations. We show the validity of the Simes test for comparing several treatments with a control, performing noninferiority and superiority tests, or testing the treatment effect in an overall and a subpopulation for the normal, binary, count, and time-to-event data. The proposed method is illustrated using an example for designing a confirmatory clinical trial. Copyright © 2016 John Wiley & Sons, Ltd.

New multiple testing method under no dependency assumption, with application to multiple comparisons problem

Traditional multiple hypotheses testing mainly focuses on constructing stepwise procedures under some error rate control, such as familywise error rate (FWER), false discovery rate, and so forth. However, most of these procedures are obtained in independent case, and when there exists correlation across tests, the dependency may increase or decrease the chance of false rejections. In this paper, a totally different testing method is proposed, which doesn’t focus on specific error control, but pays attention to the overall performance of the collection of hypotheses and the structure utilization among hypotheses. Since the main purpose of multiple testing is to pick out the false ones from the whole hypotheses and present a rejection set, motivated by the principle of simple hypothesis testing, we give the final testing result based on the estimation of the set of all the true null hypotheses. Our method can be applied in any dependent case provided that a reasonable \(p\)-value can be obtained for each intersection hypothesis. We illustrate the new procedures with application to multiple comparisons problems. Theoretical results show the consistency of our method, and investigate their FWER behavior. Simulation results suggest that our procedures have a better overall performance than some existing procedures in dependent cases, especially in the total number of type I and type II errors.

Optimizing the data combination rule for seamless phase II/III clinical trials.

We consider seamless phase II/III clinical trials that compare K treatments with a common control in phase II then test the most promising treatment against control in phase III. The final hypothesis test for the selected treatment can use data from both phases, subject to controlling the familywise type I error rate. We show that the choice of method for conducting the final hypothesis test has a substantial impact on the power to demonstrate that an effective treatment is superior to control. To understand these differences in power, we derive decision rules maximizing power for particular configurations of treatment effects. A rule with such an optimal frequentist property is found as the solution to a multivariate Bayes decision problem. The optimal rules that we derive depend on the assumed configuration of treatment means. However, we are able to identify two decision rules with robust efficiency: a rule using a weighted average of the phase II and phase III data on the selected treatment and control, and a closed testing procedure using an inverse normal combination rule and a Dunnett test for intersection hypotheses. For the first of these rules, we find the optimal division of a given total sample size between phases II and III. We also assess the value of using phase II data in the final analysis and find that for many plausible scenarios, between 50% and 70% of the phase II numbers on the selected treatment and control would need to be added to the phase III sample size in order to achieve the same increase in power.

Multiple Testing in Group Sequential Trials Using Graphical Approaches

In 1997, Robert T. O’Neill introduced the framework of structuring the experimental questions to best reflect the clinical study’s objectives. This task comprises the identification of the study’s primary, secondary, and exploratory objectives and the requirements as to when the corresponding statistical tests are considered meaningful. A topic that has been considered much less in the literature until very recently is the application of group sequential trial designs to multiple endpoints. In 2007, Hung, Wang, and O’Neill showed that borrowing testing strategies naively from trial designs without interim analyses may not maintain the familywise Type I error rate at level α. The authors gave examples in the context of testing two hierarchically ordered endpoints in two-armed group sequential trials with one interim analysis. We consider the general situation of testing multiple hypotheses repeatedly in time using recently developed graphical approaches. We focus on closed testing procedures using weighted group sequential Bonferroni tests for the intersection hypotheses. Under mild monotonicity conditions on the error spending functions, this allows the use of sequentially rejective graphical procedures in group sequential trials. The methodology is illustrated with a numerical example for a three-armed trial comparing two doses against control for two hierarchically ordered endpoints.

Correction: ‘A conditional error function approach for subgroup selection in adaptive clinical trials’

It has recently come to our attention that some of the results presented in Table II in ’A conditional error function approach for subgroup selection in adaptive clinical trials’ (Statistics in Medicine 2012; 31:4309–4320) are not correct. An R software coding issue resulted in numerical errors in the reported results for the conditional error function approach (CEF) and for the combination test approach by Spiessens and Debois (CT-SD). We have corrected these errors in the following table. We also, for reasons of consistency with the CEF approach, now present type I error rates for the CT-SD approach based purely on rejection of the intersection hypothesis inline image rather than as previously on rejection of both intersection and one or the other of the elementary hypotheses inline image and inline image. It is now clear that type I error rates are controlled at the nominal 2.5% level for both approaches, and our previous assertion that the CEF methodology was uniformly, although only marginally, more powerful than the CT-SD methodology is no longer supported by the simulation results for the selected scenarios. Correction of coding errors produced such small changes to data presented in Figures 1–3 as to be indistinguishable from normal simulation error; updated data underlying these plots are available on request from the authors. We apologize for any inconvenience this error has caused.

Multiple tests of cost‐effectiveness angles

Cost-effectiveness angles are an attractive measure of performance when comparing effects and costs of health-care therapies because they have a clear interpretation and are well suited for statistical inference. In clinical trials, a common setup is the comparison of multiple new therapies with a single control. If cost-effectiveness angles are calculated for each comparison, multiplicity issues should be taken into account when quantifying uncertainty of the point estimates. Therefore, this paper proposes a parametric test for multiple cost-effectiveness angles that guarantees strong family-wise error rate control. The idea is to replace the test of m cost-effectiveness angles as a union-intersection test of 3m linear hypotheses. Considering the correlation structure of the individual test statistics for the linear hypotheses leads to a maximum-type test for the intersection hypothesis. Inverting these test decisions then gives simultaneous CIs of cost-effectiveness angles with the appropriate coverage probabilities.

A new partition testing strategy for multiple endpoints

To evaluate efficacy in multiple endpoints in confirmatory clinical trials is a challenging problem in multiple hypotheses testing. The difficulty comes from the different importance of each endpoint and their underlying correlation. Current approaches to this problem, which test the efficacy in certain dose-endpoint combinations and collate the results, are based on closed testing or partition testing. Despite their different formulations, all current approaches test their dose-endpoint combinations as intersection hypotheses and apply various union-intersection tests. Likelihood ratio test is seldom used owing to the extensive computation and lack of consistent inferences. In this article, we first generalize the formulation of multiple endpoints problem to include the cases of alternative primary endpoints and co-primary endpoints. Then we propose a new partition testing approach that is based on consonance-adjusted likelihood ratio test. The new procedure provides consistent inferences, and yet, it is still conservative and does not rely on the estimation of endpoint correlation or independence assumptions that might be challenged by regulatory agencies.