Intersection-union Test Research Articles

Assessing Equivalence of Two Assays Using Sensitivity and Specificity

The equivalence of two assays is determined using the sensitivity and specificity relative to a gold standard. The equivalence-testing criterion is based on a misclassification rate proposed by Burdick et al. (2005) and the intersection-union test (IUT) method proposed by Berger (1982). Using a variance components model and IUT methods, we construct bounds for the sensitivity and specificity relative to the gold standard assay based on generalized confidence intervals. We conduct a simulation study to assess whether the bounds maintain the stated test size. We present a computational example to demonstrate the method described in the paper.

On a multiple endpoints testing problem

In a clinical trial comparing drug with placebo, where there are multiple primary endpoints, we consider testing problems where an efficacious drug effect can be claimed only if statistical significance is demonstrated at the nominal level for all endpoints. Under the assumption that the data are multivariate normal, the multiple endpoint-testing problem is formulated. The usual testing procedure involves testing each endpoint separately at the same significance level using two-sample t-tests, and claiming drug efficacy only if each t-statistic is significant. In this paper we investigate properties of this procedure. We show that it is identical to both an intersection union test and the likelihood ratio test. A simple expression for the p-value is given. The level and power function are studied; it is shown that the test may be conservative and that it is biased. Computable bounds for the power function are established.

Independence in multi-way contingency tables: S.N. Roy's breakthroughs and later developments

In the mid-1950s S.N. Roy and his students contributed two landmark articles to the contingency table literature [Roy, S.N., Kastenbaum, M.A., 1956. On the hypothesis of no “interaction” in a multiway contingency table. Ann. Math. Statist. 27, 749–757; Roy, S.N., Mitra, S.K., 1956. An introduction to some nonparametric generalizations of analysis of variance and multivariate analysis. Biometrika 43, 361–376]. The first article generalized concepts of interaction from 2 × 2 × 2 contingency tables to three-way tables of arbitrary size and to larger tables. In the second article, which is the source of our primary focus, various notions of independence were clarified for three-way contingency tables, Roy's union–intersection test was applied to construct chi-squared tests of hypotheses about the structure of such tables, and the chi-squared statistics were shown not to depend on the distinction between response and explanatory variables. This work pre-dates by many years later developments that expressed such results in the context of loglinear models. It pre-dates by a quarter century the development of graphical models. We summarize the main results in these key articles and discuss the connection between them and the later developments of loglinear modeling and of graphical modeling. We also mention ways in which these later developments have themselves been further generalized.

How to deal with multiple treatment or dose groups in randomized clinical trials?

Multiplicity adjustment in general is currently a controversial topic. This review focuses on the proof of efficacy in randomized clinical trials. The ICH guidelines mandate control of the familywise error rate. Confidence intervals are clinically more appropriate than P-values or yes/no decisions. Therefore, simultaneous confidence intervals are proposed for several designs and aims in clinical trials. The computation of simultaneous confidence intervals for the difference or the ratio is demonstrated by means of real data examples using the R-packages multcomp and mratios. A special problem is the evaluation of dose-finding trials with and without the assumption that the effects increase with increasing doses. Simultaneous intervals are presented not only for one-way layouts and normal distributed endpoints, but also for higher way layouts, generalized linear models, and mixed models. Under importance ordering, the conditional testing of all hypotheses at level alpha will be shown using the intersection-union test principle. Other multiplicity issues (i.e. multiple endpoints, multiple analyses, and subgroup analyses) are discussed.

How to deal with multiple endpoints in clinical trials

Multiple endpoints are common in clinical trials. This article discusses statistical methods that can be applied to control the rate of false positive conclusions at an acceptable level. The considered methods include the Bonferroni adjustment and related methods, the intersection-union test, ordered hypotheses and gatekeeper procedures, composite endpoints and global assessment measures, closed testing procedures, and combinations of different approaches.

Similarity testing using paired comparison method

Statistical model and procedure for similarity tests using paired comparison method are developed in the framework of the “two one-sided tests” (TOST) according to the intersection-union test (IUT) principle. Critical values, testing powers and sample sizes for the tests are provided based on the binomial distribution. Confidence intervals used in similarity testing are discussed. Current practice of using the conventional confidence intervals in two-sided similarity testing is criticized. A 100(1 − α)% confidence interval associated exactly with a α-level TOST is provided. Two key points are addressed in the paper. (1) Unlike difference testing, no multiplicity adjustment is needed to achieve an overall α-level in the two-sided similarity hypothesis tests. (2) The conventional two-sided confidence intervals do not match the TOST.

A Confidence Interval for the Maximal Mean QT Interval Change Caused by Drug Effect

A statistical problem of primary interest in a thorough QT/QTc study is that of deciding if a drug is noninferior to placebo in terms of QT/QTc prolongation. A standard way of approaching this problem is to construct a 90% two-sided (or a 95% one-sided) confidence interval, using the t distribution, at each time point in the study for the difference in mean QTc between drug and placebo and to conclude that the drug is noninferior to placebo if the upper end points of all of these confidence intervals is less than a prespecified constant, such as 10 ms. Under standard normality assumptions, this procedure corresponds to both an intersection-union test and the likelihood ratio test of size .05. It is not without its drawbacks, however. It is conservative in that the probability of a type I error may be smaller than the intended level .05. It is also biased, which means that the power function, for some values of parameters in the alternative space, takes values less than .05. The May 12, 2005, draft of the International Conference on Harmonisation E14 guidance states: “a negative ‘thorough QT/QTc study’ is one in which the upper bound of the 95% one-sided confidence interval for the largest time-matched mean effect of the drug on the QTc interval excludes 10 ms.” In this article, we show how an approximate confidence interval can be constructed for the largest difference in population mean QT/QTc between drug and placebo. The interval is approximate in the sense that, as sample sizes increase, the asymptotic probability of coverage is at least as large as intended. The results of simulations on a proposed one-sided 95% confidence interval are provided and discussed. Situations in which this interval works well, and does not work well, are delineated.

Challenge of multiple co‐primary endpoints: a new approach

There are many disorders where regulatory agencies have required a new treatment to demonstrate efficacy on multiple co-primary endpoints, all significant at the one-sided 2.5 per cent level, before accepting the treatment's effect for the disorder. This requirement, rooted in the intersection-union (IU) test, has led many researchers to increase the study sample size to make up for the reduction in the statistical power at the study level. Unfortunately, the increase in sample size could be substantial when the endpoints are minimally correlated and the treatment effects on the multiple endpoints are comparable. In this paper, we demonstrate that the frequentist concept of controlling the maximum false positive rate, even when applied to a restricted null space, has only limited success in keeping the sample size increase at a reasonable level. We therefore propose an approach that is based on the notion of controlling an average type I error rate. By employing an upper bound for the average type I error rate, the new approach provides an adjustment to the significance level that depends only on the correlation among the endpoints. For the most common case of two or three co-primary endpoints, the adjusted significance level is at most 5 per cent (one-sided) when the endpoints are moderately correlated. We show how sample size could be calculated under the proposed approach and contrast the needed sample size with that required under the IU test. We provide additional comments and discuss why the new approach is consistent with the principle requiring evidence of significance in the drug development and approval process.

Hypothesis testing approaches to the exon prediction problem

Many gene identification methods assign scores to gene elements prior to their assembly into predicted genes. The scoring system is often based on log-likelihood ratios. These methods usually perform well but it is difficult to interpret how significant a score is. We have developed several tests of significance for the scores: (1) a sum-of-scores test (SST), (2) an intersection-union test (IUT), based on a multiple hypothesis testing interpretation of an exon's score and (3) a meta-analytical approach (MA), which combines several P-values, corresponding to the exon's parts, to yield a global P-value. We performed simulation studies, which show that the MA has better sensitivity and specificity than other methods and is easier to interpret by non-expert users. This is an improvement over other methods and is especially relevant for users who would like to predict incomplete gene sequences.

Some Improved Tests for Multivariate One-Sided Hypotheses

Multivariate one-sided hypothesis-testing problems are very common in clinical trials with multiple endpoints. The likelihood ratio test (LRT) and union-intersection test (UIT) are widely used for testing such problems. It is argued that, for many important multivariate one-sided testing problems, the LRT and UIT fail to adapt to the presence of subregions of varying dimensionalities on the boundary of the null parameter space and thus give undesirable results. Several improved tests are proposed that do adapt to the varying dimensionalities and hence reflect the evidence provided by the data more accurately than the LRT and UIT. Moreover, the proposed tests are often less biased and more powerful than the LRT and UIT.

OIII-A-2Performance characteristics of the analysis method for “thorough QT” studies in ICH E-14

BACKGROUND/AIMS The probability of incorrectly concluding QT/QTc prolongation was evaluated using an intersection-union test (IUT - ICH E-14 guidance for “thorough QT” (TQT) studies). Bias in the estimated true mean was also evaluated. METHODS Placebo data from 4 TQT trials (2 parallel with N=30, 24/group and 2 crossover each with N=60) were used. Nonparametric bootstrapping (resampling subjects with replacement) was used to simulate 300 TQT trials under the original study designs. Each simulated trial consisted of placebo data only but with a treatment group designated arbitrarily. A study was considered positive if the maximum confidence bound was greater than 10 ms. Additionally the analysis considered true effects of 0–5 ms and various sample sizes. RESULTS The largest mean difference estimates ranged from 0.5 to 4.0 ms depending on study designs despite the true drug effect of zero. The false positive probabilities ranged from 0–62%. The false positive rate for crossover trials was = 40 and true effects up to 4 msec. For small parallel trials, the false positive rate was up to 62%. Smaller sample sizes and larger true effects increased the false positive rate. CONCLUSIONS The false positive rate and bias for TQT based on the IUT vary with study design, sample size, and the true drug effect. Large sample sizes (N>150/group) may be necessary for parallel trials to avoid incorrect conclusions about QT/QTc prolongation liability for new drugs. Clinical Pharmacology & Therapeutics (2005) 79, P29–P29; doi: 10.1016/j.clpt.2005.12.107

Power and sample size for clinical trials when efficacy is required in multiple endpoints: application to an Alzheimer's treatment trial

When the efficacy of a treatment in a randomized controlled trial is required for multiple primary endpoints, trial design and analysis differ from trial requiring efficacy in only one of the multiple endpoints. We consider a two-arm clinical trial requiring efficacy analysis for multiple primary endpoints, formulating the appropriate null and alternative hypotheses for the test of treatment efficacy. We study the significance level/statistical power of an intersection-union test (IUT) in this situation. We compare IUT with the intuitive approach (selecting the maximum sample size over those obtained from testing individual primary endpoints one by one) for determination of sample size. The proposed IUT reserves the same Type I error rate as shared by all endpoint-specific tests. The statistical power of the proposed IUT is no more than the minimum from the individual tests. The maximum sample size from multiple endpoint-specific tests is often inadequate for the test of treatment efficacy, especially when the standardized effect sizes are similar. Finally, the IUT can be applied to Alzheimer's disease treatment trials in which two primary endpoints are typically used. The IUT is a valid method for use in the design and analysis of clinical trials requiring efficacy at multiple primary endpoints.

A Simulation Study of Power in Thorough QT/QTc Studies and a Normal Approximation for Planning Purposes

We report the results of a simulation study on power in a thorough QT/QTc study with a four-period crossover design in which the treatments are placebo, positive control, higher dose of investigational drug, and therapeutic dose of investigational drug. In the study, QTc interval values were obtained for several specified times during treatment. An assessment of noninferiority of the higher dose to placebo was performed by the intersection-union test within the framework of a linear mixed-effects analysis. With a noninferiority bound of 8 ms and a peak drug effect of 4 ms, it is found that 96 or more subjects would be required to have 80% power, but with a noninferiority bound of 10 ms, 48 subjects give more than an estimated 80% power if the number of times of measurement is 10 or fewer. For the purposes of planning such a study, a power approximation based on the multivariate normal distribution of the estimator of the difference in means of high dose of investigational drug and placebo is presented. The approximated power and the power estimate from the simulation study were quite close.

A MULTIVARIATE TEST FOR SIMILARITY OF TWO DISSOLUTION PROFILES

ABSTRACT A multivariate test of size α for assessing the similarity of two dissolution profiles is proposed. The inferential procedure is developed by using the approach for the common mean problem in a multivariate setup due to Halperin (1961). The performance of the proposed method is compared with Intersection Union Test as well as f 2 criterion recommended by the FDA through a simulation study. All the methods are illustrated with real examples.

A superiority-equivalence approach to one-sided tests on multiple endpoints in clinical trials

This paper considers the problem of comparing a new treatment with a control based on multiple endpoints. The hypotheses are formulated with the goal of showing that the treatment is equivalent, i.e. not inferior, on all endpoints and superior on at least one endpoint compared to the control, where thresholds for equivalence and superiority are specified for each endpoint. Roy’s (1953) union-intersection and Berger’s (1982) intersection-union principles are employed to derive the basic test. It is shown that the critical constants required for the union-intersection test of superiority can be sharpened by a careful analysis of its type I error rate. The composite - test is illustrated by an example and compared in a simulation study to alternative tests proposed by Bloch et al. (2001) and Perlman & Wu (2004). The Bloch et al. test does not control the type I error rate because of its nonmonotone nature, and is hence not recommended. The - and the Perlman & Wu tests both control the type I error rate, but the latter test generally has a slightly higher power.

Nonparametric Tests for Positive Quadrant Dependence

We consider distributional free inference to test for positive quadrant dependence, i.e. for the probability that two variables are simultaneously small (or) large being at least as great as it would be were they dependent. Tests for its generalisation in higher dimensions, namely positive orthant dependences, are also analysed. We propose two types of testing procedures. The first procedure is based on the specification of the dependence concepts in terms of distribution functions, while the second procedure exploits the copula representation. For each specification a distance test and an intersection-union test for inequality constraints are developed depending on the definition of null and alternative hypotheses. An empirical illustration is given for US and Danish insurance claim data. Practical implications for the design of reinsurance treaties are also discussed.

Testing for Concordance Ordering

We propose inference tools to analyse the concordance (or correlation) order of random vectors. The analysis in the bivariate case relies on tests for upper and lower quadrant dominance of the true distribution by a parametric or semiparametric model, i.e. for a parametric or semiparametric model to give a probability that two variables are simultaneously small or large at least as great as it would be were they left unspecified. Tests for its generalisation in higher dimensions, namely joint lower and upper orthant dominance, are also analysed. The parametric and semiparametric settings are based on the copula representation for multivariate distribution, which allows for disentangling behaviour of margins and dependence structure. A distance test and an intersection-union test for inequality constraints are developed depending on the definition of null and alternative hypotheses. An empirical illustration is given for US insurance claim data.

Testing for Concordance Ordering

We propose inference tools to analyse the concordance (or correlation) order of random vectors. The analysis in the bivariate case relies on tests for upper and lower quadrant dominance of the true distribution by a parametric or semiparametric model, i.e. for a parametric or semiparametric model to give a probability that two variables are simultaneously small or large at least as great as it would be were they left unspecified. Tests for its generalisation in higher dimensions, namely joint lower and upper orthant dominance, are also analysed. The parametric and semiparametric settings are based on the copula representation for multivariate distribution, which allows for disentangling behaviour of margins and dependence structure. A distance test and an intersection-union test for inequality constraints are developed depending on the definition of null and alternative hypotheses. An empirical illustration is given for US insurance claim data.

Random Field–Union Intersection tests for EEG/MEG imaging

Electrophysiological (EEG/MEG) imaging challenges statistics by providing two views of the same spatiotemporal data: topographic and tomographic. Until now, statistical tests for these two situations have developed separately. This work introduces statistical tests for assessing simultaneously the significance of spatiotemporal event-related potential/event-related field (ERP/ERF) components and that of their sources. The test for detecting a component at a given time instant is provided by a Hotelling's T 2 statistic. This statistic is constructed in such a manner to be invariant to any choice of reference and is based upon a generalized version of the average reference transform of the data. As a consequence, the proposed test is a generalization of the well-known Global Field Power statistic. Consideration of tests at all time instants leads to a multiple comparison problem addressed by the use of Random Field Theory (RFT). The Union-Intersection (UI) principle is the basis for testing hypotheses about the topographic and tomographic distributions of such ERP/ERF components. The performance of the method is illustrated with actual EEG recordings obtained from a visual experiment of pattern reversal stimuli.

A Note on One‐Sided Tests with Multiple Endpoints

Testing problems with multivariate one-sided alternative hypotheses are common in clinical trials with multiple endpoints. In the case of comparing two treatments, treatment 1 is often preferred if it is superior for at least one of the endpoints and not biologically inferior for the remaining endpoints. Bloch et al. (2001, Biometrics57, 1039-1047) propose an intersection-union test (IUT) for this testing problem, but their test does not utilize the appropriate multivariate one-sided test. In this note we modify their test by an alternative IUT that does utilize the appropriate one-sided test. Empirical and graphical evidence show that the proposed test is more appropriate for this testing problem.