Step-up Multiple Test Procedure Research Articles

Asymptotic comparison of the critical values of step-down and step-up multiple comparison procedures

Abstract We consider the problem of comparing step-down and step-up multiple test procedures for testing n hypotheses when independent p -values or independent test statistics are available. The defining critical values of these procedures for independent test statistics are asymptotically equal, which yields a theoretical argument for the numerical observation that the step-up procedure is mostly more powerful than the step-down procedure. The main aim of this paper is to quantify the differences between the critical values more precisely. As a by-product we also obtain more information about the gain when we consider two subsequent steps of these procedures. Moreover, we investigate how liberal the step-up procedure becomes when the step-up critical values are replaced by their step-down counterparts or by more refined approximate values. The results for independent p -values are the basis for obtaining corresponding results when independent real-valued test statistics are at hand. It turns out that the differences of step-down and step-up critical values as well as the differences between subsequent steps tend to zero for many distributions, except for heavy-tailed distributions. The Cauchy distribution yields an example where the critical values of both procedures are nearly linearly increasing in n .

Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics

In this paper interest is focused on some theoretical investigations concerning the comparison of two popular multiple test procedures, so-called step-down and step-up procedures, in terms of their defining critical values. Such procedures can be applied, for example, to multiple comparisons with a control. In the definition of the critical values for these procedures order statistics play a central role. For $k \epsilon N_0$ fixed we consider the joint cumulative distribution function (cdf) $P(Y_{1:n} \leq c_1,\dots, Y_{n-k:n} \leq c_{n-k})$ of the first $n - k$ order statistics and the cdf$P(Y_{n-k:n} \leq c_{n-k})$ of the ($k + 1$)th largest order statistic $Y_{n-k:n}$ of $n$ random variables $Y_1,\dots, Y_n$ belonging to a sequence of exchangeable real-valued random variables. Our interest is focused on the asymptotic behavior of these cdf's and their interrelation if $n$ tends to $\infty$. It turns out that they sometimes behave completely differently compared with the iid case treated in Finner and Roters so that positive results are only possible under additional assumptions concerning the underlying distribution. We consider different sets of assumptions which then allow analogous results for the exchangeable case. Recently, Dalal and Mallows derived a result concerning the monotonicity of a certain set of critical values in connection with the joint cdf of order statistics in the iid case. We give a counterexample for the exchangeable case underlining the difficulties occurring in this situation. As an application we consider the comparison of certain step-down and step-up procedures in multiple comparisons with a control. The results of this paper yield a more theoretical explanation of the superiority of the step-up procedure which has been observed earlier by comparing tables of critical values. As a by-product we are able to quantify the tightness of the Bonferroni inequality in connection with maximum statistics.

Step-Up Multiple Testing of Parameters with Unequally Correlated Estimates

We consider the problem of simultaneously testing k > or = to 2 hypotheses on parameters theta(1), ..., theta(k) using test statistics t(1), ..., t(k) such that a specified familywise error rate alpha is achieved. Dunnett and Tamhane (1992a) proposed a step-up multiple test procedure, in which testing starts with the hypothesis corresponding to the least significant test statistic and proceeds towards the most significant, stopping the first time a significant test result is obtained (and rejecting the hypotheses corresponding to that and any remaining test statistics). The parameter estimates used in the t statistics were assumed to be normally distributed with a common variance, which was a known multiple of an unknown sigma(2), and known correlations which were equal. In the present article, we show how the procedure can be extended to include unequally correlated parameter estimates. Unequal correlations occur, for example, in experiments involving comparisons among treatment groups with unequal sample sizes. We also compare the step-up and step-down multiple testing approaches and discuss applications to some biopharmaceutical testing problems.

Power comparisons of some step-up multiple test procedures

Powers of some recently proposed step-up multiple test procedures are compared. These include procedures of Hommel (1988), Hochberg (1988) and Rom (1990), which are based on the individual p-values of the test statistics. The powers of the normal theory based step-up procedure of Dunnett and Tamhane (1992) are also included to assess further gains due to taking the correlations among the test statistics into account. Both the Hommel and Rom procedures are uniform improvements over the Hochberg procedure, but are more complicated to use (particularly the former). Power calculations show that they both offer only marginal improvements.

A Step-Up Multiple Test Procedure

Abstract We consider the problem of simultaneously testing k ≥ 2 hypotheses on parameters θ1, …, θk . In a typical application the θ's may be a set of contrasts, for instance, a set of orthogonal contrasts among population means or a set of differences between k treatment means and a standard treatment mean. It is assumed that least squares estimators 1, …, k are available that are jointly normally distributed with a common variance (known up to a scalar, namely the error variance σ 2) and a common known correlation. An independent χ 2-distributed unbiased estimator of σ 2 is also assumed to be available. We propose a step-up multiple test procedure for this problem which tests the t statistics for the k hypotheses in order starting with the least significant one and continues as long as an acceptance occurs. (By contrast, the step-down approach, which is usually used, starts with the most significant and continues as long as a rejection occurs.) Critical constants required by this step-up procedure to co...