Step-down Procedure Research Articles

Identifying effective and/or safe doses by stepwise confidence intervals for ratios.

Typical randomized clinical dose-finding studies consist of the comparison of several doses of a drug versus a placebo. Interest lies in estimating relevant doses among those under investigation for efficacy and safety variables, such as the minimum effective dose or the maximum safe dose (or estimating both doses simultaneously). Step-down procedures have been proposed for comparing the standardized differences of the dose groups against placebo. In this paper we consider the ratio of population means and propose stepwise confidence intervals for these ratios. These confidence intervals do not require multiplicity adjustments and yield the same decisions as the associated test procedures. In addition, several power concepts are investigated within the present framework. The results allow sample size determination in the design phase of a study for the probability of estimating correctly the dose of interest. Auxiliary results of a numerical study show the range of application of these methods.

Nonparametric Identification of the Minimum Effective Dose

In clinical dose-finding trials the identification of the minimum effective dose is important. Recently, a nonparametric step-down procedure based upon Wilcoxon rank sum tests was proposed for identifying the minimum effective dose. However, the recently introduced modified Baumgartner-Weiss-Schindler statistic is very powerful and, consequently, may be used in the step-down procedure instead of the Wilcoxon rank sum. The two step-down closed testing procedures based upon the Wilcoxon and the modified Baumgartner-Weiss-Schindler statistic, respectively, were compared in a Monte Carlo study. According to simulation results, the procedure based upon the modified Baumgartner-Weiss-Schindler statistic is more powerful than that based upon the Mann-Whitney or Wilcoxon test for identifying the minimum effective dose.

AC-DC current transfer step-up and step-down calibration and uncertainty calculation

New AC-DC current transfer standards are built for the current range from 0.3 mA to 20 A and the frequency range from 10 Hz to 100 kHz. A three-dimensional PTB-multijunction thermal converter (MJTC) with a heater resistance of 27 /spl Omega/ is the basic standard for the AC-DC current transfer at currents from 20 to 50 mA. At lower currents, planar MJTCs (PMJTCs) with heater resistances from 90 to 960 /spl Omega/ are used in a step-down procedure. Coaxial current shunts in parallel with PMJTCs extend the current range up to 20 A. The standard measurement uncertainties are in the range from 0.2 /spl mu/A/A at 25 mA and 1 kHz to 50 /spl mu/A/A at 20 A and 100 kHz. The high sensitivity of the PMJTC allows bigger steps in the step-up calibration and some smaller steps between these steps. Calibration of all these steps leads to an overdetermined system to check for the validity of the assumptions made in the step-up procedure. This results in reduced uncertainties at the different current levels compared to the old method.

Testing for Treatment Effects on Subsets of EndpointsThe view presented in this paper are those of authors and not necessary representing those of the U.S. Food and Drug Administration

Multiple endpoints are tested to assess an overall treatment effect and also to identify which endpoints or subsets of endpoints contributed to treatment differences. The conventional p-value adjustment methods, such as single-step, step-up, or step-down procedures, sequentially identify each significant individual endpoint. Closed test procedures can also detect individual endpoints that have effects via a step-by-step closed strategy. This paper proposes a global-based statistic for testing an a priori number, say, r of the k endpoints, as opposed to the conventional approach of testing one (r = 1) endpoint. The proposed test statistic is an extension of the single-step p-value-based statistic based on the distribution of the smallest p-value. The test maintains strong control of the FamilyWise Error (FWE) rate under the null hypothesis of no difference in any (sub)set of r endpoints among all possible combinations of the k endpoints. After rejecting the null hypothesis, the individual endpoints in the sets that are rejected can be tested further, using a univariate test statistic in a second step, if desired. However, the second step test only weakly controls the FWE. The proposed method is illustrated by application to a psychosis data set.

A dose finding procedure based on the multivariate likelihood ratio test

This paper presents a step-down procedure based on the multivariate likelihood ratio test to identify the minimal effective dose in dose-response studies. The admissibility of this procedure and some competing procedures are discussed. Simulation results show that this procedure is preferable to the other procedures in the literature.

Stepwise procedures for the identification of minimum effective dose with unknown variances

Hsu and Berger (J. Amer. Statist. Assoc. 50 (1999) 468) proposed a stepwise confidence interval procedure for identifying the minimum effective dose in dose-response studies under homoscedasticity. In practice, homogeneity of variance is often in doubt. In this paper, we extend Hsu and Berger's stepwise confidence interval procedure to the case of heteroscedasticity (i.e., inequality of variances across dosages). In addition, we show that the new procedure can properly control the family-wise error rate. We illustrate our procedure with a real example. Finally, we investigate the performance of our procedure and a step-down test procedure based on Welch's approximate t-test via a simulation study. Our procedure is shown, empirically, to perform similar to Welch's procedure.

Multiple Test Procedures for Identifying the Minimum Effective and Maximum Safe Doses of a Drug

We address the problem of determining the therapeutic window of a drug by finding its minimum effective and maximum safe doses (MINED and MAXSD). The MINED is the lowest dose that exceeds the mean efficacy of the zero dose by a specified threshold, and the MAXSD is the highest dose that does not exceed the mean toxicity of the zero dose by a specified threshold. Step-down multiple test procedures are proposed to identify the MINED and MAXSD assuming a bivariate normal model. These procedures control the type I familywise error probability of declaring any ineffective dose as effective or any unsafe dose as safe at a prespecified level α. A new multivariate t-distribution is introduced whose critical points are required to implement the exact normal theory procedures. Because these critical points depend on the unknown correlation coefficient between the efficacy and safety variables, the Bonferroni method is proposed as an alternative, which amounts to separately testing for efficacy and safety, each at type I familywise error rate of α/2. The bootstrap versions of the exact normal theory procedures provide an approximate way to jointly test for efficacy and safety without the knowledge of the correlation coefficient, as well as to relax the bivariate normality assumption. The Bonferroni and bootstrap procedures are compared in a simulation study. It is shown that significant power gains are achieved by jointly testing for both efficacy and safety using bootstrap procedures. Coded data from an arthritis drug trial are analyzed to illustrate the procedures.

Some Results on False Discovery Rate in Stepwise multiple testing procedures

The concept of false discovery rate (FDR) has been receiving increasing attention by researchers in multiple hypotheses testing. This paper produces some theoretical results on the FDR in the context of stepwise multiple testing procedures with dependent test statistics. It was recently shown by Benjamini and Yekutieli that the Benjamini–Hochberg step-up procedure controls the FDR when the test statistics are positively dependent in a certain sense. This paper strengthens their work by showing that the critical values of that procedure can be used in a much more general stepwise procedure under similar positive dependency. It is also shown that the FDR-controlling Benjamini–Liu step-down procedure originally developed for independent test statistics works even when the test statistics are positively dependent in some sense. An explicit expression for the FDR of a generalized stepwise procedure and an upper bound to the FDR of a step-down procedure are obtained in terms of probability distributions of ordered components of dependent random variables before establishing the main results.

Multiple hypotheses testing and expected number of type I. errors

The performance of multiple test procedures with respect to error control is an old issue. Assuming that all hypotheses are true we investigate the behavior of the expected number of type I errors (ENE) as a characteristic of certain multiple tests controlling the familywise error rate (FWER) or the false discovery rate (FDR) at a prespecified level. We derive explicit formulas for the distribution of the number of false rejections as well as for the ENE for single-step, step-down and step-up procedures based on independent $p$-values. Moreover, we determine the corresponding asymptotic distributions of the number of false rejections as well as explicit formulae for the ENE if the number of hypotheses tends to infinity. In case of FWER-control we mostly obtain Poisson distributions and in one case a geometric distribution as limiting distributions; in case of FDR control we obtain limiting distributions which are apparently not named in the literature. Surprisingly, the ENE is bounded by a small number regardless of the number of hypotheses under consideration. Finally, it turns out that in case of dependent test statistics the ENE behaves completely differently compared to the case of independent test statistics.

Multiplicity, directional (Type III) errors, and the Null Hypothesis.

L. V. Jones and J. W. Tukey (2000) pointed out that the usual 2-sided, equal-tails null hypothesis test at level alpha can be reinterpreted as simultaneous tests of 2 directional inequality hypotheses, each at level alpha/2, and that the maximum probability of a Type I error is alpha/2 if the truth of the null hypothesis is considered impossible. This article points out that in multiple testing with familywise error rate controlled at alpha, the directional error rate (assuming all null hypotheses are false) is greater than alpha/2 and can be arbitrarily close to alpha. Single-step, step-down, and step-up procedures are analyzed, and other error rates, including the false discovery rate, are discussed. Implications for confidence interval estimation and hypothesis testing practices are considered.

A more powerful step-up procedure for controlling the false discovery rate under independence

Recently, Benjamini and Hochberg (J. Roy. Statist. Soc. B 57 (1995) 289) and Benjamini and Liu (J. Statist. Plann. Inference 82 (1999) 163) proposed step-up and step-down multiple hypotheses testing procedures, respectively, for controlling the false discovery rate (FDR) when all the test statistics are independent. In this paper, we propose another step-up procedure such that the FDR is still controlled at a specified significance level. Based on a simulation study, the new procedure is more powerful than both existing step-up and step-down procedures, especially when most of the hypotheses are false.

Multiple Test Procedures for Identifying the Maximum Safe Dose

We consider dose response studies for safety assessment of crop protection compounds and drugs, and we offer a hypothesis testing approach for identifying the maximum dose level that is guaranteed to be safe with preassigned confidence. The focus is on step-down (SD) multiple test procedures for identifying the maximum safe dose. We propose two classes of contrasts among the dose means as test statistics for these procedures: pairwise contrasts (PC) and Helmert contrasts (HC). The first procedure (SD2PC) consists of a sequence of ordinary t tests and is thus easy to apply, but it can have very low power for certain step response functions. The second procedure (SD1HC) does not suffer from this drawback, but it requires a weak monotonicity assumption for its mathematical validity. The powers of SD2PC and SD1HC are studied via Monte Carlo simulation. We recommend SD2PC for linear response functions and SD1HC for step response functions. The procedures are illustrated by applying them to data from an aquatic toxicity laboratory experiment conducted to assess the safe level of a pesticide.

Relationships among Edaphic, Climatic, and Vegetation Conditions at Release Sites and Aphthona nigriscutis Population Density

Abstract The role of site conditions on the success of leafy spurge control by Aphthona nigriscutis is poorly understood. Our objective was to determine the relationships among the climatic and edaphic conditions of A. nigriscutis release sites, A. nigriscutis population density, leafy spurge cover and density, and grass cover. We sampled 13 field sites in eastern Montana where A. nigriscutis had been established for 6 to 8 years. Sampling was done in June and July of 1998 along a randomly determined transect from the center of each release point to densely infested leafy spurge. Data were analyzed by step-down regression procedures using A. nigriscutis density, leafy spurge cover and density, cover of grass, forbs, litter, and bare ground as the dependent variables. Independent variables included vegetative and insect data sampled as well as site conditions. Average annual precipitation was the only edaphic or climatic site characteristic that influenced A. nigriscutis density; the beetle population density increased as precipitation increased. There was a negative association between A. nigriscutis and number of leafy spurge flowering stems and leafy spurge cover. Grass, litter, and bare ground cover increased as A. nigriscutis numbers increased. Within the range of site conditions sampled in this study, successful establishment and population expansion of A. nigriscutis appears to be related to the cover of leafy spurge, grass, forbs other than leafy spurge, litter, and bare ground. This information may aid in site selection for release of this important biological control agent or in creating conditions for optimum establishment.

Least Favorable Parameter Configurations for a Step-down Subset Selection Procedure

Given a k-dimensional vector X, k≥2, the range-type statistic with J⊆I≡{1, …, k}, plays an important role in stepwise subset selection as well as in testing whether a<?tlb> prespecified subset of k populations exclusively consists of good ones. Although in previous papers least favorable parameter configurations (LFC's) for this statistic, which are worth knowing for the calculations of critical values, have been already shown to be from a small finite subset of the parameter space, further reduction has been conjectured. Under the assumption of a log-concave and symmetric Lebesgue density with shift parameter, it is proved that in many cases the LFC can be uniquely given or, at least, found among only a few candidates. The resulting step-down selection procedure will be illustrated for data from a balanced incomplete block design.

Optimally weighted, fixed sequence and gatekeeper multiple testing procedures

Abstract We consider a class of closed multiple test procedures indexed by a fixed weight vector. The class includes the Holm weighted step-down procedure, the closed method using the weighted Fisher combination test, and the closed method using the weighted version of Simes’ test. We show how to choose weights to maximize average power, where “average power” is itself weighted by importance assigned to the various hypotheses. Numerical computations suggest that the optimal weights for the multiple test procedures tend to certain asymptotic configurations. These configurations offer numerical justification for intuitive multiple comparisons methods, such as downweighting variables found insignificant in preliminary studies, giving primary variables more emphasis, gatekeeping test strategies, pre-determined multiple testing sequences, and pre-determined sequences of families of tests. We establish that such methods fall within the envelope of weighted closed testing procedures, thus providing a unified view of fixed sequences, fixed sequences of families, and gatekeepers within the closed testing paradigm. We also establish that the limiting cases control the familywise error rate (or FWE), using well-known results about closed tests, along with the dominated convergence theorem.

Sample size determination in step-down and step-up multiple tests for comparing treatments with a control

Abstract We address the problem of sample size determination in multiple comparisons of k treatments with a control for step-down and step-up testing, assuming normal data and homogeneous variances. We define power as the probability of correctly rejecting all hypotheses for which the treatment vs. control difference exceeds a specified value. Our paper supplements papers by Hayter and Tamhane (J. Statist. Plann. Inference 27 (1991) 271–290) who solved the problem for one-sided comparisons using the step-down procedure and by Liu (J. Statist. Plann. Inference 62 (1997b) 255–261) who considered the two-sided case using the single-step method. We provide expressions that allow computer evaluation of the power and necessary sample sizes for one- and two-sided tests using either step-down or step-up procedures. Tables are given from which sample sizes to guarantee a specified power can be determined.

Current AC-DC transfer measurements in the microampere range

A relation of the AC-DC transfer differences contributing to the current AC-DC transfer difference of a current shunt in combination with a thermal transfer standard with amplified millivolt ranges is given. Two step-down procedures for measurement of ac current in the microamp range using a thermal transfer standard with amplified millivolt ranges together with current shunts are described. The results of measurements using the two methods are compared and found to be in good agreement with the predictions.

Social environment, behavior, and schistosomiasis in an urban population in the northeast of Brazil.

The objective of our study was to gain greater insight into the pattern of water contact in relation to schistosomiasis among residents of São Lourenço da Mata, a town in Pernambuco, a state in the Northeast of Brazil. We had two primary objectives: 1) to identify water contact activities that were more likely to produce infection and 2) to examine the socioeconomic factors behind water contact activities. Using a sample of persons 10-25 years old, we carried out a population-based case-control study to investigate the first objective, and a cross-sectional study for the second objective. We found that leisure water contact with flowing water (stream or river) was significantly associated with schistosomiasis. Variables showing a statistically significant association with leisure water contact were economic sector, income, and level of education of the head of the household; type of housing; possessions inside the house; type of domestic water supply; and method of excreta collection. We introduced these variables into a multivariate model to select the ones that were most closely associated with leisure water contact. We used a stepdown procedure, and two variables were retained in the final model: type of domestic water supply and type of housing. We concluded that a decrease in leisure water contact was associated with better socioeconomic conditions. Our results suggest that with the subjects we studied in São Lourenço da Mata there was a socioeconomic determination for leisure water contact. Our data highlight the importance of a broad and integrated approach in studying water contact activities and in implementing behavioral interventions for schistosomiasis prevention and control.

Selection of an Adaptive Test Statistic for Use with Multiple Comparison Analyses of Neuroimaging Data

Statistical analysis of neuroimages is commonly approached with intergroup comparisons made by repeated application of univariate or multivariate tests performed on the set of the regions of interest sampled in the acquired images. The use of such large numbers of tests requires application of techniques for correction for multiple comparisons. Standard multiple comparison adjustments (such as the Bonferroni) may be overly conservative when data are correlated and/or not normally distributed. Resampling-based step-down procedures that successfully account for unknown correlation structures in the data have recently been introduced. We combined resampling step-down procedures with the Minimum Variance Adaptive method, which allows selection of an optimal test statistic from a predefined class of statistics for the data under analysis. As shown in simulation studies and analysis of autoradiographic data, the combined technique exhibits a significant increase in statistical power, even for small sample sizes (n = 8, 9, 10).

Stepwise normal theory multiple test procedures controlling the false discovery rate

Abstract The false discovery rate (FDR), or the expected proportion of falsely rejected null hypotheses to rejected null hypotheses, has recently been proposed as an error rate that multiple testing procedures should in certain circumstances control. So far, only a step-up procedure for independent test statistics has been created explicitly to control the FDR ( Benjamini and Hochberg, 1995 ). In this paper, step-down and step-up procedures are described which asymptotically (as N→∞) control the FDR when the test statistics are the t statistics from consistent multivariate normal estimators of the tested parameters. Determination of the necessary critical constants for the normal theory procedures is achieved using numerical integration when the correlations are equal, or through simulation using the multivariate t distribution when the correlations are arbitrary. The critical constants of the normal theory procedures are compared to those of the Benjamini and Hochberg procedure under the normal assumption, and a large potential power increase is found. Simulation strongly supports the use of critical constants, obtained by an asymptotic argument, in small samples for as many as 30 tests. Adjusted FDR values can be found to quantify the evidence against a given hypothesis.