Closed Testing Procedure Research Articles

Closed Testing Procedures for Pairwise Comparisons in Multi-Sample Models

Under the assumptions of the continuous distribution and equality of variance, we consider multiple comparisons tests for the differences among mean responses in k samples. Closed testing procedures are superior to the Tukey-Kramar type multiple tests which are single-step procedures in many cases. The closed testing procedure more powerful than the Tukey-Welsch test and the REGW test is proposed under the usual normality. Furthermore the nonparametric closed testing procedure is discussed based on ranks. Last the proposed nonparametric closed testing procedure is illustrated with data regarding sarcoidosis from the assay of serum angiotensin-converting enzyme (SACE).

Consonant closed likelihood ratio test procedures with application to dose–response study

Coherence and Consonance are two important concepts in multiple testing procedures (MTP). Closed test is a well-known coherent hypothesis testing procedure, which is not necessarily consonant. In this paper, we propose two consonant closed likelihood ratio tests which are compared with the step-down Dunnett test (SD) and Dunnett-Tamhane step-up test (SU) in several aspects. Simulation and dose-response study examples show that the new procedures have certain advantages over the SD and SU test. For example, the rejection region of our new procedures is larger than that of the SD test particularly when most of the null hypotheses are false. The new procedures control the family-wise error rate (FWER) strongly without the equal correlation assumption, which is a necessary condition for the SU test. In terms of computing effort, the new procedures require similar moderate computation for critical constants as the SU test and they follow the same steps as any closed test procedures. We also provide guideline in applying consonance adjustment in multivariate analysis with mixed model and repeated measurements.

Multicenter, phase II, placebo‐controlled, double‐blind, randomized study of aprepitant in Japanese patients receiving high‐dose cisplatin

Aprepitant is a new neurokinin-1 (NK(1) ) receptor antagonist developed as a treatment for chemotherapy-induced nausea and vomiting (CINV). To evaluate the efficacy and safety of aprepitant used in combination with standard therapy (granisetron and dexamethasone), we conducted a multicenter, phase II, placebo-controlled, double-blind, randomized study in Japanese cancer patients who received cancer chemotherapy including cisplatin (≥70mg/m(2) ). Aprepitant was administered for 5days. A total of 453 patients were enrolled. In the three study groups, (i) standard therapy, (ii) aprepitant 40/25mg (40mg on day 1 and 25mg on days 2-5) and (iii) aprepitant 125/80mg (125mg on day 1 and 80mg on days 2-5), the percentage of patients with complete response (no emesis and no rescue therapy) was 50.3% (75/149 subjects), 66.4% (95/143 subjects) and 70.5% (103/146 subjects), respectively. This shows that efficacy was significantly higher in the aprepitant 40/25mg and 125/80mg groups than in the standard therapy group (χ(2) test [closed testing procedure]: P=0.0053 and P=0.0004, respectively) and highest in the aprepitant 125/80mg group. The delayed phase efficacy (days 2-5) was similar to the overall phase efficacy (days 1-5), indicating that aprepitant is effective in the delayed phase when standard therapy is not very effective. In terms of safety, aprepitant was generally well tolerated in Japanese cancer patients. (ClinicalTrials.gov number, NCT00212602.)

Maternal Low-Level Lead Exposure and Fetal Growth

BackgroundLimited epidemiologic studies have examined the association between maternal low-level lead exposure [blood lead (PbB) < 10 μg/dL] and fetal growth.ObjectiveWe examined whether maternal low-level lead exposure is associated with decreased fetal growth.MethodsWe linked New York State Heavy Metals Registry records of women who had PbB measurements with birth certificates to identify 43,288 mother–infant pairs in upstate New York in a retrospective cohort study from 2003 through 2005. We used multiple linear regression with fractional polynomials and logistic regression to relate birth weight, preterm delivery, and small for gestational age to PbB levels, adjusting for potential confounders. We used a closed-test procedure to identify the best fractional polynomials for PbB among 44 combinations.ResultsWe found a statistically significant association between PbB (square root transformed) and birth weight. Relative to 0 μg/dL, PbBs of 5 and 10 μg/dL were associated with an average of 61-g and 87-g decrease in birth weight, respectively. The adjusted odds ratio for PbBs between 3.1 and 9.9 μg/dL (highest quartile) was 1.04 [95% confidence interval (CI), 0.89–1.22] for preterm delivery and 1.07 (95% CI, 0.93–1.23) for small for gestational age, relative to PbBs ≤ 1 μg/dL (lowest quartile). No clear dose–response trends were evident when all of the quartiles were assessed.ConclusionsLow-level PbB was associated with a small risk of decreased birth weight with a supralinear dose–response relationship, but was not related to preterm birth or small for gestational age. The results have important implications regarding maternal PbB.

Shortcuts for Locally Consonant Closed Test Procedures

The closed testing principle provides a general and simple framework to construct powerful multiple test procedures for k elementary null hypotheses while controlling the familywise error rate in the strong sense. However, the closed testing principle has the disadvantage of leading to the evaluation of O(2k) intersection hypotheses. Multiple test procedures based on the closed testing principle may thus require substantial computational efforts. Consonant closed test procedures for unrestricted hypotheses have the advantage of rejecting at least one elementary null hypothesis whenever the global null hypothesis is rejected and thus admit shortcuts of size k. If the elementary null hypotheses are restricted by logical constraints, the closure of common tests, such as max-t or min-p tests, may not be consonant in the usual sense. In this article we introduce a weaker consonance property, denoted as local consonance, and show that many closed test procedures with restricted hypotheses satisfy this condition. We describe a general algorithm to construct related shortcuts and illustrate the results with several applications.

Apixaban for Reduction In Stroke and Other ThromboemboLic Events in Atrial Fibrillation (ARISTOTLE) trial: Design and rationale

Atrial fibrillation (AF) is associated with increased risk of stroke that can be attenuated with vitamin K antagonists (VKAs). Vitamin K antagonist use is limited, in part, by the high incidence of complications when patients' international normalized ratios (INRs) deviate from the target range. The primary objective of ARISTOTLE is to determine if the factor Xa inhibitor, apixaban, is noninferior to warfarin at reducing the combined endpoint of stroke (ischemic or hemorrhagic) and systemic embolism in patients with AF and at least 1 additional risk factor for stroke. We have randomized 18,206 patients from over 1,000 centers in 40 countries. Patients were randomly assigned in a 1:1 ratio to receive apixaban or warfarin using a double-blind, double-dummy design. International normalized ratios are monitored and warfarin (or placebo) is adjusted aiming for a target INR range of 2 to 3 using a blinded, encrypted point-of-care device. Minimum treatment is 12 months, and maximum expected exposure is 4 years. Time to accrual of at least 448 primary efficacy events will determine treatment duration. The key secondary objectives are to determine if apixaban is superior to warfarin for the combined endpoint of stroke (ischemic or hemorrhagic) and systemic embolism, and for all-cause death. These will be tested after the primary objective using a closed test procedure. The noninferiority boundary is 1.38; apixaban will be declared noninferior if the 95% CI excludes the possibility that the primary outcome rate with apixaban is >1.38 times higher than with warfarin. ARISTOTLE will determine whether apixaban is noninferior or superior to warfarin in preventing stroke and systemic embolism; whether apixaban has particular benefits in the warfarin-naïve population; whether it reduces the combined rate of stroke, systemic embolism, and death; and whether it impacts bleeding.

On stepwise control of the generalized familywise error rate

A classical approach for dealing with a multiple testing problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of at least one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of a procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In this article, a unified approach is presented for deriving the k-FWER controlling procedures. We first generalize the well-known closure principle in the context of the FWER to the case of controlling the k-FWER. Then, we discuss how to derive the k-FWER controlling stepup procedures based on marginal p-values using this principle. We show that, under certain conditions, generalized closed testing procedures can be reduced to stepup procedures, and any stepup procedure is equivalent to a generalized closed testing procedure. Finally, we generalize the well-known Hommel procedure in two directions, and show that any generalized Hommel procedure is equivalent to a generalized closed testing procedure with the same critical values.

P62 An unexpected evolution of a pelvic osteolysis and cognitive disorders in a man treated for a renal-cell carcinoma

In metastatic breast cancer, nab-paclitaxel has been shown to significantly increase progression-free survival compared with solvent-based paclitaxel. The GeparSepto (GBG 69) trial assessed whether weekly nab-paclitaxel could increase the proportion of patients achieving pathological complete response compared with weekly solvent-based paclitaxel, both followed by epirubicin plus cyclophosphamide as neoadjuvant treatment.In a phase 3 randomised trial, we enrolled patients with previously untreated unilateral or bilateral primary invasive breast cancer and randomly assigned them in a 1:1 ratio using dynamic allocation and Pocock minimisation by breast cancer subtype, Ki67 and SPARC expression. Patients were treated for 12 weeks with either intravenous nab-paclitaxel 150 mg/m2 (after study amendment, 125 mg/m2) on days 1, 8, and 15 for four 3-week cycles, or solvent-based intravenous paclitaxel 80 mg/m2 on days 1, 8, and 15 for four 3-week cycles. Taxane treatment was followed in both groups by intravenous epirubicin 90 mg/m2 plus intravenous cyclophosphamide 600 mg/m2 on day 1 for four 3-week cycles. Patients with HER2-positive tumours received concurrent trastuzumab 6 mg/kg (loading dose 8 mg/kg) and pertuzumab 420 mg (loading dose 840 mg) on day 1 of every 3-week cycle. Trastuzumab and pertuzumab were given every 3 weeks concomitantly with chemotherapy for all cycles. This report is the final analysis of the primary endpoint, pathological complete response (ypT0 ypN0), analysed for all patients who started treatment (modified intention to treat). We used a closed test procedure to test for non-inferiority, with the nab-paclitaxel group calculated as non-inferior to the solvent-based paclitaxel group if the lower 95% CI for the OR was above 0·858 (OR equivalent to pathological complete response [33%] minus a 10% non-inferiority margin [3·3%]; 29·7%). We planned to test for superiority only in case of a positive non-inferiority test, using an α of 0·05. Safety was assessed in all patients who received study drug. The trial is registered with ClinicalTrials.gov, number NCT01583426.Between July 30, 2012, and Dec 23, 2013, we randomly assigned 1229 women, of whom 1206 started treatment (606 with nab-paclitaxel and 600 with solvent-based paclitaxel). The nab-paclitaxel dose was reduced after enrolment of 464 participants to 125 mg/m2 due to increased treatment discontinuation and sensory neuropathy in this group. Pathological complete response occurred more frequently in the nab-paclitaxel group (233 [38%, 95% CI 35–42] patients) than in the solvent-based paclitaxel group (174 [29%, 25–33] patients; OR 1·53, 95% CI 1·20–1·95; unadjusted p=0·00065). The incidence of grade 3–4 anaemia (13 [2%] of 605 patients in the nab-paclitaxel group vs four [1%] of patients in the solvent-based paclitaxel group; p=0·048) and peripheral sensory neuropathy grade 3–4 (63 [10%] patients receiving any nab-paclitaxel dose; 31 [8%] of patients starting with 125 mg/m2 and 32 [15%] of patients starting with 150 mg/m2; vs 16 [3%] in the solvent-based paclitaxel group, p<0·001) was significantly higher for nab-paclitaxel than for solvent-based paclitaxel. Overall, 283 (23%) patients were noted to have at least one serious adverse event (based on study drug received), 156 (26%) in the nab-paclitaxel group and 127 (21%) in the solvent-based paclitaxel group (p=0·057). There were three deaths (during epirubicin plus cyclophosphamide treatment) in the nab-paclitaxel group (due to sepsis, diarrhoea, and accident unrelated to the trial) versus one in the solvent-based paclitaxel group (during paclitaxel treatment; cardiac failure).Substituting solvent-based paclitaxel with nab-paclitaxel significantly increases the proportion of patients achieving a pathological complete response rate after anthracycline-based chemotherapy. These results might lead to an exchange of the preferred taxane, solvent-based paclitaxel, for nab-paclitaxel in therapy for primary breast cancer.Celgene, Roche.

A Randomized, Controlled, Dose-ranging Study Investigating Single Doses of GW406381, Naproxen Sodium, or Placebo in Patients With Acute Pain After Third Molar Tooth Extraction

To evaluate the efficacy and tolerability of 4 doses of GW406381, a cyclooxygenase-2 inhibitor, compared with placebo in a standard model of acute inflammatory pain. This randomized, double-blind, placebo-controlled, single-center study compared single doses of GW406381 (10 to 70 mg) or naproxen sodium 550 mg with placebo in patients after extraction of 2 or more partially bony impacted third molar teeth. A total of 300 patients were randomized (50 per group). The primary efficacy variable was the pain relief intensity difference score at each time point, which was calculated as the sum of the pain intensity difference and pain relief categorical scores at each time point. Each treatment was compared with placebo at each time point using an ordered hierarchical approach with closed testing procedures and last observation carried forward imputation methods. Pain relief intensity differences from placebo were statistically significant beginning at 1.5 hours postdosing for GW406381 70 and 50 mg and at 2-hour postdosing for GW406381 25 and 10 mg. The median time to onset of analgesia was 71 minutes for GW406381 50 mg, 72 minutes for GW406381 70 mg, and 36 minutes for naproxen. The median duration of analgesia was 5.9 hours for GW406381 50 mg, 7.9 hours for GW406391 70 mg, and 11.3 hours for naproxen. All treatments were well tolerated. GW406381 50 and 70 mg demonstrated clinically meaningful analgesia in this acute pain setting, although the onset of analgesia was greater than 1 hour.

Alternative Confidence Regions for Bonferroni‐Based Closed‐Testing Procedures that are not Alpha‐Exhaustive

This article complements the results in Guilbaud (Biometrical Journal 2008; 50:678-692). Simultaneous confidence regions were derived in that article that correspond to any given multiple testing procedure (MTP) in a fairly large class of consonant closed-testing procedures based on marginal p-values and weighted Bonferroni tests for intersection hypotheses. This class includes Holm's MTP, the fixed-sequence MTP, gatekeeping MTPs, fallback MTPs, multi-stage fallback MTPs, and recently proposed MTPs specified through a graphical representation and associated rejection algorithm. More general confidence regions are proposed in this article. These regions are such that for certain underlying MTPs which are not alpha-exhaustive, they lead to confidence assertions that may be sharper than rejection assertions for some rejected null hypotheses H when not all Hs are rejected, which is not the case with the previously proposed regions. In fact, various alternative confidence regions may be available for such an underlying MTP. These results are shown through an extension of the previous direct arguments (without invoking the partitioning principle), and under the same general setup; so for instance, estimated quantities and marginal confidence regions are not restricted to be of any particular kinds/dimensions. The relation with corresponding confidence regions of Strassburger and Bretz (Statistics in Medicine 2008; 27:4914-4927) is described. The results are illustrated with fallback and parallel-gatekeeping MTPs.

Testing Marginal Homogeneity Against Stochastic Order in Multivariate Ordinal Data

Many assessment instruments used in the evaluation of toxicity, safety, pain, or disease progression consider multiple ordinal endpoints to fully capture the presence and severity of treatment effects. Contingency tables underlying these correlated responses are often sparse and imbalanced, rendering asymptotic results unreliable or model fitting prohibitively complex without overly simplistic assumptions on the marginal and joint distribution. Instead of a modeling approach, we look at stochastic order and marginal inhomogeneity as an expression or manifestation of a treatment effect under much weaker assumptions. Often, endpoints are grouped together into physiological domains or by the body function they describe. We derive tests based on these subgroups, which might supplement or replace the individual endpoint analysis because they are more powerful. The permutation or bootstrap distribution is used throughout to obtain global, subgroup, and individual significance levels as they naturally incorporate the correlation among endpoints. We provide a theorem that establishes a connection between marginal homogeneity and the stronger exchangeability assumption under the permutation approach. Multiplicity adjustments for the individual endpoints are obtained via stepdown procedures, while subgroup significance levels are adjusted via the full closed testing procedure. The proposed methodology is illustrated using a collection of 25 correlated ordinal endpoints, grouped into six domains, to evaluate toxicity of a chemical compound.

Adaptive designs for confirmatory clinical trials

Adaptive designs play an increasingly important role in clinical drug development. Such designs use accumulating data of an ongoing trial to decide how to modify design aspects without undermining the validity and integrity of the trial. Adaptive designs thus allow for a number of possible adaptations at midterm: Early stopping either for futility or success, sample size reassessment, change of population, etc. A particularly appealing application is the use of adaptive designs in combined phase II/III studies with treatment selection at interim. The expectation has arisen that carefully planned and conducted studies based on adaptive designs increase the efficiency of the drug development process by making better use of the observed data, thus leading to a higher information value per patient.In this paper we focus on adaptive designs for confirmatory clinical trials. We review the adaptive design methodology for a single null hypothesis and how to perform adaptive designs with multiple hypotheses using closed test procedures. We report the results of an extensive simulation study to evaluate the operational characteristics of the various methods. A case study and related numerical examples are used to illustrate the key results. In addition we provide a detailed discussion of current methods to calculate point estimates and confidence intervals for relevant parameters.

Clinical trials: healing of erosive oesophagitis with dexlansoprazole MR, a proton pump inhibitor with a novel dual delayed‐release formulation – results from two randomized controlled studies

Dexlansoprazole MR employs a dual delayed-release delivery system that extends drug exposure and prolongs pH control compared with lansoprazole. To assess the efficacy and safety of dexlansoprazole MR in healing erosive oesophagitis (EO). Patients in two identical double-blind, randomized controlled trials (n = 4092) received dexlansoprazole MR 60 or 90 mg or lansoprazole 30 mg once daily. Week 8 healing was assessed using a closed testing procedure--first for non-inferiority, then superiority, vs. lansoprazole. Secondary endpoints included week 4 healing and week 8 healing in patients with moderate-to-severe disease (Los Angeles Classification grades C and D). Life-table and crude rate analyses were performed. Symptoms and tolerability were assessed. Dexlansoprazole MR achieved non-inferiority to lansoprazole, allowing testing for superiority. Using life-table analysis, dexlansoprazole MR healed 92-95% of patients in individual studies vs. 86-92% for lansoprazole; the differences were not statistically significant (P > 0.025). Using crude rate analysis, dexlansoprazole MR 90 mg was superior to lansoprazole in both studies and 60 mg was superior in one study. Week 4 healing was > 64% with all treatments in both studies. In an integrated analysis of 8-week healing in patients with moderate-to-severe EO, dexlansoprazole MR 90 mg was superior to lansoprazole. All treatments effectively relieved symptoms and were well tolerated. Dexlansoprazole MR is highly effective in healing EO and offers benefits over lansoprazole, particularly in moderate-to-severe disease.

A recycling framework for the construction of Bonferroni‐based multiple tests

In this paper we describe Bonferroni-based multiple testing procedures (MTPs) as strategies to split and recycle test mass. Here, 'test mass' refers to (parts of) the nominal level alpha at which the family-wise error rate is controlled. Briefly, test mass is split between different null hypotheses, and whenever a null hypothesis is rejected, the part of alpha allocated to it may be recycled to the testing of other hypotheses. These recycling MTPs are closed testing procedures based on raw p-values associated with testing the individual null hypotheses, and the class of such MTPs includes, for example, serial and parallel gatekeeping, fallback and Holm procedures. Graphical displays and a concise algebraic notation are provided for such MTPs. This recycling approach has pedagogical advantages and may facilitate the tailoring of MTPs for different purposes.

Efficacy and tolerability of MK-0974 (telcagepant), a new oral antagonist of calcitonin gene-related peptide receptor, compared with zolmitriptan for acute migraine: a randomised, placebo-controlled, parallel-treatment trial

Calcitonin gene-related peptide (CGRP) probably has a role in migraine pathophysiology, and antagonism of its receptors might provide treatment without the vasoconstrictor effects of triptans. We aimed to assess the clinical profile of MK-0974 (telcagepant), an orally bioavailable antagonist of CGRP receptor. In a randomised, parallel-treatment, placebo-controlled, double-blind, trial at 81 sites in the Europe and the USA, adults with migraine diagnosed by International Headache Society criteria treated moderate or severe attacks with either oral telcagepant 150 mg or 300 mg, zolmitriptan 5 mg, or placebo. The five co-primary endpoints were pain freedom, pain relief, or absence of photophobia, phonophobia, or nausea at 2 h after treatment. Analysis was by the full analysis set and multiplicity was controlled for with a step-down closed-testing procedure. This trial is registered with ClinicalTrials.gov, number NCT00442936. 1380 patients were randomly assigned to receive telcagepant 150 mg (n=333) or 300 mg (354), zolmitriptan (345), or placebo (348). Telcagepant 300 mg was more effective than placebo for pain freedom (95 [27%] of 353 patients vs 33 [10%] of 343 [p<0.0001]), pain relief (194 [55%] of 353 vs 95 [28%] of 343 [p<0.0001]), and absences of phonophobia (204 [58%] of 353 vs 126 [37%] of 342 [p<0.0001]), photophobia (180 [51%] of 353 vs 99 [29%] of 342 [p<0.0001]), and nausea (229 [65%] of 352 vs 189 [55%] of 342 [p=0.0061]). Efficacy of telcagepant 300 mg and zolmitriptan 5 mg were much the same, and both were more effective than telcagepant 150 mg. Adverse events were recorded for 31% taking telcagepant 150 mg, 37% taking telcagepant 300 mg, 51% taking zolmitriptan 5 mg, and 32% taking placebo. Telcagepant 300 mg is effective as an acute treatment for migraine with efficacy comparable to that of zolmitriptan 5 mg, but with fewer associated adverse effects. Merck Research Laboratories.

Comparison of several testing strategies for combination drug efficacy trials based on the closure principle

The author discusses three multiple testing procedures for identifying the minimum efficacious doses in a balanced factorial combination drug trial. All of these procedures utilize the closed testing principle, and hence strongly control the overall error rate and satisfy the coherence property, that is, if a hypothesis is retained then any hypothesis implied by it is also retained. While coherence is an essential requirement for any multiple testing procedure, consonance is a highly desirable characteristic. In the considered settings if a testing procedure is consonant then it always provides a set of all minimum efficacious combinations as a result, otherwise, it may lead to ambiguity. Although the coherence property is satisfied for any closed testing procedure and thus, does not depend on the test used for an individual hypothesis, whether the considered procedures satisfy the consonance property depends entirely on the nature of the test statistic. The author identifies the consonant and non-consonant procedures among the presented procedures and discusses possible drawbacks of non-consonant procedures with respect to combination drug efficacy trials. Additional properties of these procedures are assessed by simulations.

Simultaneous Confidence Regions Corresponding to Holm's Step‐Down Procedure and Other Closed‐Testing Procedures

Holm's (1979) step-down multiple-testing procedure (MTP) is appealing for its flexibility, transparency, and general validity, but the derivation of corresponding simultaneous confidence regions has remained an unsolved problem. This article provides such confidence regions. In fact, simultanenous confidence regions are provided for any MTP in the class of short-cut consonant closed-testing procedures based on marginal p -values and weighted Bonferroni tests for intersection hypotheses considered by Hommel, Bretz and Maurer (2007). In addition to Holm's MTP, this class includes the fixed-sequence MTP, recently proposed gatekeeping MTPs, and the fallback MTP. The simultaneous confidence regions are generally valid if underlying marginal p -values and corresponding marginal confidence regions (assumed to be available) are valid. The marginal confidence regions and estimated quantities are not assumed to be of any particular kinds/dimensions. Compared to the rejections made by the MTP for the family of null hypotheses H under consideration, the proposed confidence regions provide extra free information. In particular, with Holm's MTP, such extra information is provided: for all nonrejected H s, in case not all H s are rejected; or for certain (possibly all) H s, in case all H s are rejected. In case not all H s are rejected, no extra information is provided for rejected H s. This drawback seems however difficult to overcome. Illustrations concerning clinical studies are given.

Multiple rank-based testing for ordered alternatives with incomplete data

Alvo and Cabilio [Alvo, M., Cabilio, P., 1995b. Testing ordered alternatives in the presence of incomplete data. J. Amer. Statist. Assoc. 90, 1015–1024] extended the Page and Jonckheere tests when randomized blocks are incomplete. Here we use the step-down testing procedure of Marcus et al. [Marcus, R., Peritz, E., Gabriel, K.R., 1976. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63, 655–660] to identify which treatments differ, and illustrate the method by an example.

Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni‐based closed tests

We consider the problem of simultaneously testing multiple one-sided null hypotheses. Single-step procedures, such as the Bonferroni test, are characterized by the fact that the rejection or non-rejection of a null hypothesis does not take the decision for any other hypothesis into account. For stepwise test procedures, such as the Holm procedure, the rejection or non-rejection of a null hypothesis may depend on the decision of other hypotheses. It is well known that stepwise test procedures are by construction more powerful than their single-step counterparts. This power advantage, however, comes only at the cost of increased difficulties in constructing compatible simultaneous confidence intervals for the parameters of interest. For example, such simultaneous confidence intervals are easily obtained for the Bonferroni method, but surprisingly hard to derive for the Holm procedure. In this paper, we discuss the inherent problems and show that ad hoc solutions used in practice typically do not control the pre-specified simultaneous confidence level. Instead, we derive simultaneous confidence intervals that are compatible with a certain class of closed test procedures using weighted Bonferroni tests for each intersection hypothesis. The class of multiple test procedures covered in this paper includes gatekeeping procedures based on Bonferroni adjustments, fixed sequence procedures, the simple weighted or unweighted Bonferroni procedure by Holm and the fallback procedure. We illustrate the results with a numerical example.

Cumulative Statistical Power for Hierarchical Comparisons to Evaluate Two Combination Drug Doses

According to the regulatory requirements for multiple-dose factorial designs, a combination drug must have confirmatory evidence for being more effective than each component drug alone. An incomplete factorial design may be employed to evaluate some combination drugs because of resource constraints and priorities. In this paper, we compare the powers for four different patterns of sample size allocations in two incomplete factorial designs, with two combination drug doses and fixed total sample size. A hierarchical closed testing procedure is employed for the treatment comparisons of interest as a method to control type I error for multiple comparisons. The overall cumulative powers of contradicting all null hypotheses at a particular stage of the hierarchy and all preceding stages are determined by simulation for the respective stages of the hierarchy. The purpose is to identify the allocation of sample size so as to enable better power in a hierarchical evaluation of comparisons.