Nonparametric Behrens-Fisher Problem Research Articles

Group sequential methods for the Mann-Whitney parameter.

Late phase clinical trials are occasionally planned with one or more interim analyses to allow for early termination or adaptation of the study. While extensive theory has been developed for the analysis of ordered categorical data in terms of the Wilcoxon-Mann-Whitney test, there has been comparatively little discussion in the group sequential literature on how to provide repeated confidence intervals and simple power formulas to ease sample size determination. Dealing more broadly with the nonparametric Behrens-Fisher problem, we focus on the comparison of two parallel treatment arms and show that the Wilcoxon-Mann-Whitney test, the Brunner-Munzel test, as well as a test procedure based on the log win odds, a modification of the win ratio, asymptotically follow the canonical joint distribution. In addition to developing power formulas based on these results, simulations confirm the adequacy of the proposed methods for a range of scenarios. Lastly, we apply our methodology to the FREEDOMS clinical trial (ClinicalTrials.gov Identifier: NCT00289978) in patients with relapse-remitting multiple sclerosis.

Ranking procedures for repeated measures designs with missing data: Estimation, testing and asymptotic theory.

We develop purely nonparametric methods for the analysis of repeated measures designs with missing values. Hypotheses are formulated in terms of purely nonparametric treatment effects. In particular, data can have different shapes even under the null hypothesis and therefore, a solution to the nonparametric Behrens-Fisher problem in repeated measures designs will be presented. Moreover, global testing and multiple contrast test procedures as well as simultaneous confidence intervals for the treatment effects of interest will be developed. All methods can be applied for the analysis of metric, discrete, ordinal, and even binary data in a unified way. Extensive simulation studies indicate a satisfactory control of the nominal type-I error rate, even for small sample sizes and a high amount of missing data (up to 30%). We apply the newly developed methodology to a real data set, demonstrating its application and interpretation.

Directional-sum test for nonparametric Behrens-Fisher problem with applications to the dietary intervention trial.

For a nonparametric Behrens-Fisher problem, a directional-sum test is proposed based on division-combination strategy. A one-layer wild bootstrap procedure is given to calculate its statistical significance. We conduct simulation studies with data generated from lognormal, t and Laplace distributions to show that the proposed test can control the type I error rates properly and is more powerful than the existing rank-sum and maximum-type tests under most of the considered scenarios. Applications to the dietary intervention trial further show the performance of the proposed test.

The nonparametric Behrens-Fisher problem in partially complete clustered data.

In randomized trials or observational studies involving clustered units, the assumption of independence within clusters is not practical. Existing parametric or semiparametric methods assume specific dependence structures within a cluster. Furthermore, parametric model assumptions may not even be realistic when data are measured in a nonmetric scale as commonly happens, for example, in quality-of-life outcomes. In this paper, nonparametric effect-size measures for clustered data that allow meaningful and interpretable probabilistic comparisons of treatments or intervention programs will be introduced. The dependence among observations within a cluster can be arbitrary. Point estimators along with their asymptotic properties for computing confidence intervals and performing hypothesis test will be discussed. Small sample approximations that retain some of the optimal asymptotic behaviors will be presented. In our setup, some clusters may involve observations coming from both intervention groups (referred to as complete clusters), while others may contain observations from one group only (referred to as incomplete clusters). In deriving the asymptotic theories, we do not impose any relation in the rate of divergence of the numbers of complete and incomplete clusters. Simulations show favorable performance of the methods for arbitrary combinations of complete and incomplete clusters. The developed nonparametric methods are illustrated using data from a randomized trial of indoor wood smoke reduction to improve asthma symptoms and a cluster-randomized trial for smokingcessation.

The nonparametric Behrens-Fisher problem with dependent replicates.

Purely nonparametric methods are developed for general two-sample problems in which each experimental unit may have an individual number of possibly correlated replicates. In particular, equality of the variances, or higher moments, of the distributions of the data is not assumed, even under the null hypothesis of no treatment effect. Thus, a solution for the so-called nonparametric Behrens-Fisher problem is proposed for such models. The methods are valid for metric, count, ordered categorical, and even dichotomous data in a unified way. Point estimators of the treatment effects as well as their asymptotic distributions will be studied in detail. For small sample sizes, the distributions of the proposed test statistics are approximated using Satterthwaite-Welch-type t-approximations. Extensive simulation studies show favorable performance of the new methods, in particular, in small sample size situations. A real data set illustrates the application of the proposed methods.

Multiplication-combination tests for incomplete paired data.

We consider statistical procedures for hypothesis testing of real valued functionals of matched pairs with missing values. In order to improve the accuracy of existing methods, we propose a novel multiplication combination procedure. Dividing the observed data into dependent (completely observed) pairs and independent (incompletely observed) components, it is based on combining separate results of adequate tests for the two sub data sets. Our methods can be applied for parametric as well as semiparametric and nonparametric models and make use of all available data. In particular, the approaches are flexible and can be used to test different hypotheses in various models of interest. This is exemplified by a detailed study of mean- as well as rank-based approaches under different missingness mechanisms with different amount of missing data. Extensive simulations show that in most considered situations, the proposed procedures are more accurate than existing competitors particularly for the nonparametric Behrens-Fisher problem. A real data set illustrates the application of the methods.

Nonparametric MANOVA in meaningful effects

Multivariate analysis of variance (MANOVA) is a powerful and versatile method to infer and quantify main and interaction effects in metric multivariate multi-factor data. It is, however, neither robust against change in units nor meaningful for ordinal data. Thus, we propose a novel nonparametric MANOVA. Contrary to existing rank-based procedures, we infer hypotheses formulated in terms of meaningful Mann–Whitney-type effects in lieu of distribution functions. The tests are based on a quadratic form in multivariate rank effect estimators, and critical values are obtained by bootstrap techniques. The newly developed procedures provide asymptotically exact and consistent inference for general models such as the nonparametric Behrens–Fisher problem and multivariate one-, two-, and higher-way crossed layouts. Computer simulations in small samples confirm the reliability of the developed method for ordinal and metric data with covariance heterogeneity. Finally, an analysis of a real data example illustrates the applicability and correct interpretation of the results.

A non-parametric maximum test for the Behrens–Fisher problem

ABSTRACTNon-normality and heteroscedasticity are common in applications. For the comparison of two samples in the non-parametric Behrens–Fisher problem, different tests have been proposed, but no single test can be recommended for all situations. Here, we propose combining two tests, the Welch t test based on ranks and the Brunner–Munzel test, within a maximum test. Simulation studies indicate that this maximum test, performed as a permutation test, controls the type I error rate and stabilizes the power. That is, it has good power characteristics for a variety of distributions, and also for unbalanced sample sizes. Compared to the single tests, the maximum test shows acceptable type I error control.

Permutation-based inference for the AUC: A unified approach for continuous and discontinuous data.

We investigate rank-based studentized permutation methods for the nonparametric Behrens-Fisher problem, that is, inference methods for the area under the ROC curve. We hereby prove that the studentized permutation distribution of the Brunner-Munzel rank statistic is asymptotically standard normal, even under the alternative. Thus, incidentally providing the hitherto missing theoretical foundation for the Neubert and Brunner studentized permutation test. In particular, we do not only show its consistency, but also that confidence intervals for the underlying treatment effects can be computed by inverting this permutation test. In addition, we derive permutation-based range-preserving confidence intervals. Extensive simulation studies show that the permutation-based confidence intervals appear to maintain the preassigned coverage probability quite accurately (even for rather small sample sizes). For a convenient application of the proposed methods, a freely available software package for the statistical software R has been developed. A real data example illustrates the application.

Nparcomp: AnRSoftware Package for Nonparametric Multiple Comparisons and Simultaneous Confidence Intervals

One-way layouts, i.e., a single factor with several levels and multiple observations at each level, frequently arise in various fields. Usually not only a global hypothesis is of interest but also multiple comparisons between the different treatment levels. In most practical situations, the distribution of observed data is unknown and there may exist a number of atypical measurements and outliers. Hence, use of parametric and semiparametric procedures that impose restrictive distributional assumptions on observed samples becomes questionable. This, in turn, emphasizes the demand on statistical procedures that enable us to accurately and reliably analyze one-way layouts with minimal conditions on available data. Nonparametric methods offer such a possibility and thus become of particular practical importance. In this article, we introduce a new R package nparcomp which provides an easy and user-friendly access to rank-based methods for the analysis of unbalanced one-way layouts. It provides procedures performing multiple comparisons and computing simultaneous confidence intervals for the estimated effects which can be easily visualized. The special case of two samples, the nonparametric Behrens-Fisher problem, is included. We illustrate the implemented procedures by examples from biology and medicine.

Rank-based multiple test procedures and simultaneous confidence intervals

We study simultaneous rank procedures for unbalanced designs with independent observations. The hypotheses are formulated in terms of purely nonparametric treatment effects. In this context, we derive rank-based multiple contrast test procedures and simultaneous confidence intervals which take the correlation between the test statistics into account. Hereby, the individual test decisions and the simultaneous confidence intervals are compatible. This means, whenever an individual hypothesis has been rejected by the multiple contrast test, the corresponding simultaneous confidence interval does not include the null, i.e. the hypothetical value of no treatment effect. The procedures allow for testing arbitrary purely nonparametric multiple linear hypotheses (e.g. many-to-one, all-pairs, changepoint, or even average comparisons). We do not assume homogeneous variances of the data; in particular, the distributions can have different shapes even under the null hypothesis. Thus, a solution to the multiple nonparametric Behrens-Fisher problem is presented in this unified framework.

A studentized permutation test for the nonparametric Behrens-Fisher problem in paired data

We consider nonparametric ranking methods for matched pairs, whose distributions can have different shapes even under the null hypothesis of no treatment effect. Although the data may not be exchangeable under the null, we investigate a permutation approach as a valid procedure for finite sample sizes. In particular, we derive the limit of the studentized permutation distribution under alternatives, which can be used for the construction of $(1-\alpha)$-confidence intervals. Simulation studies show that the new approach is more accurate than its competitors. The procedures are illustrated using a real data set.

Resampling methods for the nonparametric and generalized Behrens-Fisher problems

Defining a location parameter as a generalization of the median, a robust test is proposed for (a) the nonparametric Behrens-Fisher problem, where the underlying distributions may have different scales and could be skewed, and (b) the generalized Behrens-Fisher problem, where the distributions may even have different shapes. We propose to bootstrap a signed rank statistic based on differences of sample values and derive rigorous bootstrap central limit theorems for its probabilistic justification, allowing for the so-called m-out-of-n bootstrap. The location parameter of interest is the pseudo-median of the distribution of the difference between a control measurement and an observation from the treatment group. It reduces to (a) the shift in the two sample location model and (b) the difference between the centers of symmetry in the nonparametric Behrens-Fisher model, under the additional assumption that the distributions are symmetric. Due to its importance for applications, we also extend our results to an ANOVA design where each treatment is compared with the control group. Finally, we compare our test with competitors on the basis of theory as well as simulation studies. It turns out that our approach yields a substantial improvement for distributions close to the generalized extreme value type, which makes it attractive for applications in engineering as well as finance. Several heteroscedastic data sets from electrical engineering, astro physics, energy research, analytical chemistry and psychology are used to illustrate our solution.

Ranking procedures for matched pairs with missing data — Asymptotic theory and a small sample approximation

Nonparametric methods for matched pairs with data missing completely at random are considered. It is not assumed that the observations are coming from distribution functions belonging to a certain parametric or semi-parametric family. In particular, the distributions can have different shapes under the null hypothesis. Hence, the so-called nonparametric Behrens–Fisher problem for matched pairs with missing data is considered. Moreover, a new approach for confidence intervals for nonparametric effects is presented. In particular, no restriction on the ratio of the number of complete and incomplete cases is required to derive the asymptotic results. Simulations show that for arbitrary settings of complete data and missing values, the resulting confidence intervals maintain the pre-assigned coverage probability quite accurately. Regarding the power, none of the proposed tests is uniformly superior to the other. A real data set illustrates the application.

Two sample tests for the nonparametric Behrens–Fisher problem with clustered data

In this paper, we consider the nonparametric Behrens–Fisher problem with cluster-correlated data. A class of weighted Mann–Whitney test statistics is introduced and studied. In particular, a comparison with other recent testing procedures for related problems is provided. The new tests are valid when the distributions do not have the same scales and/or shapes under the null hypothesis. A general class of weighted U-statistics for clustered data, encompassing the Mann–Whitney statistic, is also introduced. A simulation studies the type I error robustness and the power of the new and of some recently proposed procedures. This study shows that the incorporation of appropriate weights can greatly improve the power of the test. A real data example illustrates the use of the tests.

Testing and estimation of purely nonparametric effects in repeated measures designs

The several sample case of the so-called nonparametric Behrens–Fisher problem in repeated measures designs is considered. That is, even under the null hypothesis, the marginal distribution functions in the different groups may have different shapes, and are not assumed to be equal. Moreover, the continuity of the marginal distribution functions is not required so that data with ties and, particularly, ordered categorical data are covered by this model. A multiple relative treatment effect is defined which can be estimated by using the mid-ranks of the observations within pairwise samples. The asymptotic distribution of this estimator is derived, along with a consistent estimator of its asymptotic covariance matrix. In addition, a multiple contrast test and related simultaneous confidence intervals for the relative marginal effects are derived and compared to rank-based Wald-type and ANOVA-type statistics. Simulations show that the ANOVA-type statistic and the multiple contrast test appear to maintain the pre-assigned level of the test quite accurately (even for rather small sample sizes) while the Wald-type statistic leads, as expected, to somewhat liberal decisions. Regarding the power, none of the statistics is uniformly superior. A real data set illustrates the application.

Rank methods for the analysis of clustered data in diagnostic trials

Methods for comparing areas under receiver operating characteristic curves usually depend on the assumption of independence between diseased and nondiseased units in the trial. However, if several parts of the same subject have to be classified as diseased or nondiseased, such observations are no longer independent. This situation is referred to as clustered data. First ideas for the analysis of such data are based on the theory of U-statistics. The idea of the multivariate nonparametric Behrens–Fisher problem is extended to clustered data and to factorial designs. ANOVA-type statistics are suggested for both small and moderate sample sizes. They are evaluated in a simulation study and applied to a real-data example.

A studentized permutation test for the non-parametric Behrens–Fisher problem

For the non-parametric Behrens–Fisher problem a permutation test based on the studentized rank statistic of Brunner and Munzel is proposed. This procedure is applicable to count or ordered categorical data. By applying the central limit theorem of Janssen, it is shown that the asymptotic permutational distribution of this test statistic is a standard normal distribution. For very small and very different sample sizes, frequently occurring in medical and biological applications, an extensive simulation study suggests that this permutation test works well for data from several underlying distributions. The proposed test is applied to data from a clinical trial.

The Chen–Luo test in case of heteroscedasticity

Chen and Luo (2004, Comm. Statist. Simulation Comput. 33, 1007–1020) presented a modification of the Wilcoxon–Mann–Whitney test that can be less conservative than the original test. The results shown here indicate that the possible power advantage of the modification is solely due to its being less conservative. Because the modified statistic is variance adjusted using a variance estimator based on the empirical data situation independent of the null hypothesis, the test may be useful in case of heteroscedasticity. However, it is demonstrated here that the modified test can be much more anticonservative than the Wilcoxon–Mann–Whitney test when the population means are identical, but the population variances differ. Therefore, the modified test cannot be recommended for the nonparametric Behrens–Fisher problem. Possible approaches in that case are the Brunner–Munzel test and Cliff's method.

Adjusting O'Brien's Test to Control Type I Error for the Generalized Nonparametric Behrens–Fisher Problem

O'Brien (1984, Biometrics 40, 1079-1087) introduced a simple nonparametric test procedure for testing whether multiple outcomes in one treatment group have consistently larger values than outcomes in the other treatment group. We first explore the theoretical properties of O'Brien's test. We then extend it to the general nonparametric Behrens-Fisher hypothesis problem when no assumption is made regarding the shape of the distributions. We provide conditions when O'Brien's test controls its error probability asymptotically and when it fails. We also provide adjusted tests when the conditions do not hold. Throughout this article, we do not assume that all outcomes are continuous. Simulations are performed to compare the adjusted tests to O'Brien's test. The difference is also illustrated using data from a Parkinson's disease clinical trial.