Behrens-Fisher Research Articles

On some treatment analyses

Fawzy et al. [1] and Jamigorn et al. [2] illustrate systemic incorrect statistical analysis, in their use of two-sample t tests, paired or independent. These tests compare normal population means assuming unknown equal variances. The central limit theorem fully justiWes normality for inferences on means, but unknown variances need not be equal. This makes two-sample t tests, together with analysis of variance, unsuitable for general mean inferences. Since Linnik [3] has shown that two-sample t-statistics for the diVerence of means based on their suYcient statistics do not exist when variances are unknown, this Behrens–Fisher problem is not avoided by uselessly [4] testing for the equality of variances. Nor is it bypassed with rank tests such as the Mann–Whitney test. Being a comparison of distributions, these rank tests say nothing about means if signiWcant, and are biased [5] to one side in a two-sided test. Tsakok [6] has solved the Behrens–Fisher problem of comparing the means of normal distributions with unknown variances at exact signiWcant levels and shown that the Tsakok solution is more eVective in detecting signiWcant mean diVerences even with unknown equal variances. There is an indication [7] that the Tsakok solution applies to dependent samples. Its exposition [8] is available. The software GSP implements the Tsakok technique, and is used to compare means at 0.02 signiWcance level (one signiWcant Wgure) per pair. For Table 2 [1], there are signiWcant mean diVerences between the enoxaparin group and placebo in their estimated gestational age (EGA) at birth, EGA at spontaneous vaginal delivery (SVD) and EGA at caesarean section. For Table 4 [1], there is signiWcant mean diVerence between placebo and the enoxaparin group in their EGA at loss. For Table 2 [2], there is signiWcant mean diVerence between the acupressure group and the vitamin B6 group in their weight at the end of the trial. After the attention given to the data, they deserve correct analysis. The Tsakok technique is extended to the non-parametric two-sample problem with the article by Tsakok [9] on constructing exact uniformly most powerful unbiased tests, superseding above rank tests. The Tsakok articles are reprinted [10] with further results.

On the Behrens–Fisher problem: A globally convergent algorithm and a finite-sample study of the Wald, LR and LM tests

In this paper we provide a provably convergent algorithm for the multivariate Gaussian Maximum Likelihood version of the Behrens–Fisher Problem. Our work builds upon a formulation of the log-likelihood function proposed by Buot and Richards [5]. Instead of focusing on the first order optimality conditions, the algorithm aims directly for the maximization of the log-likelihood function itself to achieve a global solution. Convergence proof and complexity estimates are provided for the algorithm. Computational experiments illustrate the applicability of such methods to high-dimensional data. We also discuss how to extend the proposed methodology to a broader class of problems. We establish a systematic algebraic relation between the Wald, Likelihood Ratio and Lagrangian Multiplier Test (W≥LR≥LM) in the context of the Behrens–Fisher Problem. Moreover, we use our algorithm to computationally investigate the finite-sample size and power of the Wald, Likelihood Ratio and Lagrange Multiplier Tests, which previously were only available through asymptotic results. The methods developed here are applicable to much higher dimensional settings than the ones available in the literature. This allows us to better capture the role of high dimensionality on the actual size and power of the tests for finite samples.

Multiple solutions to the likelihood equations in the Behrens–Fisher problem

The Behrens–Fisher problem concerns testing the equality of the means of two normal populations with possibly different variances. The null hypothesis in this problem yields a statistical model for which the likelihood function may have more than one local maximum. We show that if the null hypothesis is valid then the probability of multimodality of the likelihood function converges to zero when both sample sizes tend to infinity. Additional results include a finite-sample bound on the probability of multimodality under the null and asymptotics for the probability of multimodality under the alternative.

A Revisit to the Behrens–Fisher Problem: Comparison of Five Test Methods

We revisit the well-known Behrens–Fisher problem and apply a newly developed ‘Computational Approach Test’ (CAT) to test the equality of two population means where the populations are assumed to be normal with unknown and possibly unequal variances. An advantage of the CAT is that it does not require the explicit knowledge of the sampling distribution of the test statistic. The CAT is then compared with three widely accepted tests—Welch–Satterthwaite test (WST), Cochran–Cox test (CCT), ‘Generalized p-value’ test (GPT)—and a recently suggested test based on the jackknife procedure, called Singh–Saxena–Srivastava test (SSST). Further, model robustness of these five tests are studied when the data actually came from t-distributions, but wrongly perceived as normal ones. Our detailed study based on a comprehensive simulation indicate some interesting results including the facts that the GPT is quite conservative, and the SSST is not as good as it has been claimed in the literature. To the best of our knowledge, the trends observed in our study have not been reported earlier in the existing literature.

Hotelling's one-sample and two-sample [formula omitted] tests and the multivariate Behrens–Fisher problem under nonnormality

This paper provides a powerful method for obtaining an asymptotic expansion for the expectation of a function of the normalized sample mean vector N 1 / 2 U ¯ and the sample covariance matrix S U under general distributions, without using the Edgeworth expansion of N - 1 / 2 ∑ i = 1 N U i and N - 1 / 2 ∑ i = 1 N vech ( U i U i ′ - Σ ) . It is shown that asymptotic expansions of the nonnull distributions of some multivariate test statistics on mean vectors under general distributions are derived in a unified way, by finding the differential operator associated with the expectation according to situations under consideration. Unlike Kano [1995. An asymptotic expansion of the distribution of Hotelling's T 2 -statistic under general distributions. Amer. J. Math. Manage. Sci. 15, 317–341] and Fujikoshi [1997. An asymptotic expansion for the distribution of Hotelling's T 2 -statistic under nonnormality. J. Multivariate Anal. 61, 187–193], our routine for getting asymptotic expansions is to collect some patterned derivatives, without constructing the Edgeworth expansion of some basic statistics. In this sense, our approach seems to be more convenient, at least, for Hotelling's T 2 -type statistics and other related statistics on mean vectors.

Simultaneous testing of multiple hypotheses using generalized p-values

In the context of simultaneously testing many hypotheses, Benjamini and Hochberg [1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B (Methodological) 57, 289–300] propose a procedure that guarantees that the false discovery rate ( FDR) will be less than or equal to a specified value. Here, the FDR is the expected value of the ratio of the number of incorrectly rejected hypotheses and the total number of rejected hypotheses. To our knowledge, all of the existing research assumes that a usual p-value is available for each hypothesis. However, in circumstances when nuisance parameters are present, the usual p-values may not be free of the nuisance parameters. We develop a simultaneous testing procedure to control the FDR for the problem of simultaneously testing many Behrens–Fisher problems. Our multiple testing procedure, based on generalized p-values [Tsui, K.-W., Weerahandi, S., 1989. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Assoc. 84(406), 602–607], is then illustrated with an application to data from a microarray experiment.

A Test for the Equality of Parameters for Separate Regression Models in the Presence of Heteroskedasticity

Testing for the equality of regression coefficients across two regressions is a problem considered by analysts in a variety of fields. If the variances of the errors of the two regressions are not equal, then it is known that the standard large sample F-test used to test the equality of the coefficients is compromised by the fact that its actual size can differ substantially from the stated level of significance in small samples. This article addresses this problem and borrows from the literature on the Behrens-Fisher problem to provide some simple modifications of the large sample test which allows one to better control the probability of committing a Type I error. Empirical evidence is presented which indicates that the suggested modifications provide tests which are superior to well-known alternative tests over a wide range of the parameter space.

On Selecting Tests for Equality of Two Normal Mean Vectors

The conventional approach for testing the equality of two normal mean vectors is to test first the equality of covariance matrices, and if the equality assumption is tenable, then use the two-sample Hotelling T 2 test. Otherwise one can use one of the approximate tests for the multivariate Behrens–Fisher problem. In this article, we study the properties of the Hotelling T 2 test, the conventional approach, and one of the best approximate invariant tests (Krishnamoorthy & Yu, 2004) for the Behrens–Fisher problem. Our simulation studies indicated that the conventional approach often leads to inflated Type I error rates. The approximate test not only controls Type I error rates very satisfactorily when covariance matrices were arbitrary but was also comparable with the T 2 test when covariance matrices were equal.

Exact solutions to the Behrens–Fisher Problem: Asymptotically optimal and finite sample efficient choice among

The problem of testing the equality of two normal means when variances are not known is called the Behrens–Fisher Problem. This problem has three known exact solutions, due, respectively, to Chapman, to Prokof’yev and Shishkin, and to Dudewicz and Ahmed. Each procedure has level alpha and power beta when the means differ by a given amount delta, both set by the experimenter. No single-sample statistical procedures can make this guarantee. The most recent of the three procedures, that of Dudewicz and Ahmed, is asymptotically optimal. We review the procedures, and then compare them with respect to both asymptotic efficiency and also (using simulation) in finite samples. Of these exact procedures, based on finite-sample comparisons the Dudewicz–Ahmed procedure is recommended for practical use.

A Sequential Approach to the Behrens-Fisher Problem

This article reexamines the scope of a two-stage methodology for constructing a fixed-width confidence interval for the difference between two independent normal population means. We address directly the problem in a Behrens-Fisher situation, that is, when the variances of the two populations are both unknown and unequal. However, we show that the proposed methodology remains valid when the two variances are equal but unknown. We apply a two-stage procedure to the Behrens-Fisher situation using two approaches. In the first approach, we work with the differences of each observation in the first sample with all observations in the second sample. In the second approach, we work with the differences of paired observations from samples 1 and 2. Next, to overcome the problem of oversampling associated with the two-stage procedures, we apply the bootstrap methodology to modify the two-stage procedures. These methods, with and without bootstrapping, are compared with the help of a large-scale simulation study.

Distributional properties for the generalized p-value for the Behrens–Fisher problem

The generalized p-value method introduced by Tsui and Weerahandi [1989. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Assoc. 84 (406), 602–607] has been successfully used to provide small sample solutions for many hypothesis testing problems when nuisance parameters are present. Simulation studies show that generalized p-values have similar distributional properties as ordinary p-values. It is desirable to study theoretical properties of generalized p-values. Given a sample d, let p ( d ) be the generalized p-value for the Behrens–Fisher problem of testing the difference of two independent normal distribution means with possibly unequal distributional variances, as given in Tsui and Weerahandi [1989. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Assoc. 84 (406), 602–607]. We derive a closed form expression to show that, for small samples, the probability P ( p ( d ) ⩽ r ) is approximately less than or equal to r, for 0 ⩽ r ⩽ 0.5 .

Rank methods for the analysis of clustered data in diagnostic trials

Methods for comparing areas under receiver operating characteristic curves usually depend on the assumption of independence between diseased and nondiseased units in the trial. However, if several parts of the same subject have to be classified as diseased or nondiseased, such observations are no longer independent. This situation is referred to as clustered data. First ideas for the analysis of such data are based on the theory of U-statistics. The idea of the multivariate nonparametric Behrens–Fisher problem is extended to clustered data and to factorial designs. ANOVA-type statistics are suggested for both small and moderate sample sizes. They are evaluated in a simulation study and applied to a real-data example.

A studentized permutation test for the non-parametric Behrens–Fisher problem

For the non-parametric Behrens–Fisher problem a permutation test based on the studentized rank statistic of Brunner and Munzel is proposed. This procedure is applicable to count or ordered categorical data. By applying the central limit theorem of Janssen, it is shown that the asymptotic permutational distribution of this test statistic is a standard normal distribution. For very small and very different sample sizes, frequently occurring in medical and biological applications, an extensive simulation study suggests that this permutation test works well for data from several underlying distributions. The proposed test is applied to data from a clinical trial.

The Chen–Luo test in case of heteroscedasticity

Chen and Luo (2004, Comm. Statist. Simulation Comput. 33, 1007–1020) presented a modification of the Wilcoxon–Mann–Whitney test that can be less conservative than the original test. The results shown here indicate that the possible power advantage of the modification is solely due to its being less conservative. Because the modified statistic is variance adjusted using a variance estimator based on the empirical data situation independent of the null hypothesis, the test may be useful in case of heteroscedasticity. However, it is demonstrated here that the modified test can be much more anticonservative than the Wilcoxon–Mann–Whitney test when the population means are identical, but the population variances differ. Therefore, the modified test cannot be recommended for the nonparametric Behrens–Fisher problem. Possible approaches in that case are the Brunner–Munzel test and Cliff's method.

A Bootstrap Test for the Analysis of Microarray Experiments with a Very Small Number of Replications

In microarray studies it is common that the number of replications (i.e. the sample size) is small and that the distribution of expression values differs from normality. In this situation, permutation and bootstrap tests may be appropriate for the identification of differentially expressed genes. However, unlike bootstrap tests, permutation tests are not suitable for very small sample sizes, such as three per group. A variety of different bootstrap tests exists. For example, it is possible to adjust the data to have a common mean before the bootstrap samples are drawn. For small significance levels, which can occur when a large number of genes is investigated, the original bootstrap test, as well as a bootstrap test suggested for the Behrens-Fisher problem, have no power in cases of very small sample sizes. In contrast, the modified test based on adjusted data is powerful. Using a Monte Carlo simulation study, we demonstrate that the difference in power can be huge. In addition, the different tests are illustrated using microarray data.

A note on Hall's triple sampling procedure: A multiple sample second order sequential analogue of the Behrens–Fisher problem

In many statistical problems, the variances of the populations cannot be assumed to be equal. These inhomogeneity problems are often more difficult to handle than the corresponding homogeneity problems. In sequential estimation, this often means that only first order sequential procedures are available in the statistics literature for inhomogeneity problems. The purpose of this paper is to illustrate by using the classical Behrens–Fisher problem how to construct a second order sequential procedure using the batch sampling idea of Hall [Ann. Statist. 9 (1981) 1229–1238]; the cost of assuming variance inhomogeneity even when the two variances are equal turns out to be very limited. The approach of this paper can readily be applied to many other inhomogeneous problems.

Three Approximate Solutions to the Multivariate Behrens–Fisher Problem

ABSTRACT This article provides three approximate solutions to the multivariate Behrens–Fisher problem: the F statistic, the Bartlett, as well as the modified Bartlett corrected statistics. Empirical results indicate that the F statistic outperforms the other two and five existing procedures. The modified Bartlett corrected statistic is also very competitive.

Partial Sequential Tests for Generalized Behrens-Fisher Problem

The present article provides two partially sequential nonparametric tests (see Wolfe, 1977a) for testing the identity of two location parameters without making any assumptions on the shapes of the distributions of the underlying populations. The proposed tests use 0–1 scores while defining the stopping variable, and are based on the partial sequential placement statistics introduced by Orban and Wolfe (1980). Some asymptotic results related to the proposed procedures are studied. Competitive tests based on fixed sample sizes are also found. Further various exact and asymptotic performance measures of the tests are investigated. Recommended by Edit Gombay

Adjusting O'Brien's Test to Control Type I Error for the Generalized Nonparametric Behrens–Fisher Problem

O'Brien (1984, Biometrics 40, 1079-1087) introduced a simple nonparametric test procedure for testing whether multiple outcomes in one treatment group have consistently larger values than outcomes in the other treatment group. We first explore the theoretical properties of O'Brien's test. We then extend it to the general nonparametric Behrens-Fisher hypothesis problem when no assumption is made regarding the shape of the distributions. We provide conditions when O'Brien's test controls its error probability asymptotically and when it fails. We also provide adjusted tests when the conditions do not hold. Throughout this article, we do not assume that all outcomes are continuous. Simulations are performed to compare the adjusted tests to O'Brien's test. The difference is also illustrated using data from a Parkinson's disease clinical trial.

On some osteoporosis analyses

The articles by Reginster et al. [1] and Bhattoa et al. [2] are flawed, because the analysis of variance and the twosample t -tests or variations to it are inappropriate. The Central Limit Theorem justifies the normality assumed by these methods for mean inferences. The problem is that the two above methods also assume that the unknown variances of the groups of interest are equal. Since unknown variances need not be equal, both the analysis of variance and the two-sample t -tests are not generally applicable to comparing means. Futilely [3] testing for the equality of variances does not remove this problem. Nor does avoiding normality with nonparametric rank tests such as the KruskalWallis test. Being a comparison of distributions, the latter says nothing about the means or any specific moment of the distributions if significant, and is biased to one side [4] in a two-sided test. Tsakok [5] has solved this Behrens-Fisher problem of exactly comparing the means of normal distributions from samples in its generalized form, simultaneously showing that it is more effective in detecting significant mean differences, even with unknown equal variances. Its exposition [6] is available. The software GSP implements the Tsakok technique. Applying it to compare means at .02 (one significant figure) significance level per comparison, GSP finds significant differences in calcium of Table 5 [1] between placebo and the strontium ranelate (SR) 1 g/day group at M12 and M24. For Table 3 [1], GSP finds significant differences of BMD percentage change in femoral neck between placebo and SR 125 mg/day, in femoral neck between placebo and SR 500 mg/day and in total hip between placebo and SR 500 mg/day. For Table 3 [2], GSP finds significant differences in the intention-to-treat C4 between estradiol and placebo at months 3, 6, 9 and 12. For on-treatment total complement activity (CH50/ml) estradiol of Table 3 [2], there are significant differences between baseline and months 9 and 12. For on-treatment C3 estradiol (Table 3 [2]), there are also significant differences between baseline and months 9 and 12. These were overlooked. The care acquiring the data means that they deserve correct analysis. The article by Tsakok [7] on constructing exact, unconditional uniformly most powerful unbiased tests extends the Tsakok technique to the nonparametric problem of comparing samples, superseding rank tests, the chi-squared test or the Fisher’s ‘‘exact’’ test (it is neither exact nor unconditional). There is an indication [8] that the Tsakok technique applies to dependent samples. The Tsakok articles are reprinted [9] with further results.