Unequal Covariances Research Articles

Two approaches to extend classical MANOVA tests to the unequal covariances case

This article presents two approaches to extend classical MANOVA tests to the unequal covariances case. The first approach is illustrated by extending the classical Wilks test, which is valid only when covariances are equal. Such tests will be based on exact probabilities of certain extreme regions. We will also show how tests numerically equivalent to the parametric bootstrap tests could be easily obtained without using any bootstrap sampling arguments, so that resulting p-values are also based on exact probabilities of well defined extreme regions. Being systematic approaches, by taking similar approaches, researchers should be able to derive generalized tests in MANCOVA, higher-way MANOVA, and in RM MANOVA under heteroscedasticity, in which the parametric bootstrap type approaches run into difficulties.

Quantitative Decision Under Unequal Covariances and Post-Treatment Variances: A Kidney Disease Application

Proof-of-concept (PoC) trials enable sponsors decide whether to continue or discontinue a compound’s development based on preliminary evidence of safety and efficacy. Many accounts exist on using quantitative approaches for this decision. Still, these accounts are devoid of quantitative decisions under unequal covariances and post-treatment variances for continuous endpoints. Filling this void is important because randomization to treatment groups only guarantees equal variances before treatment initiation. In this article, I show how unequal covariances and post-treatment variances affect the consider zone when using a three-outcome frequentist decision framework. Results herein suggest that the consider zone decreases with increase in correlation between baseline and post-baseline measurements for any sample size. Compared to decrease in consider zone under equal covariances and post-treatment variances, decrease in consider zone was greater under unequal covariances and post-treatment variances. The results suggest that ignoring a baseline covariate correlated with an outcome robs the study team of an opportunity to effectively minimize uncertain decision in PoC trial scoped to support quantitative decisions. Used for illustration is design of a PoC trial in kidney disease.

Longitudinal analysis of pre- and post-treatment measurements with equal baseline assumptions in randomized trials.

For continuous variables of randomized controlled trials, recently, longitudinal analysis of pre‐ and posttreatment measurements as bivariate responses is one of analytical methods to compare two treatment groups. Under random allocation, means and variances of pretreatment measurements are expected to be equal between groups, but covariances and posttreatment variances are not. Under random allocation with unequal covariances and posttreatment variances, we compared asymptotic variances of the treatment effect estimators in three longitudinal models. The data‐generating model has equal baseline means and variances, and unequal covariances and posttreatment variances. The model with equal baseline means and unequal variance–covariance matrices has a redundant parameter. In large sample sizes, these two models keep a nominal type I error rate and have high efficiency. The model with equal baseline means and equal variance–covariance matrices wrongly assumes equal covariances and posttreatment variances. Only under equal sample sizes, this model keeps a nominal type I error rate. This model has the same high efficiency with the data‐generating model under equal sample sizes. In conclusion, longitudinal analysis with equal baseline means performed well in large sample sizes. We also compared asymptotic properties of longitudinal models with those of the analysis of covariance (ANCOVA) and t‐test.

Spectral correlation functions of the sum of two independent complex Wishart matrices with unequal covariances

We compute the spectral statistics of the sum H of two independent complex Wishart matrices, each of which is correlated with a different covariance matrix. Random matrix theory enjoys many applications including sums and products of random matrices. Typically ensembles with correlations among the matrix elements are much more difficult to solve. Using a combination of supersymmetry, superbosonisation and bi-orthogonal functions we are able to determine all spectral k-point density correlation functions of H for arbitrary matrix size N. In the half-degenerate case, when one of the covariance matrices is proportional to the identity, the recent results by Kumar for the joint eigenvalue distribution of H serve as our starting point. In this case the ensemble has a bi-orthogonal structure and we explicitly determine its kernel, providing its exact solution for finite N. The kernel follows from computing the expectation value of a single characteristic polynomial. In the general non-degenerate case the generating function for the k-point resolvent is determined from a supersymmetric evaluation of the expectation value of k ratios of characteristic polynomials. Numerical simulations illustrate our findings for the spectral density at finite N and we also give indications how to do the asymptotic large-N analysis.

Dawai: AnRPackage for Discriminant Analysis with Additional Information

The incorporation of additional information into discriminant rules is receiving increasing attention as the rules including this information perform better than the usual rules. In this paper we introduce an R package called dawai, which provides the functions that allow to define the rules that take into account this additional information expressed in terms of restrictions on the means, to classify the samples and to evaluate the accuracy of the results. Moreover, in this paper we extend the results and definitions given in previous papers (Fernández, Rueda, and Salvador 2006, Conde, Fernández, Rueda, and Salvador 2012, Conde, Salvador, Rueda, and Fernández 2013) to the case of unequal covariances among the populations, and consequently define the corresponding restricted quadratic discriminant rules. We also define estimators of the accuracy of the rules for the general more than two populations case. The wide range of applications of these procedures is illustrated with two data sets from two different fields, i.e., biology and pattern recognition.

Analysis of covariance with pre‐treatment measurements in randomized trials: Comparison of equal and unequal slopes

In randomized trials, an analysis of covariance (ANCOVA) is often used to analyze post-treatment measurements with pre-treatment measurements as a covariate to compare two treatment groups. Random allocation guarantees only equal variances of pre-treatment measurements. We hence consider data with unequal covariances and variances of post-treatment measurements without assuming normality. Recently, we showed that the actual type I error rate of the usual ANCOVA assuming equal slopes and equal residual variances is asymptotically at a nominal level under equal sample sizes, and that of the ANCOVA with unequal variances is asymptotically at a nominal level, even under unequal sample sizes. In this paper, we investigated the asymptotic properties of the ANCOVA with unequal slopes for such data. The estimators of the treatment effect at the observed mean are identical between equal and unequal variance assumptions, and these are asymptotically normal estimators for the treatment effect at the true mean. However, the variances of these estimators based on standard formulas are biased, and the actual type I error rates are not at a nominal level, irrespective of variance assumptions. In equal sample sizes, the efficiency of the usual ANCOVA assuming equal slopes and equal variances is asymptotically the same as those of the ANCOVA with unequal slopes and higher than that of the ANCOVA with equal slopes and unequal variances. Therefore, the use of the usual ANCOVA is appropriate in equal sample sizes.

Analysis of covariance with pre-treatment measurements in randomized trials under the cases that covariances and post-treatment variances differ between groups

When primary endpoints of randomized trials are continuous variables, the analysis of covariance (ANCOVA) with pre-treatment measurements as a covariate is often used to compare two treatment groups. In the ANCOVA, equal slopes (coefficients of pre-treatment measurements) and equal residual variances are commonly assumed. However, random allocation guarantees only equal variances of pre-treatment measurements. Unequal covariances and variances of post-treatment measurements indicate unequal slopes and, usually, unequal residual variances. For non-normal data with unequal covariances and variances of post-treatment measurements, it is known that the ANCOVA with equal slopes and equal variances using an ordinary least-squares method provides an asymptotically normal estimator for the treatment effect. However, the asymptotic variance of the estimator differs from the variance estimated from a standard formula, and its property is unclear. Furthermore, the asymptotic properties of the ANCOVA with equal slopes and unequal variances using a generalized least-squares method are unclear. In this paper, we consider non-normal data with unequal covariances and variances of post-treatment measurements, and examine the asymptotic properties of the ANCOVA with equal slopes using the variance estimated from a standard formula. Analytically, we show that the actual type I error rate, thus the coverage, of the ANCOVA with equal variances is asymptotically at a nominal level under equal sample sizes. That of the ANCOVA with unequal variances using a generalized least-squares method is asymptotically at a nominal level, even under unequal sample sizes. In conclusion, the ANCOVA with equal slopes can be asymptotically justified under random allocation.

Sufficient Conditions for Torpid Mixing of Parallel and Simulated Tempering

We obtain upper bounds on the spectral gap of Markov chains constructed by parallel and simulated tempering, and provide a set of sufficient conditions for torpid mixing of both techniques. Combined with the results of Woodard, Schmidler and Huber (2009), these results yield a two-sided bound on the spectral gap of these algorithms. We identify a persistence property of the target distribution, and show that it can lead unexpectedly to slow mixing that commonly used convergence diagnostics will fail to detect. For a multimodal distribution, the persistence is a measure of how ``spiky'', or tall and narrow, one peak is relative to the other peaks of the distribution. We show that this persistence phenomenon can be used to explain the torpid mixing of parallel and simulated tempering on the ferromagnetic mean-field Potts model shown previously. We also illustrate how it causes torpid mixing of tempering on a mixture of normal distributions with unequal covariances in R^M, a previously unknown result with relevance to statistical inference problems. More generally, anytime a multimodal distribution includes both very narrow and very wide peaks of comparable probability mass, parallel and simulated tempering are shown to mix slowly.

Fisher information for a Gaussian random variable with unequal real and imaginary covariances: Derivation and application to beamforming

There are many physical situations in which a received signal may be modeled as a complex Gaussian random variable. One such situation occurs when an acoustic wave is strongly scattered as a result of propagation through a random medium such as the atmosphere or ocean. However, there are also many conditions under which this model fails to provide an adequate representation. A specific example is the geometric acoustics regime, in which diffraction and scattering by the medium are both weak and the variance of the phase fluctuations is much larger than the variance of the log-amplitude fluctuations. The reduced wavefunction is the wavefunction of a signal propagating in an inhomogeneous medium normalized by the wavefunction in free space. A statistical model is developed for a reduced wavefunction whose real and imaginary components are Gaussian random variables with unequal covariances. A linear transformation is performed and the probability density function of the received signal at a passive array is calculated. The Fisher information is calculated for special conditions of the transformation that are of interest to acoustic beamforming. The coupling of the Cramer–Rao lower bounds on the parameter estimates (e.g., the angles of arrival, source phase, and medium parameters) is addressed.

A guide to the literature on aggressive behavior

A guide to the literature on aggressive behavior

The Effects of Unequal Covariances and Reliabilities on Contemporaneous Inference: The Case of Hostility and Marital Happiness

Changes in attitudes and behaviors and processes of interaction in close relationships are inherently reciprocal (Copeland & White, 1991; Larzelere & Klein, 1986). Despite statistical difficulties, the number of articles actually estimating these effects has increased in recent years as researchers have learned to use both the power of structural equation modeling (Bollen, 1989; Lavee, 1988) and the advantages of prospective longitudinal designs (Booth, Johnson, White, Ameloza, & Edwards, 1988; Johnson, 1988; Kessler & Greenberg, 1981; Nesseroade & Baltes, 1979). As these more complex models gain popularity, it becomes important to clearly recognize the statistical mechanisms linking hypothesized models to actual data. These mechanisms are especially important in family research where data are often generated by multiple methods. Observational measures of concepts such as hostility, for example, may have remarkably different characteristics than similarly named concepts generated by questionnaire. As we will demonstrate, failure to recognize these differences in the modeling process may inadvertently lead to contradictory and potentially confusing results. This article, intended for researchers and practitioners, systematically examines the connections between reciprocal effects in panel models and the actual, observed data from which estimates are obtained. These connections will be investigated, first by using simplified, artificial data and then by using actual examples that link husbands' hostility to wives' marital happiness. Although panel models with additional covariates and three or more waves of data are theoretically more informative, the discussion will focus on the simple two-wave, two-variable model because it is methodologically useful in dramatizing a set of important relations (Duncan, 1969, 1975). The discussion will distinguish between cross-lagged and contemporaneous effects, and then draw on the contemporaneous model for illustrations. Although we focus on the contemporaneous model, the methods and results apply equally to cross-lagged models. Our hope is that improved understanding about the link between simple models and the data used to estimate them can provide both researchers and consumers of research with sharper intuitions with which to critically link statistical mechanisms to theoretical arguments in a variety of situations. BACKGROUND The literature on longitudinal reciprocal models distinguishes between situations in which a theoretically interesting process is just beginning (stage-state models) and those where change is an ongoing process (Dwyer, 1983; Menard, 1991). Assuming an ongoing process of change, researchers specify a time interval during which some variable X (e.g., husband's hostility) influences Y (e.g., wife's marital happiness) and Y, in turn, influences X. Many of the classic discussions have focused on the cross-lagged models in which two variables, measured at one time, are examined to infer their influence on each other when remeasured at a later time (e.g., Cook & Campbell, 1979; Shingles, 1985). But recent articles have also demonstrated contemporaneous effects, or the effects of two variables on each other within one observational period. Thus, for example, Rosenberg, Schooler, and Schoenbach (1989) examined the contemporaneous effects of self-esteem and the outcomes of school achievement, depression, and delinquency by using prior measures of the same variables plus additional covariates (e.g., prior measures of psychological states, family relations, school attitudes, etc.) as predictor variables. Kohn and Schooler (1978) measured both the contemporaneous and lagged effects of job complexity and intellectual flexibility, concluding that prior levels of intellectual flexibility predicted the complexity of the job one eventually attains (lagged effects), whereas current job complexity predicted concurrent intellectual flexibility (contemporaneous effects). …

A Monte Carlo study of the Friedman test and some competitors in the single factor, repeated measures design with unequal covariances

Single factor, repeated measures designs are employed in a variety of research settings. Assumptions about the form of the covariance matrix of the repeated measures have traditionally been applied only to parametric tests, yet the classical nonparametric alternative, the Friedman test, also possesses an implicit assumption of equal covariances of the measures. The results of a Monte Carlo study suggest that neither the Friedman rank test or other nonparametric competitors are robust to extreme departures from equal covariances when population distributions are symmetric, or to mild departures when population distributions are skewed. Surprisingly, the type I error rate of the parametric F test was less sensitive to an asymmetric⧸heavy-tailed distribution for unequal covariances than the nonparametric tests.

The Effects of Unequal Covariances on the Tukey WSD Test

The purpose of this study was to determine the effect of violating the assumption of homogeneity of covariance for the Tukey WSD test on the familywise rate of Type I error, when all other assumptions of the test were met. Determinations of the empirical significance level were made using computerized Monte Carlo simulations varying the number of treatment groups ( k = 3 and 5), sample size ( n = 5, 15, and 25), and degree of covariance heterogeneity (none, moderate, and high). For each of the resulting 18 combinations, 10,000 separate data sets were generated and analyzed at nominal α = .01 and .05 using the Tukey WSD procedure for all pairwise comparisons of means. For the conditions investigated in this study, degree of covariance heterogeneity showed a consistent effect on the empirical significance level. As the degree of covariance heterogeneity was increased, the empirical significance levels increased beyond the nominal levels. In general, different combinations of sample size and number of treatments produced no consistent effect on the empirical significance level.

Simultaneous Confidence Intervals for Ratios of Normal Means

Abstract Given one or more groups of multivariate normal samples, methods are presented for forming simultaneous confidence intervals of all ratios of linear forms of the mean vectors. The methods cover the cases of equal or unequal covariances. In a simultaneous inference context, they are multivariate extensions of a method due to Fieller (1954) for estimation of the ratio of two normal means.

Simultaneous Confidence Intervals for Ratios of Normal Means

Abstract Given one or more groups of multivariate normal samples, methods are presented for forming simultaneous confidence intervals of all ratios of linear forms of the mean vectors. The methods cover the cases of equal or unequal covariances. In a simultaneous inference context, they are multivariate extensions of a method due to Fieller (1954) for estimation of the ratio of two normal means.

Classification into one of several populations with allowance to unequal covariances

Given m independent sets of observations from each of K-variate normal distributions, the program: (1) tests the assumption of equality of covariances among groups; (2) searches for regions with homogeneous covariances; (3) estimates discrimination parameters and allocates observations; (4) evaluates the allocation procedure by calculating the misclassification matrix whose i, j entry is the number of data points that actually belong to group i and were classified into group j(i, j = 1, ...., m).

Qdf misclassification probabilities for known population parameters

Approximated QDF misclassification probabilities have been derived for bivariate normal populations with known parameter values. Tne effect of unequal covariances and population distance on the misclassification probabilities are examined

An RKHS approach to detection and estimation problems--II: Gaussian signal detection

The theory of reproducing kernel Hilbert spaces is used to obtain a simple but formal expression for the likelihood ratio (LR) for discriminating between two Gaussian processes with unequal covariances, and to develop a test by which the formal expression can be checked for validity. This LR formula can be evaluated by working separately with each covariance, thus reducing the calculations for the random signal case to those for the simpler known signal problem. In contrast, all previous LR formulas for the unequal covariance problem seem to require calculations involving both covariances simultaneously.