Equality Of Covariance Matrices Research Articles

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices

The equality of covariance matrices is an essential assumption in means and discriminant analyses for high-dimensional data. The performance of tests for covariance matrices may vary substantially depending on the covariance structure, so using inappropriate methods to verify the assumption will result in worse performance. The purpose of this study is to assess and compare the performance of three tests for two-sample high-dimensional covariance matrices: Schott’s (2007), Srivastava and Yanagihara’s (2010), and Li and Chen’s (2012) under various covariance structures. A simulation study was conducted when the covariance structures were spherical, compound symmetric, block-diagonal, and first-order autoregressive with homogenous variances. The results show that Li and Chen’s test outperforms the others with a sample size of at least 10 under particular covariance structures. When the number of variables is increased with a fixed sample size, Li and Chen’s test still performs well, whereas Schott’s performance deteriorates. Some recommendations for selecting appropriate tests are also provided in this paper.

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

AbstractWe derive an asymptotic expansion for the log‐likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal‐to‐noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation‐Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo‐electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest. © 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.

On Testing Homogeneity of Covariance Matrices with Box’s M and the Approximate Tests for Multivariate Data

Homogeneity of covariance matrices, or equal covariance matrices across groups, is one of the most important assumptions in multivariate analysis of variance (MANOVA) and in discriminant analysis. Box’s M test, an exact test, is a generally accepted method used to check the violation of this assumption. The Box’s statistic M can be transformed to the statistics serving as the approximate tests based on chi-squared and F distributions. This study aims to assess the performance of Box’s M test compared with the approximate test using chi-squared distribution for normal data. When the data are non-normal, the performance of the approximate test and the nonparametric test using the bootstrap method are examined. The results showed that under normality with equal sample sizes and certain conditions to conduct the exact test, the Box’s M and the approximate chi-squared tests perform well whereas the performance of the approximate test is slightly better for detecting heterogeneity of covariance matrices. In the case of unequal sample sizes, in which the data do not meet the conditions to conduct Box’s M test, the approximate chi-squared test also performs well so it is applicable for such a case. When dealing with non-normal data, such as multivariate t-distributed data, the performance of the approximate test as well as the nonparametric test using the bootstrap method is unsatisfactory in the situations studied so it needs to be further developed.

Empirical likelihood test for the equality of several high-dimensional covariance matrices

The testing covariance equality is of importance in many areas of statistical analysis, such as microarray analysis and quality control. Conventional tests for the finite-dimensional covariance do not apply to high-dimensional data in general, and tests for the high-dimensional covariance in the literature usually depend on some special structure of the matrix and whether the dimension diverges. In this paper, we propose a jack-knife empirical likelihood method to test the equality of covariance matrices. The asymptotic distribution of the new test is regardless of the divergent or fixed dimension. Simulation studies show that the new test has a very stable size with respect to the dimension and it is also more powerful than the test proposed by Schott (2007) and studied by Srivastava and Yanagihara (2010). Furthermore, we illustrate the method using a breast cancer dataset.

A comprehensive treatment of quadratic-form-based inference in repeated measures designs under diverse asymptotics

Split-Plot or Repeated Measures Designs with multiple groups occur naturally in sciences. Their analysis is usually based on the classical Repeated Measures ANOVA. Roughly speaking, the latter can be shown to be asymptotically valid for large sample sizes ni assuming a fixed number of groups a and time points d. However, for high-dimensional settings with d>ni, this argument breaks down and statistical tests are often based on (standardized) quadratic forms. Furthermore, analysis of their limit behaviour is usually based on certain assumptions on how d converges to ∞ with respect to ni. As this may be hard to argue in practice, we do not want to make such restrictions. Moreover, sometimes also the number of groups a may be large compared to d or ni. To also have an impression about the behaviour of (standardized) quadratic forms as test statistic, we analyze their asymptotics under diverse settings on a, d and ni. In fact, we combine all kinds of combinations, where they diverge or are bounded in a unified framework. To this aim, we assume equal covariance matrices between all groups. Studying the limit distributions in detail, we follow Sattler and Pauly (2018) and propose an approximation to obtain critical values. The resulting test and its approximation approach are investigated in an extensive simulation study focusing on the exceptional asymptotic frameworks that are the main focus of this work.

The simultaneous test of equality and circularity of several covariance matrices

The equality of covariance matrices test is widely used in many statistical procedures, and recently, matrices with a circular pattern, which are particular cases of the well-known Toeplitz matrices, have been considered in practical applications. In this work we present a statistical procedure that may allow one to test simultaneously both the equality and the circularity of a given number of covariance matrices. This procedure is based on a likelihood ratio test that is derived through an adequate decomposition of the overall null hypothesis in two partial null hypotheses, and on near-exact approximations developed for the test statistic. Numerical studies are carried out to assess the quality of these approximations and to illustrate their asymptotic properties. Simulation studies are also developed to study the properties of test.

Testing Mean Differences among Groups: Multivariate and Repeated Measures Analysis with Minimal Assumptions

ABSTRACTTo date, there is a lack of satisfactory inferential techniques for the analysis of multivariate data in factorial designs, when only minimal assumptions on the data can be made. Presently available methods are limited to very particular study designs or assume either multivariate normality or equal covariance matrices across groups, or they do not allow for an assessment of the interaction effects across within-subjects and between-subjects variables. We propose and methodologically validate a parametric bootstrap approach that does not suffer from any of the above limitations, and thus provides a rather general and comprehensive methodological route to inference for multivariate and repeated measures data. As an example application, we consider data from two different Alzheimer’s disease (AD) examination modalities that may be used for precise and early diagnosis, namely, single-photon emission computed tomography (SPECT) and electroencephalogram (EEG). These data violate the assumptions of classical multivariate methods, and indeed classical methods would not have yielded the same conclusions with regards to some of the factors involved.

Multi-arm trials with multiple primary endpoints and missing values.

We present an extension of multiple contrast tests for multiple endpoints to the case of missing values. The endpoints are assumed to be normally distributed and correlated and to have equal covariance matrices for the different treatments. Different multivariate t distributions will be applied, differing in endpoint-specific degrees of freedom. In contrast to competing methods, the familywise error type I is maintained in the strong sense in an admissible range, and the problem of different marginal errors type I is avoided. The information of all observations is exploited, thereby enabling a gain in power compared with a complete case analysis.

Localization of vectors–patterns in the problems of parametric classification with the purpose of increasing its accuracy

The problem of the accuracy of classification, determined by the share of the correctly classified objects with the use of parametric methods of pattern recognition, is examined. In particular, we analyzed the influence of deviation from the normal law of distribution of variables of the space of features – random variables, in essence – and the inequality of covariance matrices of classes on the accuracy of classification. It was found that the inequality of covariance matrices of the divided classes leads to the shift of the dividing surface; however, the magnitude of this displacement may not become the essential factor, which influences the accuracy of classification. It is shown for the space of variables of dimensionality (N×2) that for the randomly selected classes inside the square with the length of the edge, equal to two, the accuracy of classification for classes A and B proves to be different and depends on the position of the straight line, which divides the classes. It is shown that localization of vectors-patterns in the space of features can be selected in accordance with the plans of full factorial experiment (FFE). Due to this localization, the equality of covariance matrices of the classes is ensured and the share of the correctly classified objects increases. We proposed, to overcoming the main shortcoming of the given research, connected to the absence of functional dependencies, which make it possible to quantitatively evaluate the accuracy of classification at various variants of arrangement of FFE plans, to use the methods of experimental optimization. Its purpose is the selection of such coordinates of the centers of plans and their geometric characteristics for classes A and B, which ensure the maximum share of the correctly classified objects based on the obtained decision rules. The realization of this approach may become the reserve for the increase in the accuracy of classification with application of the methods of parametric optimization.

Robust Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is the most commonly employed method for classification. This method which creates a linear discriminant function yields optimal classification rule between two or more groups under the assumptions of normality and homoscedasticity (equal covariance matrices). However, the calculation of parametric LDA highly relies on the sample mean vectors and pooled sample covariance matrix which are sensitive to non-normality. To overcome the sensitivity of this method towards non-normality as well as homoscedasticity, this study proposes two new robust LDA models. In these models, an automatic trimmed mean and its corresponding winsorized mean are employed to replace the mean vector in the parametric LDA. Meanwhile, for the covariance matrix, this study introduces two robust approaches namely the winsorization and the multiplication of Spearman's rho with the corresponding robust scale estimator used in the trimming process. Simulated and real financial data are used to test the performance of the proposed methods in terms of misclassification rate. The numerical result shows that the new method performs better if compared to the parametric LDA and the robust LDA with S-estimator. Thus, these new models can be recommended as alternatives to the parametric LDA when non-normality and heteroscedasticity (unequal covariance matrices) exist.

Performance evaluation of likelihood-ratio tests for assessing similarity of the covariance matrices of two multivariate normal populations

In analyzing two multivariate normal data sets, the assumption about equality of covariance matrices is usually used as a default for doing subsequence inferences. If this equality doesn’t hold, later inferences will be more complex and usually approximate. If one detects some identical components between two decomposed non equal covariance matrices and uses this extra information, one expects that subsequence inferences can be more accurately performed. For this purpose, in this article we consider some statistical tests about the equality of components of decomposed covariance matrices of two multivariate normal populations. Our emphasis is on the spectral decomposition of these matrices. Hypotheses about the equalities of sizes, shapes, and set of directions as components of these two covariance matrices are tested by the likelihood ratio test (LRT). Some simulation studies are carried out to investigate the accuracy and power of the LRT. Finally, analyses of two real data sets are illustrated.

English

Assessing patients’ perception of satisfaction, professionalism, competence and skill of health care providers helps health institutions and providers respond to the needs and expectations of the patients. Little work is known to have taken place to understand this phenomenon especially in radiological services in Nigerian hospitals. This study is therefore aimed at investigating the perception of patients’ satisfaction, professionalism, competence and skill of radiographers during routine radiological examination in two Nigerian hospitals. Materials and Methods: Three hundred patients were interviewed from two hospitals on their perception of satisfaction, professionalism, competence and skill of radiographers during routine radiological examination. The data was analyzed in terms of descriptive statistics using 95% confidence interval. ANOVA, Chi-Square and multivariate (intercept) statistical tools were applied. Results: The result of Test of Equality of Covariance Matrices giving a Box’s M result of 47.745 and F-value of 1.787 >Fcritical of 1.52 (p < 0.05), indicates that respondents’ perception of level of satisfaction, radiographers’ professionalism, skill and competence are dependent on the combined factors of the respondents age, type of hospital where service is received, gender, level of education and patient length of experience of radiological services. Key-words: Satisfaction, professionalism, skill and competence, Nigeria, patient perception of care, radiological examination.

A Comparison of Two Linear Discriminant Analysis Methods That Use Block Monotone Missing Training Data

We revisit a comparison of two discriminant analysis procedures, namely the linear combination classifier of Chung and Han (2000) and the maximum likelihood estimation substitution classifier for the problem of classifying unlabeled multivariate normal observations with equal covariance matrices into one of two classes. Both classes have matching block monotone missing training data. Here, we demonstrate that for intra-class covariance structures with at least small correlation among the variables with missing data and the variables without block missing data, the maximum likelihood estimation substitution classifier outperforms the Chung and Han (2000) classifier regardless of the percent of missing observations. Specifically, we examine the differences in the estimated expected error rates for these classifiers using a Monte Carlo simulation, and we compare the two classifiers using two real data sets with monotone missing data via parametric bootstrap simulations. Our results contradict the conclusions of Chung and Han (2000) that their linear combination classifier is superior to the MLE classifier for block monotone missing multivariate normal data.

Percentage points of LRC for testing equality of covariance matrices under intraclass correlation structure

ABSTRACTThe exact null distribution of the likelihood ratio test statistic for testing equality of covariance matrices of q compound symmetric Gaussian models (bivariate or trivariate) has been obtained and percentage points for q ⩽ 5 have been computed. The inverse Mellin transform and calculus of residues have been used to derive these results.

A triplot for multiclass classification visualisation

Quadratic discriminant analysis is used when the assumption of equal covariance matrices for linear discrimination does not hold. The Canonical Variate Analysis biplot is used for graphical visualisation to accompany linear discriminant analysis. However, since class specific covariance matrix estimates are needed for quadratic discrimination the canonical transformation cannot be used. An alternative method of visually representing the discrimination and classification process is proposed: representing the sample points, classification regions based on quadratic discriminant analysis and including information on the variables. The methodology is further extended to other forms of multiclass classification and illustrated for support vector machines, classification trees, k-nearest neighbours and latent class analysis. In all these triplots three aspects are represented simultaneously, allowing for the representation of the relationships between samples and variables, relative to the classification regions.

Discriminant Analysis Procedures Under Non-optimal Conditions for Binary Variables

The performance of four discriminant analysis procedures for the classification of observations from unknown populations was examined by Monte Carlo methods. The procedures examined were the Fisher Linear discriminant function, the quadratic discriminant function, a polynomial discriminant function and A-B linear procedure designed for use in situations where covariance matrices are equal. Each procedure was observed under conditions of equal sample sizes, equal covariance matrices, and in conditions where the sample was drawn from populations that have a multivariate normal distribution. When the population covariance matrices were equal, or not greatly different, the quadratic discriminant function performed similarly or marginally the same like Linear procedures. In all cases the polynomial discriminate function demonstrated the poorest, linear discriminant function performed much better than the other procedures. All of the procedures were greatly affected by non-normality and tended to make many more errors in the classification of one group than the other, suggesting that data be standardized when non-normality is suspected.

The exact and near-exact distributions of the main likelihood ratio test statistics used in the complex multivariate normal setting

In this paper the authors show how it is possible to establish a common structure for the exact distribution of the main likelihood ratio test (LRT) statistics used in the complex multivariate normal setting. In contrast to what happens when dealing with real random variables, for complex random variables it is shown that it is possible to obtain closed-form expressions for the exact distributions of the LRT statistics to test independence, equality of mean vectors and the equality of an expected value matrix to a given matrix. For the LRT statistics to test sphericity and the equality of covariance matrices, cases where the exact distribution has a non-manageable expression, easy to implement and very accurate near-exact distributions are developed. Numerical studies show how these near-exact distributions outperform by far any other available approximations. As an example of application of the results obtained, the authors develop a near-exact approximation for the distribution of the LRT statistic to test the equality of several complex normal distributions.

Improved adaptive Markov random field based super-resolution mapping for mangrove tree identification

Abstract. Traditionally, forest tree crowns are extracted using airborne or spaceborne hyper-/multi-spectral remotely sensed images or pansharpened images. However, these medium/low spatial resolution images suffer from the mixed pixel problem, and the cost to collect very high resolution image collection is high. Moreover, existing feature extraction techniques cannot extract local patterns from medium/low resolution images. Therefore, super-resolution mapping (SRM) techniques, which generate land-cover maps with finer spatial resolution than the original remotely sensed image, can be beneficial for the extraction of forest trees. The SRM methods can improve the quality of information extraction by combining spectral information and spatial context into image classification problems. In this paper we have improved an adaptive Markov random field approach for super-resolution mapping (MRF-SRM) based on spatially adaptive MRF-SPM to overcome the limitation of equal covariance matrices assumption for all classes. We applied the developed method for mangrove tree identification from multispectral image recorded by QuickBird satellite, where we generated a super-resolution map with the panchromatic image spatial resolution of 0.6 m. Moreover, the performance of the proposed technique is evaluated by employing the simulated image with different covariance matrices for each class. Our experimental results have demonstrated that the new adaptive MRF-SRM method has increased the overall accuracy by 5.1% and the termination conditions of this method were satisfied three times faster when compared to the state-of-the-art methods.

A Two Sample Test for Mean Vectors with Unequal Covariance Matrices

In this paper, we consider testing the equality of two mean vectors with unequal covariance matrices. In the case of equal covariance matrices, we can use Hotelling’s T2 statistic, which follows the F distribution under the null hypothesis. Meanwhile, in the case of unequal covariance matrices, the T2 type test statistic does not follow the F distribution, and it is also difficult to derive the exact distribution. In this paper, we propose an approximate solution to the problem by adjusting the degrees of freedom of the F distribution. Asymptotic expansions up to the term of order N− 2 for the first and second moments of the U statistic are given, where N is the total sample size minus two. A new approximate degrees of freedom and its bias correction are obtained. Finally, numerical comparison is presented by a Monte Carlo simulation.

Z-score linear discriminant analysis for EEG based brain-computer interfaces.

Linear discriminant analysis (LDA) is one of the most popular classification algorithms for brain-computer interfaces (BCI). LDA assumes Gaussian distribution of the data, with equal covariance matrices for the concerned classes, however, the assumption is not usually held in actual BCI applications, where the heteroscedastic class distributions are usually observed. This paper proposes an enhanced version of LDA, namely z-score linear discriminant analysis (Z-LDA), which introduces a new decision boundary definition strategy to handle with the heteroscedastic class distributions. Z-LDA defines decision boundary through z-score utilizing both mean and standard deviation information of the projected data, which can adaptively adjust the decision boundary to fit for heteroscedastic distribution situation. Results derived from both simulation dataset and two actual BCI datasets consistently show that Z-LDA achieves significantly higher average classification accuracies than conventional LDA, indicating the superiority of the new proposed decision boundary definition strategy.