Equality Of Covariance Matrices Research Articles

Hierarchic likelihood ratio tests for equality of covariance matrices

Powers and sizes are simulated for hierarchic components of an adjusted likelihood ratio test for equality of covariance matrices.

Statistical process control of acoustic emission for cutting tool monitoring

The problem of cutting process monitoring has been investigated in recent years, with encouraging results, using pattern recognition analysis of acoustic emission (AE) signals. The analyses are based on linear discriminant functions, which assume that the observed data (from each class) are independent random samples from multivariate normal distributions with equal covariance matrices. However, in a number of practical situations some (or all) of these assumptions may not necessarily hold, resulting in errors in the analysis. In this paper, the distributions of AE spectra generated in earlier work are first analysed, and the results indicate departure from the assumptions, although the lack of normality was not too severe. Relaxing the assumption of equality of the covariance matrices, quadratic discriminant function analysis produced improved results for tool wear and chip noise monitoring while degrading tool fracture detection. The latter is due to inadequacy of the amount of data used in training the system. It is expected that increasing the data base would improve the results for all classes. The analysis until now has focused on reducing the dimensionality of the feature space by eliminating the features with the least discriminatory power. Even though this inevitably reduces the performance of the system, it is a necessary compromise for increased computational speed. To make use of the entire feature set with a reduced matrix rank, a principal component analysis is investigated. The result is a substantial improvement in correct classification of AE signals, even under different cuting conditions.

Regional discrimination between NTS explosions and western U.S. earthquakes

AbstractA number of earthquake/explosion discriminants outlined in Pomeroy et al. (1982) are evaluated in the western United States. The data consist of 233 NTS explosions and 130 western U.S. earthquakes in the magnitude range of 2.5 to 6.5 recorded at four broadband seismic stations operated by Lawrence Livermore National Laboratory (LLNL). The stations surround NTS at distances of about 200 to 400 km. The propagation paths for the earthquakes range from approximately 175 to 1300 km and are confined mainly to the Basin and Range. The discriminants tested include mb − Ms, mb, − Msh (higher mode magnitude), long-period Love and Rayleigh wave energy density and their ratio, relative excitation of short-period SH waves, Lg/Pg, Lg/Rg, Lg/Sm short-period amplitude ratios, spectral ratios in PN, Pg and Lg, and third moment of frequency.In general, the long-period discriminants exhibit the best separation between the earthquake and explosion populations. However, the long-period measurements are often difficult to make, especially at small magnitudes. Thus, it is necessary to supplement the long-period discriminants with short-period time- and frequency-domain discriminants. The short-period time-domain discriminants generally were characterized by very poor discriminant performance. However, because of the good signal to noise ratio for many of the regional phases (such as Lg and Pg), a large number of measurements could be acquired down to small magnitudes. The usefulness of the short-period time-domain discriminants lies in the fact that when an event shows a ratio above a certain threshold, it can generally be correctly identified. Spectral discriminants appear to have good potential, even at relatively small magnitudes. The measurements are simple to make, and first-order distance corrections can easily be applied. However, a number of uncertainties regarding the lack of a physical understanding of how the spectral discriminants work makes their utilization questionable at this point.A multivariate discrimination technique is described that is designed to handle missing data while providing optimum classification of an unknown event based on a given set of discrimination variables. The discrimination variables are first transformed to be insensitive to magnitude, path, or features of the test program. Tests show that the commonly used assumption of equal covariance matrices for the earthquake and explosion populations (resulting in a linear discrimination function) is not valid. However, a quadratic discrimination function appears to be appropriate. An optimality criterion, misclassification cost minimization, is used to select the best set of discrimination variables at each station. Application of the multivariate technique to individual stations for the entire dataset shows misclassification rates ranging from 3 to 10 per cent and 4 to 26 per cent for explosions and earthquakes, respectively. Multistation discrimination shows misclassification probabilities of 0 to 2 per cent for explosions and 3 to 4 percent for earthquakes.Most misclassifications occur for mb < 4.0 where it is often difficult to obtain long-period measurements that perform well. Earthquakes show higher misclassification rates than explosions presumably because of their wider range of paths, depth, and focal mechanisms. Significant station variability in discrimination performance is observed, but we are unable to separate out effects of receiver structure from path effects.

Multivariate measures of similarity and niche overlap

Niche overlap measures are used to assess the similarity in resource use by two species. Recently researchers have used niche overlap measures as summary measures and for making inferences, typically about competition for resources. The problem of estimating niche overlap when the niches are multivariate normal distributions with equal covariance matrices has previously been studied. In this work, the assumption of equal covariance matrices is relaxed. Two general measures of similarity are evaluated assuming general multivariate normal distributions. Commonly used measures of overlap are given as special cases of these two general measures. The question of bias in estimating these measures is discussed and shown to be a potential problem, especially when there are many redundant variables or if sample sizes are small.

Asymptotic nonnull distribution of likelihood ratio statistic for testing homogeneity of complex multivariate gaussian populations

In this paper asymptotic expansion of the nonnull distribution of the likelihood ratio statistic for testing homogeneity of several multivariate complex Gaussian populations has been derived. Nonnull moments are derived under the assumption of equality of covariance matrices. The asymptotic nonnull distribution is derived under the sequence of local alternatives, using logarithmic expansion for gamma function and some results on zonal polynomials.

Tests of Hypotheses for Covariance Matrices and Distributions Under Multivariate Normal Populations

SYNOPTIC ABSTRACTThis paper treats the following tests of hypotheses on covariance matrices under multivariate normal distributions: (i) the equality of a covariance matrix to a given matrix, (ii) sphericity, (iii) independence, (iv) equality of covariance matrices, and (v) equality of covariance matrices against ‘one–sided’ alternatives. Although we have several test criteria for (v), the test criteria for the other problems are the likelihood ratio (LR) criteria the exact distributions of which are considered in the early 1970's. The main purpose of this paper is to review test criteria introduced by the author for testing hypotheses about a covariance matrix. We give the likelihood ratio criteria and forms of their distributions. These forms are asymptotic expansions that are different under the null hypothesis and the fixed alternatives, except for problem (v) for which these forms are the same.

On the tiku-balakrishnan tests for equality of covariance matrices

Tiku and Balakrishnan's (1985) method cannot be considered to be testing the equality of covariance matrices. A simple example shows that their T2 statistic can equal zero for data having unequal sample covariance matrices, but a nonsingular transformation of those same data can then render their statistic indefinitely large

Partially pooled covariance matrix estimation in discriminant analysis

The Linear Discriminant Rule (LD) is theoretically justified for use in classification when the population within-groups covariance matrices are equal, something rarely known in practice. As an alternative, the Quadratic Discriminant Rule (QD) avoids assuming equal covariance matrices, but requires the estimation of a large number of parameters. Hence, the performance of QD may be poor if the training set sizes are small or moderate. In fact, simulation studies have shown that in the two-groups case LD often outperforms QD for small training sets even when the within -groups covariance matrices differ substantially. The present article shows this to be true when there are more than two groups, as well. Thus, it would seem reasonable and useful to develop a data-based method of classification that, in effect, represents a compromise between QD and LD. In this article we develop such a method based on an empirical Bayes formulation in which the within-groups covariance matrices are assumed to be outcomes of a common prior distribution whose parameters are estimated from the data. Two classification rules are developed under this framework and, through the use of extensive simulations, are compared to existing methods when the number of groups is moderate.

Distribution of multivariate quadratic forms under certain covariance structures

AbstractNecessary and sufficient conditions are given for the covariance structure of all the observations in a multivariate factorial experiment under which certain multivariate quadratic forms are independent and distributed as a constant times a Wishart. It is also shown that exact multivariate test statistics can be formed for certain covariance structures of the observations when the assumption of equal covariance matrices for each normal population is relaxed. A characterization is given for the dependency structure between random vectors in which the sample mean and sample covariance matrix have certain properties.

Nonnull distribution of the likelihood ratio criterion for testing equality of covariance matrices under intraclass correlation model

The nonnull moments of the likelihood ratio statistic for testing equality of covariance matrices under intraclass correlation structure are obtained in terms of the Lauricella's hypergeometric functions and also in terms of zonal polynomials. Then the nonnull asymptotic distribution of the statistic is derived for certain alternatives.

Transformation considerations in discriminant analysis

The linear discriminant function (LDF) is known to be optimal in the sense of achieving an optimal error rate when sampling from multivariate normal populations with equal covariance matrices. Use of the LDF in nonnormal situations is known to lead to some strange results. This paper will focus on an evaluation of misclassification probabilities when the power transformation could have been used to achieve at least approximate normality and equal covariance matrices in the sampled populations for the distribution of the observed random variables. Attention is restricted to the two-population case with bivariate distributions.

Cluster analysis via normal mixture models

The technique of clustering uses the measurements on a set of elements to identify clusters or groups of elements, such that there is relative homogeneity within the groups and heterogeneity between the groups. Under the mixture maximum likelihood approach to clustering, the elements are assumed to he a sample from a mixture of various proportions of a specified number of populations. By adopting some parametric form for the density function in each underlying population, a likelihood can be formed in terms of the mixture density and the unknown parameters estimated using the likelihood principle. The allocation of the elements to the populations is determined on the basis of the estimated posterior probabilities of population membership.Various aspects of this approach to clustering are examined in the case where the component populations are assumed to have multivariate normal distributions. Initially, the problems associated with likelihood estimation in this context are explored. Unfortunately with mixture models, the likelihood equation usually has multiple roots, so there is the question of which root to choose. In the case of equal covariance matrices the situation is straightforward, in that the maximum likelihood estimator exists and is consistent. An example is presented, however, to demonstrate that the adoption of a homoscedastic normal model, in the presence of some heteroscedasticity, can considerably influence the likelihood estimates, in particular, of the mixing proportions, and therefore the consequent clustering of the sample at hand.With the mixture approach to clustering, each element in the sample is allocated to one of the component populations on the basis of the estimated posterior probabilities of population membership. Consideration is given to assessing the performance of the mixture approach by averaging appropriate functions of these posterior probabilities. In the case where the superpopulation does consist of individual component populations, in number equal to that specified with the particular application of this method, these functions can be regarded as estimates of the correct allocation rates for the individual populations, as well as for the overall mixture. As the proposed method of estimation can produce biased estimates, the bootstrap procedure is studied for its effectiveness in reducing this bias. Three real data sets are considered, as well as a detailed simulation study. It is shown that the proposed estimates generally provide useful information on the unobservable allocation rates of the mixture approach. Encouraging results are obtained for the bootstrap method of bias correction applied to the estimates of the individual and overall allocation rates.The role of the mixture approach is investigated, also, for the clustering of treatments from a randomized complete block design. In this context, it provides a concise way of summarizing differences amongst the treatments. It is shown that the implementation of this technique is straightforward for fixed, but not random block effects. The difficulty is that with the latter model the amount of computation becomes prohibitive for even a moderate number of treatments. This approach is illustrated by the application of the mixture method to three data sets from randomized complete block designs. The first two examples concern real data already used in the literature to illustrate other techniques for grouping means under a fixed effects model. The third example consists of data simulated according to a randomized complete block design with random block effects, where the true grouping is known and where the number of treatments is small enough for the mixture method to be implemented under this mixed model. The results under this model are compared with those obtained by treating the block effects as fixed, in an instance where they are actually random.Most available clustering techniques are applicable only to a two-way data set, where one of the modes (the elements) is being partitioned into groups on the basis of the other mode (the measurements). If, however, the data set is defined as a three-way array, then a multivariate technique, which will cluster the elements (one of the modes) on the basis of the total information available (the other two modes simultaneously), is required. It is shown that by appropriate specification of the underlying model, the mixture method of clustering can be applied in this context. This is illustrated using soybean data, which consist of multi-attribute measurements on a number of genotypes, each grown in several environments. This approach is particularly useful here, as the genotype by environment interaction is able to be incorporated directly into the underlying model. Although the problem is set in the framework of clustering genotypes, the technique is applicable to other types of three-way data, and so is of considerable potential.

Likelihood Estimation with Normal Mixture Models

SUMMARY We consider some of the problems associated with likelihood estimation in the context of a mixture of multivariate normal distributions. Unfortunately with mixture models, the likelihood equation usually has multiple roots and so there is the question of which root to choose. In the case of equal covariance matrices the choice of root is straightforward in the sense that the maximum likelihood estimator exists and is consistent. However, an example is presented to demonstrate that the adoption of a homoscedastic normal model in the presence of some heteroscedasticity can considerably influence the likelihood estimates, in particular of the mixing proportions, and hence the consequent clustering of the sample at hand.

Linear discrimination for three known normal populations

Abstract A random vector is assumed to have one of three known multivariate normal distributions with equal covariance matrices. It is desired to separate the three distributions by means of a single linear discriminant function. Such a function can lead to a classification rule. The function whose classification rule minimizes the average of the three probabilities of misclassification is found. Also the function is found whose rule minimizes the maximum of the three probabilities of misclassification.

Measurement errors - a location contamination problem in discriminant analysis

The random shift model for location contamination of two normal distributions with equal covariance matrices is introduced. Performance of Fisher1s linear discriminant rule in separating the two contaminated populations is essentially the same as for uncontaminated normal distributions.

On the choice of dimensionality and sample size for feature selection

Abstract This paper considers the problem of selection of dimensionality and sample size for feature extraction in pattern recognition. In general, the axes of the feature space are selected as the eigenvectors of matrices of the form R 2 −1 R 1 where R 1 and R 2 are real symmetric matrices. Expressions are derived for obtaining the changes in the eigenvalues and eigenvectors when there are changes of first order of smallness in the matrices R 1 and R 2. Based on this theory, a method is developed for choosing the dimensionality of the patterns. Also expressions are derived for the selection of sample size for estimating the eigenvectors, for two gaussian distributed pattern classes with equal means, unequal covariance matrices and with unequal means and equal covariance matrices.

Sequential discrimination

An analysis of the 1-stage classification decision with two candidate populations is provided in this paper. When the successive posterior probabilities follow a first order markov process it it shown that the optimal classification rules are greatly simplified. A detailed analysis and example are provided for the important case of multivariate normality with equal covariance matrices.

A New Distance Measure for Vowel Recognition

The problem of vowel recognition is introduced as the classification of autoregressive systems with normally distributed parameters upon the observation of a finite segment of the stationary output process with white Gaussian noise as the input. A new distance measure is derived which differs from the squared-Mahalanobis distance used for identifying normally-distributed classes with equal covariance matrices. Its usefulness is demonstrated via an experiment with six vowels forming the class set. This distance measure performed better than other conventional distance measures used in speech recognition.

Some complex variable transformations and exact power comparisons of two-sided tests of equality of two Hermitian covariance matrices

Abstract Power studies of tests of equality of covariance matrices of two p -variate complex normal populations σ 1 = σ 2 against two-sided alternatives have been made based on the following five criteria: (1) Roy's largest root, (2) Hotelling's trace, (4) Wilks' criterion and (5) Roy's largest and smallest roots. Some theorems on transformations and Jacobians in the two-sample complex Gaussian case have been proved in order to obtain a general theorem for establishing the local unbiasedness conditions connecting the two critical values for tests (1)–(5). Extensive unbiased power tabulations have been made for p =2, for various values of n 1 , n 2 , λ 1 and λ 2 where n 1 is the df of the SP matrix from the i th sample and λ 1 is the i th latent root of σ 1 σ -1 2 ( i =1, 2). Equal tail areas approach has also been used further to compute powers of tests (1)–(4) for p =2 for studying the bias and facilitating comparisons with powers in the unbiased case. The inferences have been found similar to those in the real case. (Chu and Pillai, Ann. Inst. Statist. Math. 31.

Alternative Methods of Estimating the Likelihood Ratio in Classification of Multivariate Normal Observations

Abstract An observation is to be classified into one of two multivariate normal distributions with equal covariance matrices. When the parameters are unknown, four methods of estimating the likelihood ratio, that is, the plug-in method, the test procedure, the Bayesian approach, and the best invariant estimate method, are reviewed. The assumptions, interpretations, and consequences of the four approaches are given. It is shown that the last three methods yield the same classification procedure.