Singular Covariance Matrix Research Articles

Regression models with unknown singular covariance matrix

In the analysis of the classical multivariate linear regression model, it is assumed that the covariance matrix is nonsingular. This assumption of nonsingularity limits the number of characteristics that may be included in the model. In this paper, we relax the condition of nonsingularity and consider the case when the covariance matrix may be singular. Maximum likelihood estimators and likelihood ratio tests for the general linear hypothesis are derived for the singular covariance matrix case. These results are extended to the growth curve model with a singular covariance matrix. We also indicate how to analyze data where several new aspects appear.

Convergence analysis of adaptive filtering algorithms with singular data covariance matrix

The paper provides a rigorous analysis of the behavior of adaptive filtering algorithms when the covariance matrix of the filter input is singular. The analysis is done in the context of adaptive plant identification. The considered algorithms are LMS, RLS, sign (SA), and signed regressor (SRA) algorithms. Both the signal and weight behavior of the algorithms are considered. The signal behavior is evaluated in terms of the moments of the excess output error of the filter. The weight behavior is evaluated in terms of the moments of the filter weight misalignment vector. It is found that the RLS and SRA diverge when the input covariance matrix is singular. The steady-state signal behavior of the LMS and SA can be made arbitrarily fine by using sufficiently small step sizes of the algorithms. Indeed, the long-term average of the mean square excess error of the LMS is proportional to the algorithm step size. The long-term average of the mean absolute excess error of the SA is proportional to the square root of the algorithm step size. On the other hand, the steady-state weight behavior of both the LMS and SA have biases that depend on the weight initialization. The analytical results of the paper are supported by simulations.

OUTLIER DETECTION AND RELIABILITY OF ADJUSTMENT MODELS WITH SINGULAR COVARIANCE MATRICES

ABSTRACT Up to now, outlier detection and reliability theory are generally based on the regular Gauss-Markov models, in which the covariance matrix of observations is positively definite. For the adjustment models with singular covariance matrix, the statistics for outlier detection are derived by the authors. The corresponding reliability theory is developed. And the application of the theory is demonstrated with a practical example.

Characterization of the multivariate Gauss-Markoff model with singular covariance matrix and missing values

The aim of this paper is to characterize the Multivariate Gauss-Markoff model (MGM) as in (2.1) with singular covariance matrix and missing values. MGMDP2 model and completed MGMDP2Q model are obtained by three transformations D, P and Q (cf. (3.21)) of MGM. The unified theory of estimation (Rao, 1973) which is of interest with respect to MGM has been used.The characterization is reached by estimation of parameters: scalar σ2 and linear combination \(\lambda '\bar B\left( {\bar B = vecB} \right)\) as in (4.8), (4.6), (4.7) as well as by the model of the form (5.1) (cf. Th. 5.1). Moreover, testing linear hypothesis in the available model MGMDP2 by test function F as in (6.3) and (6.4) is considered.It is known (Oktaba 1992) that ten quantities in models MGMDP2, and MGMDP2Q are identical (invariant). They permit to say that formulas for estimation and testing in both models are identical (Oktaba et al., 1988, Baksalary and Kala, 1981, Drygas, 1983).An algorithm and the UMGMBO program for calculations concerning estimation and testing in MGM have been presented by Oktaba and Osypiuk (1993).

Singularity and Nonnormality in the Classification of Compositional Data

Geologists may want to classify compositional data and express the classification as a map. Regionalized classification is a tool that can be used for this purpose, but it incorporates discriminant analysis, which requires the computation and inversion of a covariance matrix. Covariance matrices of compositional data always will be singular (noninvertible) because of the unit-sum constraint. Fortunately, discriminant analyses can be calculated using a pseudo-inverse of the singular covariance matrix; this is done automatically by some statistical packages such as SAS. Granulometric data from the Darss Sill region of the Baltic Sea is used to explore how the pseudo-inversion procedure influences discriminant analysis results, comparing the algorithm used by SAS to the more conventional Moore–Penrose algorithm. Logratio transforms have been recommended to overcome problems associated with analysis of compositional data, including singularity. A regionalized classification of the Darss Sill data after logratio transformation is different only slightly from one based on raw granulometric data, suggesting that closure problems do not influence severely regionalized classification of compositional data.

Projector operators in the multivariate Zyskind-Martin model

Two quadratic formsS H andS E for a testable hypothesis and for an error in the multivariate Zyskind-Martin model with singular covariance matrix are expressed by means of projector operators. Thus the results for the multivariate standard model with identity covariance matrix given by Humak (1977) and Christensen (1987, 1991) are generalized for the case of Zyskind-Martin model. Special cases of our results are formulae forS H andS E in Aitken's (1935) model. In the case of general Gauss-Markoff modelS H andS E can also be expressed by means of projector operators for some subclasses of testable hypotheses. For these hypotheses, testing in Gauss-Markoff model is equivalent to testing in a Zyskind-Martin model.

A simple derivation on the chi-square approximation of Pearson's statistic

One of the most popular test statistics is Pearson's c2 goodness-of-fit statistic which is known to have an approximate chi-square distribution for large sample size. All the proofs so far seem to make use of the chisquaredness of quadratic forms in normal variables, manipulations of quadratic forms and singular covariance matrix. An elegant simple derivation of this result is given here without making use of chisquaredness of quadratic forms or other such results. Two numerical examples are also given to caution about the use of this statistic.

Recovery of inter-block information: extensions in a two variance component model

For a two variance component mixed linear model, it is shown that under suitable conditions there exists a nonlinear unbiased estimator that is better than a best linear unbiased estimator defined with respect to a given singular covariance matrix. It is also shown how this result applies to improving on intra-block estimators and on estimators like the unweighted means estimator in a random one-way model.

Regularization of a generalized Kalman filter

The authors derive equations for optimum and near-optimum Kalman filters for the filtration problem with a singular covariance matrix for the noise in the observations in the case when the processes under consideration are described by Ito's stochastic differential equations with measure.

C430. On the singularity of the sample covariance matrix

Click to increase image sizeClick to decrease image sizeKeywords: Singularity of covariance matrices.

Densities of determinant ratios, their moments and some simultaneous confidence intervals in the multivariate Gauss-Markoff model

The following three results for the general multivariate Gauss-Markoff model with a singular covariance matrix are given or indicated. $1^\circ $ determinant ratios as products of independent chi-square distributions, $2^\circ $ moments for the determinants and $3^\circ $ the method of obtaining approximate densities of the determinants.

Estimation of the seemingly unrelated regression model when the error covariance matrix is singular

A two-step generalized least-squares (GLS) estimator proposed by Zellner for seemingly unrelated regression (SUR) models is implementable when the estimated covariance matrix of the errors in the SUR system is non-singular. Violating the premise of non-singularity is a common problem among many applications in economics, business and management. We present methods of resolving this problem and propose an efficient procedure. The simulation study shows that the estimator of Haff performs better for small-sized observations, whereas the estimator of Ullah and Racine performs better for larger sized observations. Furthermore, the Ullah-Racine estimate is simple to calculate and easy to use. The empirical analysis involves the study of the diffusion processes of videocassette recorders across different geographic regions in the US, which exhibits a singular covariance matrix. The empirical results show that the procedures efficiently deal with the problem and provide plausible estimation results.

Autoregressive Errors in Singular Systems of Equations

We consider a system of m general linear models, where the system error vector has a singular covariance matrix owing to various “adding up” requirements and, in addition, the error vector obeys an autoregressive scheme. The paper reformulates the problem considered earlier by Berndt and Savin [8] (BS), as well as others before them; the solution, thus obtained, is far simpler, being the natural extension of a restricted least-squares-like procedure to a system of equations. This reformulation enables us to treat all parameters symmetrically, and discloses a set of conditions which is different from, and much less stringent than, that exhibited in the framework provided by BS.Finally, various extensions are discussed to (a) the case where the errors obey a stable autoregression scheme of order r; (b) the case where the errors obey a moving average scheme of order r; (c) the case of “dynamic” vector distributed lag models, that is, the case where the model is formulated as autoregressive (in the dependent variables), and moving average (in the explanatory variables), and the errors are specified to be i.i.d.

Minimax estimation in linear regression with singular covariance structure and convex polyhedral constraints

Let the parameter of the linear regression model be restricted to a compact and convex polyhedron. On the basis of this additional information an estimator is shown to exist which minimizes the maximal weighted risk. This exact minimax solution turns out to be not feasible, but can be characterized as a limit point of approximate and operational minimax solutions. The results obtained are valid for general quadratic loss and possibly singular covariance matrix.

Wishart distributions in the multivariate Gauss-Markoff model with singular covariance matrix

This paper concerns generalized quadratic forms for the multivariate case. These forms are used to test linear hypotheses of parameters for the multivariate Gauss-Markoff model with singular covariance matrix. Distributions and independence of these forms are proved.

On the Exact Small Sample Distribution of the Instrumental Variable Estimator

The purpose of this note is to point out that the bimodality is a consequence of the special case considered by N-S. The exact finite sample distribution of the IV estimator has been derived in the economic literature earlier, but its tabulation or plotting involves some assumptions about the nuisance parameters. In this note we plot the densities of the exact finite sample distribution of the IV estimator for several parameter values and show that the bimodality feature noted by N-S is a consequence of a singular covariance matrix and not of a poor instrument.

Admissible linear estimators in mixed linear models

Inhomogeneous linear estimation in the general mixed model with possible singular covariance matrix is considered. It is shown that this problem reduces to the homogeneous linear estimation in a simpler model. In consequence, some necessary and sufficient conditions for the admissibility are derived from Rao (1976, Ann. Statist.4 1023–1037).

Optimized Improvement of Convergency in Least Squares Parameter Estimation of Biomedical Models

This paper investigates a method for optimally improving the convergency and stabil ity in the least squares (LS) estimation of model parameters in biological models. When we have to use a strongly correlated input for the identification and we are given only short length of noisy input-output measurement data, as arises in actual ph0ysiological experiments, the LS parameter estimates tend to fluctuate excessively and not to settle in steady-state values for the given data length. We focus on a method of adding nonnegative weighting constants into the diagonal elements of the data covariance matrix, which is normally used to mitigate the ill-conditions of a nearly singular covariance matrix. The aim of this paper is to theoretically describe the effects of the weighting constants on the mean square error (MSE) of the parameter estimates, and clarify the existence of the optimal weighting constant minimizing the MSE. Furthermore, we give a data-adaptive way of determining the optimal value of the weighting constant, which makes use of only available input-output measurement data. The effectiveness of the optimal selection of the weighting constant in the convergency improvement is investigated through both numerical simulations and modell ing of kinetics in diabetes using depancreatized dogs.

The use of generalized inverses in restricted maximum likelihood

The calculus of generalized inverses and related concepts in matrix algebra is applied to the general restricted maximum likelihood problem. Some new results on g-inverses, Kronecker products, and matrix differentials are presented. For the restricted maximum likelihood problem we obtain generalizations of the well-known results of Aitchison and Silvey [1]. We use the approach recently developed by Heijmans and Magnus [13, 14] to allow for non-i.i.d. observations. A nonlinear seemingly unrelated regressions model with possibly singular covariance matrix and linear restrictions ( NLSURSR) is analyzed, and the linear expenditure system (LES) is discussed as a special case.

A generalized specification test

Abstract In a non-linear parametric setting, a class of specification tests developed by Hausman (1978) is extended to accomodate a singular covariance matrix. An application to limited information tests for the exogeneity of instrumental variables is presented.