Non-negative Estimation Of Variance Components Research Articles

Non-negative least-squares variance and covariance component estimation using the positive-valued function for errors-in-variables models

Abstract Although (co)variance component estimation has been widely applied in the errors-in-variables (EIV) model, the occurrence of negative variance components is still a major issue in the estimated variance components. This problem may be due to the following unfavorable factors: 1) unreasonable selection of initial variance values; 2) low redundancy in the EIV functional model; 3) improper design in the EIV stochastic model, and 4) other data quality problems. Many attempts have been made to prevent the appearance of negative variance components. In this contribution, a novel and efficient non-negative least-squares variance component estimation using the PVF (PVF-NLS-VCE) is introduced, which can simultaneously estimate the unknown (co)variance components in the EIV stochastic model and the parameters in the EIV functional model. Its principle is to implicitly impose a non-negative constraint by replacing the variance component with the positive-valued function (PVF) whose range is the set of positive real numbers. Two numerical examples using real and simulated data are presented. The numerical results of linear regression are identical to those obtained based on least-squares variance component estimation (LS-VCE) with positive initial values of variance components. The numerical results of two-dimensional affine transformation are shown to prevent negative variance components and precede those obtained by LS-VCE with a negative initial value of variance component. Both numerical examples verify the effectiveness of the PVF-NLS-VCE method whether the initial values of variance components are positive or negative. The proposed PVF-NLS-VCE is a simple, convenient and flexible method to achieve the non-negative estimates of variance components, which can reduce sensitivity to initial value selection and automatically guarantee a non-negative definite covariance matrix.

Nonnegative Estimation of Variance Components for a Nested Three-Way Random Model

A nonnegative variance estimation procedure is suggested for an unbalanced data where two factors are nested in another. Since the involved factors are all random, the approach is based on a nested three-way random model. The proposed method for the estimation of variance components is compared with Henderson’s Method I and III in view of the same estimation procedure based on the method of moments. Although both the Henderson’s Method I and III are known to be useful for the estimation of variance components for balanced or unbalanced data, they often yield negative values as variance estimates whereas the estimates by the suggested method are never be negative. Hence, it points out what makes this happen and discusses how to fix the problem. The proposed method shows how to define sums of squares and the orthogonal coefficient matrices that are necessary for the evaluation of expectations. All the matrices of the quadratic forms for computing sums of squares are symmetric and idempotent. It also reveals the individual coefficient of each variance component does not change from equation to equation.

Nonnegative variance component estimation for mixed-effects models

This paper suggests three available methods for finding nonnegative estimates of variance components of the random effects in mixed models. The three proposed methods based on the concepts of projections are called projection method I, II, and III. Each method derives sums of squares uniquely based on its own method of projections. All the sums of squares in quadratic forms are calculated as the squared lengths of projections of an observation vector; therefore, there is discussion on the decomposition of the observation vector into the sum of orthogonal projections for establishing a projection model. The projection model in matrix form is constructed by ascertaining the orthogonal projections defined on vector subspaces. Nonnegative estimates are then obtained by the projection model where all the coefficient matrices of the effects in the model are orthogonal to each other. Each method provides its own system of linear equations in a different way for the estimation of variance components; however, the estimates are given as the same regardless of the methods, whichever is used. Hartley’s synthesis is used as a method for finding the coefficients of variance components.

Nonnegative estimates of variance components in a two-way random model

Nonnegative estimates of variance components in a two-way random model

On non-negative estimation of variance components in mixed linear models

Alternative estimators have been derived for estimating the variance components according to Iterative Almost Unbiased Estimation (IAUE). As a result two modified IAUEs are introduced. The relative performances of the proposed estimators and other estimators are studied by simulating their bias, Mean Square Error and the probability of getting negative estimates under unbalanced nested-factorial model with two fixed crossed factorial and one nested random factor. Finally the Empirical Quantile Dispersion Graph (EQDG), which provides a comprehensive picture of the quality of estimation, is depicted corresponding to all the studied methods.

An alternative method for non-negative estimation of variance components

A typical problem of estimation principles of variance and covariance components is that they do not produce positive variances in general. This caveat is due, in particular, to a variety of reasons: (1) a badly chosen set of initial variance components, namely initial value problem (IVP), (2) low redundancy in functional model, (3) an improper stochastic model, and (4) data’s possibility of containing outliers. Accordingly, a lot of effort has been made in order to design non-negative estimates of variance components. However, the desires on non-negative and unbiased estimation can seldom be met simultaneously. Likewise, in order to search for a practical non-negative estimator, one has to give up the condition on unbiasedness, which implies that the estimator will be biased. On the other hand, unlike the variance components, the covariance components can be negative, so the methods for obtaining non-negative estimates of variance components are not applicable. This study presents an alternative method to non-negative estimation of variance components such that non-negativity of the variance components is automatically supported. The idea is based upon the use of the functions whose range is the set of all positive real numbers, namely positive-valued functions (PVFs), for unknown variance components in stochastic model instead of using variance components themselves. Using the PVF could eliminate the effect of IVP on the estimation process. This concept is reparameterized on the restricted maximum likelihood with no effect on the unbiasedness of the scheme. The numerical results show the successful estimation of non-negativity estimation of variance components (as positive values) as well as covariance components (as negative or positive values).

Consistent nonnegative estimates of variance components

In this paper, the estimation of variance components in the linear mixed model with two random effects is investigated. The class of combination estimates based on the quadratic invariant statistics and consistent nonnegative estimates are obtained. Furthermore, it is shown that the consistent nonnegative estimate dominates ANOVA estimate under some conditions.

Bayesian Interval Estimates of Variance Components Used in Quality Improvement Studies

ABSTRACT Sampling experiments are conducted in QA and process improvement studies to characterize measurement error and classify sources of process variation. Gage R&R studies and nested sampling studies are used in these situations to estimate the variance components. There are several methods available for estimating variance components from data obtained from these studies. However, interval estimates are not available in the output of commonly used software, and only approximate standard errors of estimated variance components have been proposed for unbalanced nested designs, such as the staggered nested plans. Bayesian methods provide a solution to this problem. Bayes methods guarantee nonnegative estimates of variance components, provide exact interval estimates of individual variance components and functions of individual variance components such as gage repeatability plus reproducibility, and can be computed easily with public domain software. This article demonstrates the value of Bayesian interval estimates and illustrates their calculation using the free Excel add-in BugsXLA.

Non-negative estimation of variance components in heteroscedastic one-way random-effects ANOVA models

There is a considerable amount of literature dealing with inference about the parameters in a heteroscedastic one-way random-effects ANOVA model. In this paper, we primarily address the problem of improved quadratic estimation of the random-effect variance component. It turns out that such estimators with a smaller mean squared error compared with some standard unbiased quadratic estimators exist under quite general conditions. Improved estimators of the error variance components are also established.

Nonnegative estimation of variance components in multivariate unbalanced mixed linear models with two variance components

In this paper, we address the problem of nonnegative estimation of variance components in multivariate unbalanced mixed linear models with two variance components, thus generalizing the results in Mathew et al. (J. Multivariate Anal. 5 (1994) 83). We illustrate our results with an example.

Improved non-negative estimation of variance components for exposure assessment.

Hygiene surveys of pollutants exposure data can be analyzed by analysis of variance (ANOVA) model with a random worker effect. Typically, workers are classified into homogeneous exposure groups, so it is very common to obtain a zero or negative ANOVA estimate of the between-worker variance (sigma2B). Negative estimates are not sensible and also pose problems for estimating the probability (theta) that in a job group, a randomly selected worker's mean exposure exceeds the occupational exposure standard. Therefore, it was suggested by Rappaport et al. to replace a non-positive estimate with an approximate one-sided 60% upper confidence bound. This article develops an alternative estimator, based on the upper tolerance interval suggested by Wang and Iyer. We compared the performance of the two methods using real data and simulations with respect to estimating both the between-worker variance and the probability of overexposure in balanced designs. We found that the method of Rappaport et al. has three main disadvantages: (i) the estimated sigma2B remains negative for some data sets; (ii) the estimator performs poorly in estimating sigma2B and theta with two repeated measures per worker and when true sigma2B is quite small, which are quite common situations when studying exposure; (iii) the estimator can be extremely sensitive to small changes in the data. Our alternative estimator offers a solution to these problems.

Improved Nonnegative Estimation of Variance Components in Balanced Multivariate Mixed Models

Consider the independent Wishart matrices S1W(Σ + λΘ,q1) and S2W(Σ, q2) where Σ is an unknown positive definite (p.d.) matrix, Θ is an unknown nonnegative definite (n.n.d.) matrix, and λ is a known positive scalar. For the estimation of Θ, a class of estimators of the form Θ̂(c,ϵ) = (c/λ){S1/q1 − ϵ(S2/q2)} (c ≥ 0, ϵ ≤ 1), uniformly better than the unbiased estimator Θ̂U = (1/λ){S1/q1 − S2/q2}, is derived (for the squared error loss function). Necessary and sufficient conditions are obtained For the existence of an n.n.d. estimator of the form Θ̂(c,ϵ) uniformly better than ΘU. It turns out that such an n.n.d. estimator exists only under restrictive conditions. However, for a suitable choice of c > 0, ϵ > 0, the estimator obtained by taking the positive part of Θ̂(c, ϵ) results in an n.n.d. estimator, say Θ̂(c, ϵ) +, that is uniformly better than Θ̂U. Numerical results indicate that in terms of mean squared error, Θ̂(c, ϵ) + performs much better than both Θ̂U and the restricted maximum likelihood estimator Θ̂REML of Θ. Similar results are also obtained for the nonnegative estimation of tr Θ and a′Θa, where a is an arbitrary nonzero vector. For estimating Σ, we have derived estimators that are claimed to be uniformly better than the unbiased estimator Σ̂U = S2/q2 under the squared error loss function and the entropy loss function. We have been able to establish the claim only in the bivariate case. Numerical results are reported showing the risk improvement of our proposed estimators of Σ.

Improved Nonnegative Estimation of Variance Components in Some Mixed Models with Unbalanced Data

Mixed models involving two variance components are considered when the data are unbalanced, one variance component corresponding to a random effect and a second variance component corresponding to experimental error. For estimating the random-effect variance component, several nonnegative estimators are derived. The estimators have explicit expressions and are easily computable. The performances of the various estimators are studied by simulating their mean squared errors and biases using some one-way models with unbalanced data, and specific recommendations are made on the choice of the nonnegative estimator to be used in practical applications. Some generalizations are obtained for estimating a randomeffect variance component in a more general model with unbalanced data. The analysis of variance (ANOVA) estimator and the proposed estimators are computed using real data for two examples. The ANOVA estimator has a value that is substantially different from the other estimators in the two examples. The simulation results and the examples suggest that the ANOVA approach should not be used for obtaining point estimates of variance components; instead, the more efficient procedures developed in this article should be preferred.

Improved Nonnegative Estimation of Variance Components in Some Mixed Models With Unbalanced Data

Mixed models involving two variance components are considered when the data are unbalanced, one variance component corresponding to a random effect and a second variance component corresponding to experimental error. For estimating the random-effect variance component, several nonnegative estimators are derived. The estimators have explicit expressions and are easily computable. The performances of the various estimators are studied by simulating their mean squared errors and biases using some one-way models with unbalanced data, and specific recommendations are made on the choice of the nonnegative estimator to be used in practical applications. Some generalizations are obtained for estimating a randomeffect variance component in a more general model with unbalanced data. The analysis of variance (ANOVA) estimator and the proposed estimators are computed using real data for two examples. The ANOVA estimator has a value that is substantially different from the other estimators in the two examples. The sim...

Nonnegative estimation of variance components in an unbalanced one way random effects model

The unbalanced one way random effects model with unequal error variances is considered. Two different procedures of estimating the unknown variance components are presented. At first the minimum norm quadratic unbiased estimators are examined, whose realizations may become negative. Nonnegative minimum norm quadratic minimum biased estimators are also derived. The biases and the mean squared errors of these estimators are derived and compared.

Nonnegative estimation of variance components in unbalanced mixed models with two variance components

An unbalanced mixed linear model with two variance components is considered, one variance component (say σ 1 2 ≥ 0) corresponding to a random effect (treatments) and a second variance component (say σ 2 > 0) corresponding to the experimental errors. Sufficient conditions are obtained under which there will exist a nonnegative invariant quadratic estimator (IQE) having a uniformly smaller mean squared error (MSE) than every unbiased IQE of σ 1 2. In particular, for the one-way unbalanced ANOVA model, necessary and sufficient conditions are also obtained for a multiple of the usual treatment sum of squares to uniformly dominate the ANOVA estimator of σ 1 2. For estimating σ 2, it is shown that the best multiple of the residual sum of squares can be improved by using nonquadratic estimators. One such estimator is obtained using the idea of a testimator ( Stein, 1964, Ann. Inst. Statist. Math. 16 155–160). A second estimator is obtained following the approach in Strawderman (1974, Ann. Statist. 2 190–198). Numerical results regarding the performance of the proposed estimators of σ 1 2 are also reported.

An empirical study of the estimation of eigenvalues in connection with non-negative estimation of variance components of a one-way random effect model

A variance component is, by definition, positive. Nevertheless, the MLE and ANOVA estimate of treatment variance of σ 2 τ, of a one-way random effect model can be negative. Recently, a multivariate approach has been suggested by Sutradhar (1988) which always produces non-negative estimate of σ 2 τ. The multivariate approach exploits the estimates of the eigenvalues of the covariance matrix of the model in estimating variance components. The success of the estimation procedure depends on the precise estimation of the eigenvalues. This paper through a simulation study examines the behaviour of the estimates of eigenvalues of the covariance matrix of the model for various sample sizes. Several approaches for the estimation of eigenvalues have been considered and compared.

Estimation of Variance Components Using Residuals

Abstract We consider the estimation of variance components in a general analysis of variance model by use of residuals. Unbiased estimation by linear functions of squares and cross-products of residuals is shown to be equivalent to estimation by quadratic forms with an invariance property. This approach leads to alternative derivations of C.R. Rao's MIVQUE and MINQUE by using well-known results in linear model theory. Further, the method extends easily to utilize a priori knowledge of the parameter space, which ensures, for example, nonnegative estimates of variance components; when estimates are constrained to a strictly positive region, an iterative feedback procedure is made feasible.

Non-Negative Estimates of Variance Components

Non-Negative Estimates of Variance Components

Non-Negative Estimates of Variance Components

In many analyses of variance it is appropriate to consider the effects as randomly chosen from a population; thus providing a model II analysis in the terminology of Eisenhart. Interest then centers on the variances of the effects and on the components of variance. The usefulness of this technique is limited by the frequent occurrence of negative estimates of the variance components which are by definition non-negative quantities. This paper presents a method of data analysis, applicable to many of the more popular models, which always yields non-negative estimates. Recently a simple algorithm, the pool-the-minimum-violator algorithm, has been developed [11] as a result of maximizing the likelihood function of the mean squares subject to a set of constraints. This paper has been prepared as a less mathematical companion article to [11] in order to treat the more applied aspects of the problem. 2. AN EXAMPLE