Gauss-Markov Estimator Research Articles

Linear minimum variance estimation fusion

This paper shows that a general multisensor unbiased linearly weighted estimation fusion essentially is the linear minimum variance (LMV) estimation with linear equality constraint, and the general estimation fusion formula is developed by extending the Gauss-Markov estimation to the random parameter under estimation. First, we formulate the problem of distributed estimation fusion in the LMV setting. In this setting, the fused estimator is a weighted sum of local estimates with a matrix weight. We show that the set of weights is optimal if and only if it is a solution of a matrix quadratic optimization problem subject to a convex linear equality constraint. Second, we present a unique solution to the above optimization problem, which depends only on the covariance matrix C k . Third, if a priori information, the expectation and covariance, of the estimated quantity is unknown, a necessary and sufficient condition for the above LMV fusion becoming the best unbiased LMV estimation with known prior information as the above is presented. We also discuss the generality and usefulness of the LMV fusion formulas developed. Finally, we provide an off-line recursion of C k for a class of multisensor linear systems with coupled measurement noises.

Objective analysis for coastal regimes

Statistical interpolation of coastal oceanic fields is addressed within the general framework of Gauss–Markov estimation. Computation of the prior covariance of the field to be estimated is posed as a stochastically-forced differential equation subject to coastal boundary conditions with inhomogeneous, anisotropic parameters. Solution is readily implemented in standard finite element methodology. Examples illustrate the method for idealized one-dimensional situations. Analysis of real biological data from Massachusetts and Cape Cod Bay is shown, using a covariance function derived from velocity and diffusivity fields computed with a hydrodynamic model. The procedure defaults to standard OA methods using distance-based covariance functions, far from boundaries and inhomogeneities.

Extended Gauss–Markov Theorem for Nonparametric Mixed-Effects Models

The Gauss–Markov theorem provides a golden standard for constructing the best linear unbiased estimation for linear models. The main purpose of this article is to extend the Gauss–Markov theorem to include nonparametric mixed-effects models. The extended Gauss–Markov estimation (or prediction) is shown to be equivalent to a regularization method and its minimaxity is addressed. The resulting Gauss–Markov estimation serves as an oracle to guide the exploration for effective nonlinear estimators adaptively. Various examples are discussed. Particularly, the wavelet nonparametric regression example and its connection with a Sobolev regularization is presented.

Gauss–Markov and weighted least-squares estimation under a general growth curve model

Gauss–Markov estimator of X 1BX ′ 2 under a general growth curve model {Y,X 1BX ′ 2,V 2⊗V 1} is given. Necessary and sufficient conditions for equality between A 1YA ′ 2 and B 1YB ′ 2 , where A i=X i(X ′ iW iX i) +X ′ iW i , B i=X i(X ′ iS iX i) +X ′ iS i , with W i,S i being any matrix, i=1,2, are derived. Finally, a number of criteria for A 1YA ′ 2 to coincide with the Gauss–Markov estimator of X 1BX ′ 2 are established.

The impact of ROI definition on estimator performance on FBP reconstructed Ga-67 SPECT images

Quantitative accuracy of reconstructed images depends not only on the reconstruction strategy, but also on the definition of the estimator. The authors investigate estimators based on: (1) a dilation of the true ROI, (2) thresholding the reconstructed image, (3) edge detection on the reconstructed image, and (4) a quasi-Gauss-Markov (QGM) estimator which partially compensates for the image acquisition and reconstruction process. The Gauss-Markov estimator is also briefly compared to these first 4 for a simpler imaging problem. The task is to find the activity in small (1 cm diameter) lesions in simulated images of thoracic gallium SPECT scans. The authors consider 35 lesion locations. Reconstruction is done with Chang-corrected FBP. The authors consider both the best accuracy achievable with each estimator and how sensitive this accuracy is to errors in the specification of the lesion's size and position. A signal-to-noise ratio (SNR) which combines both bias and variance is used as the basis for comparison. The results show that while the estimators based on a dilation of the true ROI or the thresholded image can produce very high SNR, they are both very sensitive to errors in lesion size and position. The edge-detection estimator and the QGM estimator both have lower optimal SNR, but are less sensitive to some lesion-specification errors.

A Generalization of Rao's Covariance Structure with Applications to Several Linear Models

This paper presents a generalization of Rao's covariance structure. In a general linear regression model, we classify the error covariance structure into several categories and investigate the efficiency of the ordinary least squares estimator (OLSE) relative to the Gauss–Markov estimator (GME). The classification criterion considered here is the rank of the covariance matrix of the difference between the OLSE and the GME. Hence our classification includes Rao's covariance structure. The results are applied to models with special structures: a general multivariate analysis of variance model, a seemingly unrelated regression model, and a serial correlation model.

The Falstaff estimator

Correcting for heteroscedasticity in GMM estimation of the linear model can improve upon the Gauss-Markov estimator even when there is no heteroscedasticity to correct.

Least upper bound for the covariance matrix of a generalized least squares estimator in regression with applications to a seemingly unrelated regression model and a heteroscedastic model

In a general normal regression model, this paper first derives the least upper bound (LUB) for the covariance matrix of a generalized least squares estimator (GLSE) relative to the covariance matrix of the Gauss-Markov estimator. Second the result is applied to the (unrestricted) Zellner estimator in an N-equation seemingly unrelated regression (SUR) model and to the GLSE in a heteroscedastic model.

Joint detection with coherent receiver antenna diversity in CDMA mobile radio systems

In code division multiple access (CDMA) mobile radio systems, both intersymbol interference and multiple access interference arise which can be combated by using either elaborate optimum or favorable suboptimum joint detection (JD) techniques. Furthermore, the time variation of the radio channels leads to degradations of the receiver performance. These degradations can be reduced by applying diversity techniques. Using coherent receiver antenna diversity (CRAD) is especially attractive because only the signal processing at the receiver must be modified. In the present paper, the application of CRAD to the more critical uplink of CDMA mobile radio systems with suboptimum JD techniques is investigated for maximal-ratio combining. The authors study six different suboptimum JD techniques based on decorrelating matched filtering, Gauss-Markov estimation, and minimum mean square error estimation with and without decision feedback. These six suboptimum JD techniques which are well-known for single antenna receivers are extended for the application to CRAD. A main concern of the paper is the determining of the SNR performance of the presented JD techniques for CRAD and the achievable average uncoded bit error probabilities for the transmission over rural area, typical urban and bad urban mobile radio channels are determined.

On D-optimal designs for linear models under correlated observations with an application to a linear model with multiple response

Abstract In the general linear model we set conditions under which an exact D-optimal design for uncorrelated observations with common variance is also D-optimal for correlated observations. Further we determine conditions under which approximate D-optimal designs can be considered as approximate D-optimal designs for correlated observations. Then these results are applied to a regression model with multiple response generalizing Theorem 1 of Krafft and Schaefer (J. Multivariate Anal. 42, 1992). In the above context, however, a serious problem may arise if the covariance matrix is not known; for the Gauss-Markov estimator with respect to a D-optimal design does not need to be calculable for the correlated case. This leads to D-optimal-invariant designs introduced by Bischoff (Ann. Inst. Statist. Math., 44, 1992); such a design τ∗ remains D-optimal when the covariance matrix is changed, and additionally the Gauss-Markov estimator with respect to the design τ∗ stays fixed. For regression models with multiple response we determine classes of covariance matrices for which a D-optimal design for uncorrelated observations with common variance is D-optimal-invariant. As examples we consider linear models where each response belongs to a regression model with intercept term.

Inadmissibility and admissibility results for unbiased loss estimators based on gauss-markov estimators

LetY be distributed according to ann-variate normal distribution with a meanXβ and a nonsingular covariance matrixσ 2 V, where bothX andV are known,β eR p is a parameter,σ > 0 is known or unknown. Denote $$\hat \beta = (X'V^{ - 1} X)^ - X'V^{ - 1} Y$$ and $$S^2 = (Y - X\hat \beta )'V^{ - 1} (Y - X\hat \beta )$$ . Assume thatFβ is linearly estimable. Whenσ is known, it is proved that the unbiased loss estimatorσ 2tr(F(X′V −1 X)− F′) of $$(F\hat \beta - F\beta )'(F\hat \beta - F\beta )$$ is admissible for rank (F)=k≤4 and inadmissible fork ≥ 5 with the squared error loss $$[a - (F\hat \beta - F\beta )'(F\hat \beta - F\beta )]^2$$ . Whenσ is unknown and rank (X) <n, it is established that the loss estimatorcS 2, wherec is any nonnegative constant, of $$(F\hat \beta - F\beta )'(F\hat \beta - F\beta )$$ is inadmissible and that the unbiased loss estimator tr(F(X′V −1 X)− F′) of $$\sigma ^{ - 2} (F\hat \beta - F\beta )'(F\hat \beta - F\beta )$$ is admissible fork ≤ 4, and inadmissible fork ≥ 5 with squared error loss.

THE ESTIMATION OF FIXED EFFECTS IN A MIXED LINEAR MODEL

The estimation of fixed effects is considered for small, unbalanced, mixed linear models. The two-stage estimator, in which the variance components are first estimated by ML or REML, is compared to the intra-block (IB) estimator, the ordinary least squares (OLS) estimator (ignoring the random effects) and the Gauss-Markov (GM) estimator. Comparison is made, based on 100 simulated data sets each, for 6 designs (3 BIBD's and 3 unbalanced designs). In comparing loss of information, relative to the GM lower bound, the two-stage procedures (using either ML or REML) are recommended for all but the smallest and least balanced design. The study also compared estimates of the variance of the two-stage estimators, using either the GM lower bound or the Kackar-Harville (KH) approximation. Estimators of the variance using REML estimates of the variance components are recommended, since estimators using ML estimates were seriously biased downward for all designs considered.

The Restricted Singular Value Decomposition: Properties and Applications

The restricted singular value decomposition (RSVD) is the factorization of a given matrix, relative to two other given matrices. It can be interpreted as the ordinary singular value decomposition with different inner products in row and column spaces. Its properties and structure are investigated in detail as well as its connection to generalized eigenvalue problems, canonical correlation analysis and other generalizations of the singular value decomposition. Applications that are discussed include the analysis of the extended shorted operator, unitarily invariant norm minimization with rank constraints, rank minimization in matrix balls, the analysis and solution of linear matrix equations, rank minimization of a partitioned matrix and the connection with generalized Schur complements, constrained linear and total linear least squares problems, with mixed exact and noisy data, including a generalized Gauss-Markov estimation scheme. Two constructive proofs of the RSVD in terms of other generalizations of the ordinary singular value decomposition are provided as well.

The efficiency of least squares estimators of a seemingly unrelated regression model

A measure defined by Bloomfield and Watson (1975)is applied to measure the relative performance between the ordinary least squares estmates (OLES) and Gauss-Markov estimates for a two-equation model of a seemingly unrelated regression system. Results obtained are compared with those obtained by Zellner (1963), Ravankar (1974) and Chang and Lin (1984). It can be seen that this efficeiency depends only the correlation coefficient of disturbances between two regression equations and is independent of the sample size.

A New Algorithm of the Recursive Generalized Least-squares Method

A new method for estimation of ARMA model parameters is considered. A recursive version of Gauss-Markov estimate based on computation of covariance matrix elements is developed. The method uses special features of the covariance matrix. Three versions of estimators are obtained depending on the method of estimation of covariance matrix elements. Extensive simulations are made and the results of the estimation by the proposed method are compared with these ones of least squares, generalized least squares and instrumental variable methods. The results obtained by use of the new algorithm are comparable or better than these ones obtained by the existing methods.

Concentration inequalities for Gauss-Markov estimators

Let M be the regression subspace and γ the set of possible covariances for a random vector Y. The linear model determined by M and γ is regular if the identity is in γ and if Σ( M)⊆ M for all Σ∈γ. For such models, concentration inequalities are given for the Gauss-Markov estimator of the mean vector under various distributional and invariance assumptions on the error vector. Also, invariance is used to establish monotonicity results relative to a natural group induced partial ordering.

Projection filter, Wiener filter, and Karhunen-Loève subspaces in digital image restoration

In image processing by computer, the transformation from the original continuous-domain image to the degraded and sampled discrete observation image is usually modelled as a linear transformation with additive noise. The relation between two types of filters, the Wiener filter (WF) and the projection filter (PF), for the restoration of the original image from the observation is discussed. The latter is based on the same principle as pseudoinverse filtering but also suppresses the additive noise. The PF and the WF are shown to be closely related under a condition depending on the degradation-sampling operator and the Karhunen-Loève expansion for the family of original images. The relation between the PF and the Gauss-Markov estimator is also clarified.

State Estimation for Time Discrete Bilinear Filters

Abstract It is shewn that a tine discrete bilinear dynamic estimation filter can be reformulated so as to fit semi-linear estimation. The semi-linear estimation principle is considered as a generalisation of the Gauss-Markov estimation for the particular case of constant coefficient systems. This results in the existence of an estimate which is optimal in the sense that it is bias free and minimum variance. The actual solution obtainable by this method requires an approximation at the level of the weight matrices which can be expected to be very insensitive in practical applications.

A Nonlinear Version of the Gauss-Markov Theorem

Abstract This article gives a nonlinear version of the Gauss—Markov theorem. It is shown that in a linear regression model y = Xβ + u, the lower bound for the risk matrix of a nonlinear estimator of β belonging to a certain class is the covariance matrix of the Gauss—Markov estimator, provided the distribution of error term u belongs to the class of elliptically symmetric distributions with second moments.

A Nonlinear Version of the Gauss-Markov Theorem

Abstract This article gives a nonlinear version of the Gauss—Markov theorem. It is shown that in a linear regression model y = Xβ + u, the lower bound for the risk matrix of a nonlinear estimator of β belonging to a certain class is the covariance matrix of the Gauss—Markov estimator, provided the distribution of error term u belongs to the class of elliptically symmetric distributions with second moments.