Linear Minimum Variance Estimation Research Articles

Robust estimation algorithm based on prior probability statistics

AbstractFor the nonlinear systems with uncertain system parameters, based on the recursive linear minimum variance estimation framework, the effect of the system parameters on the mean squared error (MSE) is analyzed, that is, the greater the system parameters deviation (whether they are positive or negative deviation) is, the greater the MSE is. Based on the analysis results and prior probability statistics (probability distributions or sampling distributions), an innovative robust estimation algorithm is proposed for nonlinear systems with uncertain system parameters. Three examples verify the effectiveness of the proposed algorithm.

A study on input noise second-order filtering and smoothing of linear stochastic discrete systems with packet dropouts

We investigate non-Gaussian noise second-order filtering and fixed-order smoothing problems for non-Gaussian stochastic discrete systems with packet dropouts. We present a novel Kalman-like nonlinear non-Gaussian noise estimation approach based on the packet dropout probability distribution and polynomial filtering technique. By means of properties of Kronecker product we first introduce a second-order polynomial extended system and then analyze the means and variances of the Kronecker powers of the extended system noises. To generate noise estimators in forms of filtering and smoothing, we use the innovation approach. We give an example to illustrate that the presented algorithm has better robustness against packet dropouts than conventional linear minimum variance estimation.

Application of linear minimum variance estimation to the multi-model ensemble of atmospheric radioactive Cs-137 with observations

Abstract. Great efforts have been made to simulate atmospheric pollutants, but their spatial and temporal distributions are still highly uncertain. Observations can measure their concentrations with high accuracy but cannot estimate their spatial distributions due to the sporadic locations of sites. Here, we propose an ensemble method by applying a linear minimum variance estimation (LMVE) between multi-model ensemble (MME) simulations and measurements to derive a more realistic distribution of atmospheric pollutants. The LMVE is a classical and basic version of data assimilation, although the estimation itself is still useful for obtaining the best estimates by combining simulations and observations without a large amount of computer resources, even for high-resolution models. In this study, we adopt the proposed methodology for atmospheric radioactive caesium (Cs-137) in atmospheric particles emitted from the Fukushima Daiichi Nuclear Power Station (FDNPS) accident in March 2011. The uniqueness of this approach includes (1) the availability of observed Cs-137 concentrations near the surface at approximately 100 sites, thus providing dense coverage over eastern Japan; (2) the simplicity of identifying the emission source of Cs-137 due to the point source of FDNPS; (3) the novelty of MME with the high-resolution model (3 km horizontal grid) over complex terrain in eastern Japan; and (4) the strong need to better estimate the Cs-137 distribution due to its inhalation exposure among residents in Japan. The ensemble size is six, including two atmospheric transport models: the Weather Research and Forecasting – Community Multi-scale Air Quality (WRF-CMAQ) model and non-hydrostatic icosahedral atmospheric model (NICAM). The results showed that the MME that estimated Cs-137 concentrations using all available sites had the lowest geometric mean bias (GMB) against the observations (GMB =1.53), the lowest uncertainties based on the root mean square error (RMSE) against the observations (RMSE =9.12 Bq m−3), the highest Pearson correlation coefficient (PCC) with the observations (PCC =0.59) and the highest fraction of data within a factor of 2 (FAC2) with the observations (FAC2 =54 %) compared to the single-model members, which provided higher biases (GMB =1.83–4.29, except for 1.20 obtained from one member), higher uncertainties (RMSE =19.2–51.2 Bq m−3), lower correlation coefficients (PCC =0.29–0.45) and lower precision (FAC2 =10 %–29 %). At the model grid, excluding the measurements, the MME-estimated Cs-137 concentration was estimated by a spatial interpolation of the variance used in the LMVE equation using the inverse distance weights between the nearest two sites. To test this assumption, the available measurements were divided into two categories, i.e. learning and validation data; thus, the assumption for the spatial interpolation was found to guarantee a moderate PCC value (> 0.4) within an approximate distance of at least 70 km. Extra sensitivity tests for several parameters, i.e. the site number and the weighting coefficients in the spatial interpolation, the time window in the LMVE and the ensemble size, were performed. In conclusion, the important assumptions were the time window and the ensemble size; i.e. a shorter time window (the minimum in this study was 1 h, which is the observation interval) and a larger ensemble size (the maximum in this study was six, but five is also acceptable if the members are effectively selected) generated better results.

Distributed Fusion Filter for Nonlinear Multi-Sensor Systems With Correlated Noises

This paper is concerned with distributed fusion (DF) estimation problem for nonlinear multi-sensor systems with correlated noises. Based on a recursive linear minimum variance estimation (RLMVE) framework, a novel filter is developed. It is proved that the RLMVE-based filter and the existing de-correlated filter have the functional equivalence. Then, for multi-sensor cases, cross-covariance matrices between any two local filters are derived. Based on the RLMVE-based filter and cross-covariance matrices, a DF filter weighted by matrices is proposed in the sense of linear minimum variance. Finally, based on the existing de-correlated filter, the algorithm of cross-covariance for de-correlated systems and the DF algorithm weighted by matrices, a de-correlated DF filtering algorithm is proposed. An example verifies the effectiveness of the proposed RLMVE-based DF filter.

PERBANDINGAN REGRESI KOMPONEN UTAMA DENGAN REGRESI KUADRAT TERKECIL PARSIAL PADA INDEKS PEMBANGUNAN MANUSIA PROVINSI JAWA TIMUR

The multiple regression classic assumptions are used to give linear unbiased and minimum variance estimator. In Human Development Index (HDI) and influencing factors in East Java, there are two variables with VIF more than 10 so the assumption of non-multicollinearity is not fulfilled. Principal component regression (PCR) and partial least squares regression (PLS-R) can solve this problem. By doing principal component analysis, there are two linear combinations to take as the new independent variables which are free from collinearity. In the PLS-R, NIPALS algorithm is used to calculate the components and other structures and to estimate the parameter. While in PCR all independent variables are significant, the percentage of households with drinking water is feasibles is not significant in the model. PLS-R’s is 95,85% is greater than PCR’s = 93,42%. PCR’s PRESS = 81,78 is greater than PLS-R’s PRESS = 61,0595.Keywords: Human Development Index (HDI), Multicollinearity, Principal Component Regression, Partial Least Squares Regression, , PRESS

Finite Element Model Updating in Bridge Structures Using Kriging Model and Latin Hypercube Sampling Method

Computational cost reduction and best model updating method seeking are the key issues during model updating for different kinds of bridges. This paper presents a combined method, Kriging model and Latin hypercube sampling method, for finite element (FE) model updating. For FE model updating, the Kriging model is serving as a surrogate model, and it is a linear unbiased minimum variance estimation to the known data in a region which have similar features. To predict the relationship between the structural parameters and responses, samples are preselected, and then Latin hypercube sampling (LHS) method is applied. To verify the proposed algorithm, a truss bridge and an arch bridge are analyzed. Compared to the predicted results obtained by using a genetic algorithm, the proposed method can reduce the computational time without losing the accuracy.

Constrained Extended Kalman Filter for Target Tracking in Directional Sensor Networks

The target tracking problem in directional sensor networks (DSNs) is attracting increasing attention. Unlike the traditional omnidirectional sensor, a directional sensor has a special angle of view. It can offer direction information rather than just the sensing signal measurement with respect to the detected target. The existing tracking approaches in DSNs always separately consider the direction and measurement information; they hardly promise the tracking performance of minimum variance. In this paper, the field of view of directional sensor is approximated to a rectangle; as such the constrained area in which the target is bound to be is constructed. Then, the target tracking problem is formulated as a constrained estimation problem, and a constrained extended Kalman filter (CEKF) tracking algorithm integrating the direction and measurement information is presented; its structural and statistical properties are rigorously derived. It is proved that CEKF is the linear unbiased minimum variance estimator, and CEKF can yield a smaller error covariance than the unconstrained traditional extended Kalman filter using only sensor measurements. Simulation results show that the CEKF has superior tracking performance for directional wireless networks.

Linear Unbiased Optimal Filter for Discrete-Time Systems with One-Step Random Delays and Inconsecutive Packet Dropouts

This paper is concerned with the linear unbiased minimum variance estimation problem for discrete-time stochastic linear control systems with one-step random delay and inconsecutive packet dropout. A new model is developed to describe the phenomena of the one-step delay and inconsecutive packet dropout by employing a Bernoulli distributed stochastic variable. Based on the model, a recursive linear unbiased optimal filter in the linear minimum variance sense is designed by the method of completing the square. The solution to the linear filter is given by three equations including a Riccati equation, a Lyapunov equation and a simple difference equation. A sufficient condition for the existence of the steady-state filter is given. A simulation shows the effectiveness of the proposed algorithm. DOI: http://dx.doi.org/10.11591/telkomnika.v11i11.3523

Design and Simulation of Servo System of Based on Kalman Filter

In order to enhance metrical precision of radar on the vessel, Kalman filter based on Singer model is designed to filter angle-error signals in servo system of the radar in this paper. Precision of angle-error signal in the servo system is a key factor influencing precision of tracking and control. But it inevitably corrupted by external disturbances and internal noise during transmission and detection. Kalman filter, which is a kind of linear minimum variance estimation, can estimate state vector at any time when a parameter variable changing with time (state vector) and a linear model (system model) are provided. Therefore, its extremely suitable for filtering the angle-error signals in servo system. Experiments on practical signal show that Kalman filter can reduce noise of the angle-error signals effectively and hence enhancing metrical precision of the radar.

Application of Kalman Filter in the Large-Diameter Antenna of Ship-Borne

In order to enhance metrical precision of radar on the vessel, Kalman filter based on Singer model is designed to filter angle-error signals in servo system of the radar in this paper. Precision of angle-error signal in the servo system is a key factor influencing precision of tracking and control. But it inevitably corrupted by external disturbances and internal noise during transmission and detection. Kalman filter, which is a kind of linear minimum variance estimation, can estimate state vector at any time when a parameter variable changing with time (state vector) and a linear model (system model) are provided. Therefore, it’s extremely suitable for filtering the angle-error signals in servo system. Experiments on practical signal show that Kalman filter can reduce noise of the angle-error signals effectively and hence enhancing metrical precision of the radar.

Phase-Based, Time-Domain Estimation of the Frequency and Phase of a Single Sinusoid in AWGN—The Role and Applications of the Additive Observation Phase Noise Model

This paper presents the theoretical foundation for time-domain, phase-based estimation of the frequency and phase of a single sinusoid in additive white Gaussian noise (AWGN), analogous to the theoretical foundation provided by Rife and Boorstyn for frequency-domain, Fourier-transform-based estimation. It is shown from the maximum a posteriori probability (MAP) and the maximum likelihood (ML) estimation principles that with the additive observation phase noise (AOPN), due to the AWGN, being described by its a posteriori distribution conditioned on the received signal magnitude, the received signal phase is a sufficient statistic for estimating the single-sinusoid angle parameters. Using a geometric approach, the exact statistical model for the AOPN is derived, where the a posteriori probability density function (pdf) and the corresponding a priori pdf are given by explicit, closed-form expressions that are valid for arbitrary signal-to-noise ratios (SNRs). The a posteriori pdf is Tikhonov, and is of particular interest as it establishes the AOPN model for phase-based frequency/phase MAP/ML estimation in the time domain. It is further illustrated that the results derived can yield various AOPN models as special cases, and the underlying physical insights and interconnections that exist among these models are revealed. It is shown that the model derived by Tretter is an ultimate specialization in the high SNR limit of the AOPN models developed here. For high SNR, the a posteriori Tikhonov pdf can be accurately approximated by a Gaussian distribution, which leads to the best linearized AOPN model. The applications of these AOPN models to the design of linear estimators, including the linear minimum mean square error (LMMSE) estimator, the linear minimum variance estimator, and the LMMSE implementation of the weighted phase averager are presented, and their estimation performances are compared through computer simulations, with the Cramer-Rao lower bound (CRLB) and the Bayesian CRLB as the benchmark. To facilitate estimator design, the a priori statistical models of the frequency and phase are proposed from the information-theoretic perspective, and an improved phase unwrapping algorithm over that given by Fu and Kam is presented. It is shown that by incorporating all the information available in the AOPN, the estimation accuracy can be much improved.

Concentration Ellipsoids, Their Planes of Support, and the Linear Regression Model

The relationship between the concentration ellipsoid of a random vector and its planes of support is exploited to provide a geometric derivation and interpretation of existing results for a general form of the linear regression model. In particular, the planes of support whose points of tangency to the ellipsoid are contained in the range (or column space) of the design matrix are the source of all linear unbiased minimum variance estimators. The connection between this idea and estimators based on projections is explored, as is also its use in obtaining and interpreting some existing relative efficiency results.

Prediction of Sediment Concentration in Rivers by Recursive Least-Squares and Linear Minimum Variance Estimators

AbstractPeaks of suspended sediment concentration in rivers appear before peak discharge appears. The observation of discharge is usually done in many rivers for river management, but the suspended sediment concentration is not always observed. The suspended sediment concentration is expressed by a sediment-rating curve over a long period of time. Coefficients in the sediment-rating curve are time varying. The time variation of the suspended sediment concentration is predicted by the sediment-rating curve with the recursive least-squares and the linear minimum variance estimators. Because the sediment-rating curve is linearized by the logarithmic transformation, we can apply the simple linear models such as recursive least-squares and linear minimum variance estimators for the prediction of the suspended sediment concentration. These methods indicate that the coefficients in the sediment-rating curve are time varying. These methods can estimate sediment discharge by identifying the coefficients in the sed...

Analysis of efficient markets

A simple trading model is presented in which Bayes’ rule is used to aggregate traders’ forecasts about risky assets’ future returns. In this financial market, Bayes’ rule operates like an omnipotent market-maker performing functions that in 1776 Adam Smith attributed to an “invisible hand.” We have analyzed two distinct cases: in the first scenario, the traders’ forecast errors are uncorrelated, and in the second scenario, the traders’ forecast errors are correlated. The contribution of our paper is fourfold: first, we prove that the “efficient market” mean-return can be expressed as a complex linear combination of the traders’ forecasts. The weights depend on the forecast variances, as well as on the correlations among the traders’ forecasts. Second we show that the “efficient” variance is equal to the inverse of the sum of the traders’ precision errors, and is also related to the correlations among the traders’ forecast errors. Third, we prove that the efficient market return is the best linear minimum variance estimator (BLMVE) of the security’s mean return (in the sense that it minimizes the sum of the traders’ mean squared forecast errors). Thus, an efficient market aggregates traders’ heterogeneous information in an optimal way. Fourth, we prove that an efficient market produces a mean return (price) as a Blackwell sufficient (most informative) experiment among all possible aggregated expected return (price) forecasts.

Linear minimum variance estimators for systems with bounded random measurement delays and packet dropouts

For discrete-time stochastic linear systems with bounded random measurement delays and packet dropouts, the optimal estimators including filter, predictor and smoother are developed in the linear minimum variance sense based on the innovation analysis approach. Some binary distributed random variables with known distributions are employed to describe the phenomenon of random delays and packet dropouts. Compared with the augmented approach, the computational cost is reduced. Furthermore, the proposed algorithm also gives a suboptimal estimate for systems with unbounded delays and packet dropouts by selecting a sufficient large upper bound. A simulation shows the effectiveness of the proposed algorithms.

Multi‐sensor track‐to‐track fusion via linear minimum variance sense estimators

AbstractAn integrated approach that consists of sensor‐based filtering algorithms, local processors, and a global processor is employed to describe the distributed fusion problem when several sensors execute surveillance over a certain area. For the sensor tracking systems, each filtering algorithm utilized in the reference Cartesian coordinate system is presented for target tracking, with the radar measuring range, bearing, and elevation angle in the spherical coordinate system (SCS). For the local processors, each track‐to‐track fusion algorithm is used to merge two tracks representing the same target. The number of 2‐combinations of a set with N distinct sensors is considered for central track fusion. For the global processor, the data fusion algorithms, simplified maximum likelihood (SML) estimator and covariance matching method (CMM), based on linear minimum variance (LMV) estimation fusion theory, are developed for use in a centralized track‐to‐track fusion situation. The resulting global fusers can be implemented in a parallel structure to facilitate estimation fusion calculation. Simulation results show that the proposed SML estimator has a more robust capability of improving tracking accuracy than the CMM and the LMV estimators. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society

Exact phase noise model for single-tone frequency estimation in noise

The a posteriori and the a priori statistical models for the measurement phase noise (MPN) in estimating the frequency and phase of a single tone over the additive white Gaussian noise channel are derived. Both models are described by exact, explicit, closed-form probability density functions (pdf's) that are valid for arbitrary signal-to-noise ratio. It is shown that as far as estimating the single-tone angle parameters is concerned, the received signal phase is a sufficient statistic, provided that the MPN is modelled by the a posteriori pdf. It is illustrated that the results derived can yield various MPN models as special cases, and the underlying physical insights and interconnections that exist among these models are revealed. The application of these models to the design of a linear minimum variance estimator is presented.

A history of parametric statistical inference from Bernoulli to Fisher, 1713-1935

The Three Revolutions in Parametric Statistical Inference.- The Three Revolutions in Parametric Statistical Inference.- Binomial Statistical Inference.- James Bernoulli's Law of Large Numbers for the Binomial, 1713, and Its Generalization.- De Moivre's Normal Approximation to the Binomial, 1733, and Its Generalization.- Bayes's Posterior Distribution of the Binomial Parameter and His Rule for Inductive Inference, 1764.- Statistical Inference by Inverse Probability.- Laplace's Theory of Inverse Probability, 1774-1786.- A Nonprobabilistic Interlude: The Fitting of Equations to Data, 1750-1805.- Gauss's Derivation of the Normal Distribution and the Method of Least Squares, 1809.- Credibility and Confidence Intervals by Laplace and Gauss.- The Multivariate Posterior Distribution.- Edgeworth's Genuine Inverse Method and the Equivalence of Inverse and Direct Probability in Large Samples, 1908 and 1909.- Criticisms of Inverse Probability.- The Central Limit Theorem and Linear Minimum Variance Estimation by Laplace and Gauss.- Laplace's Central Limit Theorem and Linear Minimum Variance Estimation.- Gauss's Theory of Linear Minimum Variance Estimation.- Error Theory. Skew Distributions. Correlation. Sampling Distributions.- The Development of a Frequentist Error Theory.- Skew Distributions and the Method of Moments.- Normal Correlation and Regression.- Sampling Distributions Under Normality, 1876-1908.- The Fisherian Revolution, 1912-1935.- Fisher's Early Papers, 1912-1921.- The Revolutionary Paper, 1922.- Studentization, the F Distribution, and the Analysis of Variance, 1922-1925.- The Likelihood Function, Ancillarity, and Conditional Inference.

Kalman filtering for time-delayed linear systems

This paper is to study the linear minimum variance estimation for discrete-time systems. A simple approach to the problem is presented by developing re-organized innovation analysis for the systems with instantaneous and double time-delayed measurements. It is shown that the derived estimator involves solving three different standard Kalman filtering with the same dimension as the original system. The obtained results form the basis for solving some complicated problems such as H∞ fixed-lag smoothing, preview control, H∞ filtering and control with time delays.

Kalman filtering for multiple time-delay systems

This paper is to study the linear minimum variance estimation for discrete-time systems with instantaneous and l -time delayed measurements by using re-organized innovation analysis. A simple approach to the problem is presented in this paper. It is shown that the derived estimator involves solving l + 1 different standard Kalman filtering with the same dimension as the original system.