Estimation Of Noise Covariance Matrices Research Articles

Joint Learning of Full-Structure Noise in Hierarchical Bayesian Regression Models.

We consider the reconstruction of brain activity from electroencephalography (EEG). This inverse problem can be formulated as a linear regression with independent Gaussian scale mixture priors for both the source and noise components. Crucial factors influencing the accuracy of the source estimation are not only the noise level but also its correlation structure, but existing approaches have not addressed the estimation of noise covariance matrices with full structure. To address this shortcoming, we develop hierarchical Bayesian (type-II maximum likelihood) models for observations with latent variables for source and noise, which are estimated jointly from data. As an extension to classical sparse Bayesian learning (SBL), where across-sensor observations are assumed to be independent and identically distributed, we consider Gaussian noise with full covariance structure. Using the majorization-maximization framework and Riemannian geometry, we derive an efficient algorithm for updating the noise covariance along the manifold of positive definite matrices. We demonstrate that our algorithm has guaranteed and fast convergence and validate it in simulations and with real MEG data. Our results demonstrate that the novel framework significantly improves upon state-of-the-art techniques in the real-world scenario where the noise is indeed non-diagonal and full-structured. Our method has applications in many domains beyond biomagnetic inverse problems.

An enhanced adaptive Kalman filtering for linear systems with inaccurate noise statistics

AbstractThis paper investigates the simultaneous state and noise covariance estimation for linear systems with inaccurate noise statistics. An enhanced adaptive Kalman filtering (EAKF) based on dynamic recursive nominal covariance estimation (DNRCE) and modified variational Bayesian (VB) inference is presented. The EAKF realizes the concurrently estimation of state and noise covariance matrices by introducing a nominal parameter in the traditional recursive covariance estimation and designing a new adaptive forgotten factor for the dynamic model of the estimated information propagation. The simulation of a target tracking example shows that, compared with the existing filters, the proposed filter has good adaptive performance for inaccurate and time‐varying noise covariance matrices.

Multi-Fading Factor and Updated Monitoring Strategy Adaptive Kalman Filter-Based Variational Bayesian.

Aiming at the problem that the performance of adaptive Kalman filter estimation will be affected when the statistical characteristics of the process and measurement of the noise matrices are inaccurate and time-varying in the linear Gaussian state-space model, an algorithm of multi-fading factor and an updated monitoring strategy adaptive Kalman filter-based variational Bayesian is proposed. Inverse Wishart distribution is selected as the measurement noise model and the system state vector and measurement noise covariance matrix are estimated with the variational Bayesian method. The process noise covariance matrix is estimated by the maximum a posteriori principle, and the updated monitoring strategy with adjustment factors is used to maintain the positive semi-definite of the updated matrix. The above optimal estimation results are introduced as time-varying parameters into the multiple fading factors to improve the estimation accuracy of the one-step state predicted covariance matrix. The application of the proposed algorithm in target tracking is simulated. The results show that compared with the current filters, the proposed filtering algorithm has better accuracy and convergence performance, and realizes the simultaneous estimation of inaccurate time-varying process and measurement noise covariance matrices.

Toward security monitoring of industrial Cyber-Physical systems via hierarchically distributed intrusion detection

Industrial Cyber-physical systems (ICPSs), integrating communication, computation and control of industrial processes are referred to as a core technology to approach the Industry 4.0. Ensuring the ICPS security is of paramount importance in smart manufacturing. Considering the characteristics of large-scale, geographically-dispersed and multi-dimensional heterogeneous, federated and life-critical natures of ICPSs, this paper investigates a hierarchically distributed intrusion detection scheme that seeks to achieve the all-round safety protection of ICPSs according to the system structure and attacking types of each ICPS layer. For physical system-relevant perceptual executive layer, potential and covert attacks are detected by the clustered sensory system state residual anomaly monitoring based on a process noise and measurement noise-adaptive Kalman filter (PNMN-AKF). PNMN-AKF can perform a joint recursive estimation of dynamic system states, time-varying process and measurement noise covariance matrices by the variational Bayes approximation framework. In cyberspace, potential cyber-attacks are detected by the anomaly monitoring of the statistical distribution of the network transmission characteristics of data transmission layer by introducing a forgetting factor-induced recursive Gaussian mixture model (FF-RGMM). In the application control layer, a regularized sparse deep belief network model is introduced to characterize the misuse behavior for detecting potential attacks. Extensive validation and comparative experiments have been conducted on a numerical simulation system and a comprehensive ICPS simulation platform by using OPNET and a commonly-used benchmark simplified Tennessee Eastman process (STEP) based on Matlab/Simulink. Experimental results demonstrate that the proposed hierarchically distributed intrusion detection method can efficiently recognize potential and covert cyber-attacks in each ICPSs link with low false alarm rate and missing detection rate, which lays a foundation for the overall security monitoring of ICPSs.

Collaborative Sequential State Estimation Under Unknown Heterogeneous Noise Covariance Matrices

We study the problem of distributed sequential estimation of common states and measurement noise covariance matrices of hidden Markov models by networks of collaborating nodes. We adopt a realistic assumption that the true covariance matrices are possibly different (heterogeneous) across the network. This setting is frequent in many distributed real-world systems where the sensors (e.g., radars) are deployed in a spatially anisotropic environment, or where the networks may consist of sensors with different measuring principles (e.g., using different wavelengths). Our solution is rooted in the variational Bayesian paradigm. In order to improve the estimation performance, the measurements and the posterior estimates are communicated among adjacent neighbors within one network hop distance using the information diffusion strategy. The resulting adaptive algorithm selects neighbors with compatible information to prevent degradation of estimates.

Maximum likelihood estimation of noise covariance matrices for state estimation of autonomous hybrid systems

Abstract A critical aspect of developing Bayesian state estimators for hybrid systems, that involve a combination of continuous and discrete state variables, is to have a reasonably accurate characterization of the stochastic disturbances affecting their dynamics. Recently, Bavdekar et al. (2011) have proposed a maximum likelihood (ML) based framework for estimation of the noise covariance matrices from operating input–output data when an EKF is used for state estimation. In this work, the ML framework is extended to estimation of the noise covariance matrices associated with autonomous hybrid systems, and, to a wider class of recursive Bayesian filters. Under the assumption that the innovations generated by an estimator form a white noise sequence, the proposed ML framework computes the noise covariance matrices such that they maximize the log-likelihood function of the estimator innovations. The efficacy of the proposed scheme is demonstrated through the simulation and experimental studies on the benchmark three-tank system.

Estimation of Noise Covariance Matrices for Linear Systems with Nonlinear Measurements

The paper deals with the estimation of the noise covariance matrices of the state space models. In particular, the stress is laid on the autocovariance least-squares method providing unbiased estimates of the noise covariance matrices of linear time-invariant systems. The method is extended to be applicable also for the stable linear systems with time-varying linear measurement equation and for the systems with nonlinear measurement equation. The theoretical results are discussed and illustrated in numerical examples.

Estimation of noise covariance matrices for periodic systems

Estimation of the noise covariance matrices for linear time-variant stochastic dynamic periodic systems is treated. The novel offline method for estimation of the covariance matrices of the state and measurement noises is designed. The method is based on analysis of second-order statistics of the state estimate produced by the linear multi-step predictor. The estimates of the noise covariance matrices are unbiased and converge to the true values with increasing number of data. The theoretical results are illustrated in numerical examples. Copyright © 2011 John Wiley & Sons, Ltd.

Methods for Estimating State and Measurement Noise Covariance Matrices: Aspects and Comparison

The paper deals with estimation of noise covariance matrices in state and measurement equations of linear discrete-time stochastic dynamic systems. In the last decade several novel methods for noise covariance matrices estimation, which are based on state estimation techniques, have been proposed. Unfortunately, the novel methods have been compared mainly with classical methods proposed in the seventies only. The aim of the paper is to analyse identifiability of state noise parameters by means of the Bayesian approach and to summarise and compare the novel methods from both theoretical and numerical point of view.

Estimation of State and Measurement Noise Covariance Matrices by Multi-Step Prediction

Estimation of noise covariance matrices for linear or nonlinear stochastic dynamic systems is treated. The novel off-line technique for estimation of the covariance matrices of the state and measurement noises is designed. The technique is based on the multi-step prediction error and on knowledge of the system initial condition and it takes an advantage of the well-known standard relations from the area of state estimation techniques and least square method. The theoretical results are illustrated in numerical examples.

Neural Network Aided Adaptive Extended Kalman Filtering Approach for DGPS Positioning

The extended Kalman filter, when employed in the GPS receiver as the navigation state estimator, provides optimal solutions if the noise statistics for the measurement and system are completely known. In practice, the noise varies with time, which results in performance degradation. The covariance matching method is a conventional adaptive approach for estimation of noise covariance matrices. The technique attempts to make the actual filter residuals consistent with their theoretical covariance. However, this innovation-based adaptive estimation shows very noisy results if the window size is small. To resolve the problem, a multilayered neural network is trained to identify the measurement noise covariance matrix, in which the back-propagation algorithm is employed to iteratively adjust the link weights using the steepest descent technique. Numerical simulations show that based on the proposed approach the adaptation performance is substantially enhanced and the positioning accuracy is substantially improved.

Adaptive estimation of noise covariance matrices in real-time preprocessing of geophysical data

Modern data acquisition systems record large volumes of data which are often not suitable for direct computer processing-a first stage of preprocessing (or data "editing") is usually needed. In earlier work G. Noriega et al. (1992) the authors have developed an algorithm for multichannel data preprocessing, based on Kalman filtering and suitable for real-time geophysical data collection applications. The present work presents results of further investigations in the area of adaptive methods for estimation of noise covariance matrices Q and R, within the time-variant, fixed-lag Kalman filtering framework of the original problem. An algorithm is developed whereby asymptotically normal, unbiased, and consistent estimates are produced based on the correlation-innovations method introduced by Mehra (1970). This provides for direct estimation of R, and leads to a set of (n.m/sup 2/) equations, not linearly independent, from which an appropriate subset must be selected to achieve estimation of up to (n.m) unknowns in Q. For the model considered (n=m.M, with M=4 the single-channel system order, and n and m the state and measurement vector dimensions respectively), an explicit algorithm has been developed far estimation of the (m.m) unknowns in Q, based on a least squares fit of a subset of the equations available. A new approach is also introduced to ensure positive-definiteness of the covariance matrices. Since the time variant nature of the model prevents direct application of the adaptive algorithm, a parallel implementation is proposed. A first processor implements the time variant Kalman filter, using estimates of Q and R updated every N/sub s/ samples. A second processor computes these estimates, operating on the output of a steady state Kalman filter based on a simplified model, in which data editing features, responsible for rendering the model time variant, have been removed. A spike/step removal filter, and a Riccati equation solver are also implemented by this second processor. Computational requirements are analyzed and compared against those of other approaches. Simulations demonstrate the performance of the method proposed, and show it to be superior to other alternatives. An example showing application to real geophysical data is also presented.

Estimation - Correlation, Modeling and Identification in Adaptive Array Processors

This paper studies the system modeling and identification issues that arise in implementation of maximum likelihood detectors and adaptive array processors of signals that have propagated over randomly time-varying propagation and scattering channels. A multidimensional estimator correlator processor is used to combine the ideas of system modeling and identification with detection theory and adaptive array processing. Part of the estimator structure can be obtained from received data by the usual adaptive array processing techniques (estimation and inversion of noise covariance matrices), but the channel output estimator requires prior knowledge of the signal part of the channel output covariance matrix. This requires modeling and identification of deterministic and stochastic propagation and scattering operators. A number of convenient matrix representations for these operators have been developed. Identification of stochastic operators is based on the vector Karhunen-Loeve expansion. It is shown that in the special case of wide-sense stationary stochastic processes and periodic signal vector, orthonormal basis and eigenvalues for channel modeling and identification can be determined by discrete Fourier transform and singular value decomposition.

Estimation of noise covariance matrices for a linear time-varying stochastic process

An algorithm is given to estimate the noise covariance matrices for a linear, discrete, time-varying stochastic system. If these matrices are linear with respect to a set of aparameters, it is found that the correlation products of the innovations sequence is also linear in these parameters. The fact is used to derive a least-squares algorithm, which takes a particularly simple form in the stationary case. Two examples are given.

Estimation of Noise Covariance Matrices for a Linear Time-Varying Stochastic Process

In th is paper, an algorithm is given to estimate the noise covariance matrices for a linear, discrete, time-varying stochastic system. If these matrices are linear in a set of parameters, it is found that the lagged products of the innovations sequence is also linear in these parameters. This fact is used to derive a least-squares estimation algorithm, plus a much simplified algorithm for the stationary case. Examples are given, including that of a state formulation of a difference equation identification problem.