Variational Bayesian Approach Research Articles

Robust Poisson Multi-Bernoulli Mixture Filter With Inaccurate Process and Measurement Noise Covariances

This paper proposes a robust Poisson multi-Bernoulli mixture (PMBM) filter with inaccurate process and measurement noise covariances. A derivation of the robust PMBM filter is provided for jointly estimating the kinematic state, the predicted state covariance, and the measurement noise covariance. By modeling the augmented state as a Gaussian inverse Wishart inverse Wishart (GIWIW) distribution, a computationally feasible implementation of the robust PMBM (GIWIW-PMBM) filter is given for linear Gaussian systems. To guarantee the conjugacy of the GIWIW distribution, the variational Bayesian (VB) approach is employed to approximate the posterior density. Finally, simulation results show that the GIWIW-PMBM filter has the best overall performance compared to existing state-of-the-art filters regarding computational cost and filtering performance.

A Novel Nonlinear Algorithm for Non-Gaussian Noises and Measurement Information Loss in Underwater Navigation

The ocean environment is complex and changeable because of all kinds of noise interferences, such as salt cliffs, ships around and other electromagnetic interferences, so the measurement information is prone to be lost. It is difficult to describe the complex noise and the acquisition probability of measurement information. In this paper, a continuous discrete variational Bayesian filter (CD VBF) is proposed to solve the problems of the heavy tailed noises and measurement random loss for state estimation. The variational Bayesian (VB) approach can effectively estimate the state vector, scale matrices, degree of freedom (DOF) parameters, Bernoulli random variables and the acquisition probability of measurement information. The performances of the proposed algorithm and traditional algorithms are tested in simulations and underwater experiments. The simulation results illustrate that the CD VBF has better localization accuracy and robustness. The experimental results demonstrate that the localization accuracy is improved by CD VBF and an optimal solution of iteration number is acquired.

A Novel Dynamic Baysian Canonical Correlation Analysis Method for Fault Detection

In the field of Multivariate Statistical Process Monitoring (MSPM), process dynamics has always been the focus. Besides, considering the uncertainty in chemical processes, latent variable models are extended to the probabilistic framework, in which maximum likelihood estimation with expectation maximization (EM) algorithm is adopted for parameter learning. However, the modelling performance is restricted owing to the reason that these models either neglect the static characteristics reflecting process structure or suffer from over fitting and local optimum. To tackle these issues, a dynamic Baysian canonical correlation analysis (DBCCA) model is developed through combining the consideration of process dynamics with the variational CCA and utilized for fault detection. More specifically, both static structural characteristics and process dynamics can be simultaneously captured in DBCCA model. In essence, the variational Bayesian approach renders effects of regularization, alleviating the dilemma in traditional maximum likelihood estimation methods by nature. The effectiveness of proposed method is testified on the well-known Tennessee Eastman (TE) benchmark, where improvements are attained.

Robust Localization Using IMM Filter Based on Skew Gaussian-Gamma Mixture Distribution in Mixed LOS/NLOS Condition

This article proposes a new skewed outlier-robust localization algorithm that is based on time-difference of arrival (TDOA) measurements at an airport. A new outlier-robust filtering framework is derived based on the skew Gaussian-gamma mixture (SGGM) distribution, where the state, a mixing parameter, a shape parameter, a scale matrix, and the degrees of freedom (DOFs) are inferred simultaneously using variational Bayesian (VB) approach. An interacting multiple-model (IMM) filter with different kinematic system models is implemented to handle the multimodal dynamics of the vehicle, yielding the IMM-SGGM algorithm. In particular, a new measurement likelihood based on the SGGM distribution is derived utilizing VB inference for the combination procedure in the proposed IMM-SGGM algorithm. Car-mounted experiments using TDOA measurements at an airport were conducted to verify the effectiveness of the proposed algorithm. The performance of the proposed IMM-SGGM algorithm is evaluated through comparisons with the state-of-the-art approaches. The experimental results demonstrate that the proposed IMM-SGGM algorithm has better localization accuracy and robustness to skewed outlier measurements than the state-of-the-art approaches.

A Novel Adaptive Kalman Filter With Unknown Probability of Measurement Loss

A novel variational Bayesian (VB)-based adaptive Kalman filter (AKF) is proposed to solve the filtering problem of a linear system with unknown probability of measurement loss. The sum of two likelihood functions is transformed into an exponential multiplication form, and the state vector, the Bernoulli random variable and the probability of measurement loss are jointly inferred based on the VB approach. Simulation results demonstrate the superiority of the proposed AKF as compared with the existing filtering algorithms with unknown probability of measurement loss.

Estimating the rate constant from biosensor data via an adaptive variational Bayesian approach

The means to obtain the rate constants of a chemical reaction is a fundamental open problem in both science and the industry. Traditional techniques for finding rate constants require either chemical modifications of the reactants or indirect measurements. The rate constant map method is a modern technique to study binding equilibrium and kinetics in chemical reactions. Finding a rate constant map from biosensor data is an ill-posed inverse problem that is usually solved by regularization. In this work, rather than finding a deterministic regularized rate constant map that does not provide uncertainty quantification of the solution, we develop an adaptive variational Bayesian approach to estimate the distribution of the rate constant map, from which some intrinsic properties of a chemical reaction can be explored, including information about rate constants. Our new approach is more realistic than the existing approaches used for biosensors and allows us to estimate the dynamics of the interactions, which are usually hidden in a deterministic approximate solution. We verify the performance of the new proposed method by numerical simulations, and compare it with the Markov chain Monte Carlo algorithm. The results illustrate that the variational method can reliably capture the posterior distribution in a computationally efficient way. Finally, the developed method is also tested on the real biosensor data (parathyroid hormone), where we provide two novel analysis tools—the thresholding contour map and the high order moment map—to estimate the number of interactions as well as their rate constants.

An Improved Kalman Filter With Adaptive Estimate of Latency Probability

In this brief, an improved Kalman filter is proposed for a linear system with one-step randomly delayed measurement and unknown latency probability. The measurement likelihood function which is a weighted sum of two Gaussian distributions is transformed into an exponential multiplication form via importing a discrete Bernoulli random variable. Then, an hierarchical Gaussian form of the state-space model is established. Finally, an improved Kalman filter is deduced to estimate jointly the augmented state vector and the unknown parameters employing the variational Bayesian and state augmentation approaches. Simulation study indicates that the improved method has superior performance in estimation accuracy than the existing methods on the basis of accurate estimation of the unknown and time-varying latency probability.

Robust Interacting Multiple Model Filter Based on Student's t-Distribution for Heavy-Tailed Measurement Noises.

In maneuvering target tracking applications, the performance of the traditional interacting multiple model (IMM) filter deteriorates seriously under heavy-tailed measurement noises which are induced by outliers. A robust IMM filter utilizing Student’s t-distribution is proposed to handle the heavy-tailed measurement noises in this paper. The measurement noises are treated as Student’s t-distribution, whose degrees of freedom (dof) and scale matrix are assumed to be governed by gamma and inverse Wishart distributions, respectively. The mixing distributions of the target state, dof, and scale matrix are achieved through the interacting strategy of IMM filter. These mixing distributions are used for the initialization of time prediction. The posterior distributions of the target state, dof, and scale matrix conditioned on each mode are obtained by employing variational Bayesian approach. Then, the target state, dof, and scale matrix parameters are jointly estimated. A variational method is also given to estimate the mode probability. The unscented transform is utilized to solve the nonlinear estimation problem. Simulation results show that the proposed filter improves the estimation accuracy of target state and mode probability over existing filters under heavy-tailed measurement noises.

Parametric Bayesian estimation of point-like pollution sources of groundwater layers

This paper considers the problem of estimating point-like pollution sources of groundwater layers. To cope with the ill-posed character of this problem, a parametric Bayesian framework has been recently established. In this framework, where the priors for the source parameters are either uniform or Gaussian and the observation noise is homogeneous, a stochastic Markov Chain Monte Carlo (MCMC) algorithm has been proposed to compute the posterior distribution of both source parameters and noise variance. Here, we consider a more general model with truncated-Gaussian priors for pollution quantity and spreading time parameter, which gathers advantages of uniform and Gaussian choices, and an inhomogeneous noise, which accounts for the spatial diversity among sensors. For this model, we extend the existing stochastic algorithm, then propose a concurrent deterministic algorithm based on the variational Bayesian (VB) approach. This approach designs an approximation of the posterior law based on a separable from. The proposed MCMC and VB algorithms target the exact posterior and the approximated posterior, respectively. It is further shown that the former is more accurate, while the latter is computationally more efficient. Results of numerical experiments conducted using an experimental platform to compare the performances of the proposed schemes are presented.

Extended-Object or Group-Target Tracking Using Random Matrix With Nonlinear Measurements

For extended-object/group-target tracking (EOT/ GTT), the random-matrix approach assumes that measurements are linear in the state and noise with a covariance being a random matrix to represent the object extension or target group. In practice, however, most measurements are nonlinear in the state and noise. This paper proposes a random-matrix approach for EOT/GTT using nonlinear measurements. First, a matched linearization (ML) is proposed to linearize nonlinear measurements. The linearized form has two parts. The first is linear in the state, and it is optimized in the sense of minimum mean square error (MMSE). The second part is linear in the extension-related noise with a preserved second moment, which is important since extension information is contained in the covariance of this noise. The linearized measurements can be incorporated into existing random-matrix algorithms after a simple conversion under certain conditions. Second, a variational Bayesian (VB) scheme is proposed for EOT/GTT using the linearized measurements. This approach can be generally applied no matter whether the linearized measurements are converted or not. The effectiveness of the proposed ML and VB approach is demonstrated by simulation results compared with existing random-matrix algorithms.

Deterministic and Bayesian Sparse Signal Processing Algorithms for Coherent Multipath Directions-of-Arrival (DOAs) Estimation

In this paper, we propose novel deterministic and Bayesian methods to identify the directions-of-arrival (DOAs) of coherent multipath signals assuming that a very few number of multipath signals are present and multiple snapshots are available. For both types of methods, we exploit both sparsity and the underlying structure of coherent signal propagation. For the deterministic case, we formulate the problem of estimating the DOAs of coherent multipath signals as a biconvex optimization problem and then solve it by an alternating convex search approach. This is in contrast to the widely used 11 sparse signal recovery convex optimization problem, which only exploits sparsity of the signal under consideration. For the Bayesian case, we propose two novel algorithms based on the mean field variational Bayesian expectation maximization approach. The first algorithm assumes that the true scenario DOAs of the multipath signals are exactly aligned with the angular grids. We next extend the on-grid model to deal with the off-grid problem, which is the second proposed Bayesian algorithm of this paper. These two Bayesian algorithms are in contrast to the widely used sparse Bayesian learning relevance vector machine algorithm, which only exploits sparsity in spatial domain. A simulation study is carried out in terms of root-mean-squared error in the DOA estimates to compare the performances of different algorithms. Finally, we demonstrate the application of the proposed off-grid Bayesian algorithm by analyzing data from the shallow water HF97 ocean acoustic experiment.

Robust Kalman Filters Based on Gaussian Scale Mixture Distributions With Application to Target Tracking

In this paper, a new robust Kalman filtering framework for a linear system with non-Gaussian heavy-tailed and/or skewed state and measurement noises is proposed through modeling one-step prediction and likelihood probability density functions as Gaussian scale mixture (GSM) distributions. The state vector, mixing parameters, scale matrices, and shape parameters are simultaneously inferred utilizing standard variational Bayesian approach. As the implementations of the proposed method, several solutions corresponding to some special GSM distributions are derived. The proposed robust Kalman filters are tested in a manoeuvring target tracking example. Simulation results show that the proposed robust Kalman filters have a better estimation accuracy and smaller biases compared to the existing state-of-the-art Kalman filters.

Joint Target Detection and Tracking in Multipath Environment: A Variational Bayesian Approach

We consider multitarget detection and tracking problem for a class of multipath detection system where one target may generate multiple measurements via multiple propagation paths, and the association relationship among targets, measurements, and propagation paths is unknown. In order to effectively utilize multipath measurements from one target to improve detection and tracking performance, a tracker has to handle high-dimensional estimation of latent variables including target existence state, target kinematic state, and multipath data association. Based on variational Bayesian inference, we propose a novel joint detection and tracking algorithm that incorporates multipath data association, target detection, and target state estimation in a unified Bayesian framework. The posterior probabilities of these latent variables are derived in a closed-form iterative manner, which is effective for reducing the performance deterioration caused by the coupling between estimation errors and identification errors. Loopy belief propagation is exploited to approximately calculate the probability of multipath data association, saving the computational cost significantly. Simulation results of over-the-horizon radar multitarget tracking show that the proposed algorithm outperforms multihypothesis multipath track fusion and multidetection (hypothesis-oriented) multiple hypothesis tracker, especially under low signal-to-noise ratio circumstance.

Identification of jump Markov autoregressive exogenous systems with missing measurements

In this paper, the identification problem of jump Markov autoregressive exogenous (JMARX) systems, in which the measurements are missing completely at random (MCAR), is considered. With the employment of the variational Bayesian (VB) inference, the distributions of the local model parameters and the precision of the process noise are determined. Meanwhile, the transition probabilities of the Markov chain are estimated. Furthermore, the reconstruction of the missing measurements is discussed. A numerical example and a simulated continuous fermentation reactor (CFR) example are implemented to verify the validity of the proposed method. It shows that, the influence of the missing measurements can be effectively eliminated using the variational Bayesian inference approach.

A Novel Progressive Gaussian Approximate Filter for Tightly Coupled GNSS/INS Integration

In this article, we focus on addressing the nonlinear filtering problem with large prior uncertainty but high measurement accuracy, which may be encountered in the application of tightly coupled global navigation satellite system (GNSS)/inertial navigation system (INS) integration. Although the existing methods, such as progressive Gaussian approximate filter (PGAF), can address this problem, it has poor estimation accuracy. To improve the estimation accuracy of PGAF, the step sizes as well as the measurement noise covariance matrix (MNCM) are jointly estimated based on the variational Bayesian approach, from which a novel PGAF with variable step size is developed. Tightly coupled GNSS/INS integration simulations illustrate that the proposed filter outperforms the existing methods both in estimation accuracy and rate of convergence.

Prediction of Multidimensional Spatial Variation Data via Bayesian Tensor Completion

This paper presents a multidimensional computational method to predict the spatial variation data inside and across multiple dies of a wafer. This technique is based on tensor computation. A tensor is a high-dimensional generalization of a matrix or a vector. By exploiting the hidden low-rank property of a high-dimensional data array, the large amount of unknown variation testing data may be predicted from a few random measurement samples. The tensor rank, which decides the complexity of a tensor representation, is decided by an available variational Bayesian approach. Our approach is validated by a practical chip testing data set, and it can be easily generalized to characterize the process variations of multiple wafers. Our approach is more efficient than the previous virtual probe techniques in terms of memory and computational cost when handling high-dimensional chip testing data.

A Variational Bayesian Approach for Identification of Time-Delay Markov Jump Autoregressive Exogenous Systems

This paper is concerned with the identification problem of Markov jump autoregressive exogenous systems with unknown slow-varying time-delay. With the implementation of variational Bayesian inference approach, the distribution characteristics of local model parameters are obtained. Meanwhile, the transition probability matrix and the unknown time-delay are determined. Furthermore, a numerical example and a simulated continuous fermentation reactor example are performed to verify the effectiveness of the proposed algorithm.

GEMINI: a variational Bayesian approach to identify genetic interactions from combinatorial CRISPR screens

Systems for CRISPR-based combinatorial perturbation of two or more genes are emerging as powerful tools for uncovering genetic interactions. However, systematic identification of these relationships is complicated by sample, reagent, and biological variability. We develop a variational Bayes approach (GEMINI) that jointly analyzes all samples and reagents to identify genetic interactions in pairwise knockout screens. The improved accuracy and scalability of GEMINI enables the systematic analysis of combinatorial CRISPR knockout screens, regardless of design and dimension. GEMINI is available as an open source R package on GitHub at https://github.com/sellerslab/gemini.

A Novel Robust Gaussian–Student's t Mixture Distribution Based Kalman Filter

In this paper, a novel Gaussian-Student's t mixture (GSTM) distribution is proposed to model non-stationary heavy-tailed noises. The proposed GSTM distribution can be formulated as a hierarchical Gaussian form by introducing a Bernoulli random variable, based on which a new hierarchical linear Gaussian state-space model is constructed. A novel robust GSTM distribution based Kalman filter is proposed based on the constructed hierarchical linear Gaussian state-space model using the variational Bayesian approach. The Kalman filter and robust Student's t based Kalman filter (RSTKF) with fixed distribution parameters are two existing special cases of the proposed filter. The novel GSTM distributed Kalman filter has the important advantage over the RSTKF that the adaptation of the mixing parameter is much more straightforward than learning the degrees of freedom parameter. Simulation results illustrate that the proposed filter has better estimation accuracy than those of the Kalman filter and RSTKF for a linear state-space model with non-stationary heavy-tailed noises.

Parameter estimation of Markov‐switching Hammerstein systems using the variational Bayesian approach

Parameter estimation of Markov-switching Hammerstein systems is presented in this study. The random switching mode is described by a hidden Markov model. An input non-linear controlled autoregressive Hammerstein model is applied to describe the dynamic process of each local model. By applying the decomposition technique to the non-linear mapping of the input signal, each local model is written as a linear-in-parameter form. The variational Bayesian approach is applied to identify the switched Markov Hammerstein models. Both the discrete-valued switching mode state and the unknown parameters in each local model are estimated. Moreover, instead of the point estimation, posterior distributions of unknown parameters for each local model are obtained. Numerical simulation examples and a hybrid tank experiment are presented to verify the effectiveness of the proposed approach.