Variational Bayesian Approximation Method Research Articles

Ensemble variational Bayesian approximation for the inversion and uncertainty quantification of Darcy flows in heterogeneous porous media with random parameters

The inversion and uncertainty quantification of physical parameters are important for many science and engineering problems. For nonlinear parameter inversion problems of flows in hydrocarbon reservoirs, estimating the true posterior probability distribution function for the random parameter is difficult when the dimension of the parameter is relatively high. Inversion methods such as gradient-based or heuristic optimization methods searching for the maximum a posterior are generally point-estimate, while uncertainty quantification using MCMC is computationally expensive. In the current study, a new ensemble variational Bayesian approximation method is developed to estimate the posterior probability density using an ensemble of realizations regarded as a trial sample distribution. The objective is to optimize the trial sample distribution such that the evidence lower bound is maximized indicating that the distance between the trial and true posterior distribution is minimized. A subspace is built using principle component analysis based on finite samples to obtain a diagonal positive-definite covariance matrix, and ensemble-based data assimilation is used to optimize the trial distribution efficiently. Heterogeneous 2D and 3D test cases of transient single- and two-phase Darcy flows are presented for validation.

Trajectory PHD Filter for Adaptive Measurement Noise Covariance Based on Variational Bayesian Approximation

In order to solve the problem that the measurement noise covariance may be unknown or change with time in actual multi-target tracking, this paper brings the variational Bayesian approximation method into the trajectory probability hypothesis density (TPHD) filter and proposes a variational Bayesian TPHD (VB-TPHD) filter to obtain measurement noise covariance adaptively. By modeling the unknown covariance as the random matrix that obeys the inverse gamma distribution, VB-TPHD filter minimizes the Kullback–Leibler divergence (KLD) and estimates the sequence of multi-trajectory states with noise covariance matrices simultaneously. We propose the Gaussian mixture VB-TPHD (AGM-VB-TPHD) filter under adaptive newborn intensity for linear Gaussian models and also give the extended Kalman (AEK-VB-TPHD) filter and unscented Kalman (AUK-VB-TPHD) filter in nonlinear Gaussian models. The simulation results prove the effectiveness of the idea that the VB-TPHD filter can form robust and stable trajectory filtering while learning adaptive measurement noise statistics. Compared with the tag-VB-PHD filter, the estimated error of the VB-TPHD filter is greatly reduced, and the estimation of the trajectory number is more accurate.

Variational Bayesian approximation based robust adaptive single beacon navigation method

For the practical application of single beacon navigation system, effective sound velocity (ESV) setting error, measurement outliers and unknown noise statistical parameters are the major sources of positioning error. Although the unknown ESV of single beacon navigation system has been well studied, the other two sources are less concerned. The performance of existing state-of-art navigation methods will deteriorate when there are measurement outliers or noise statistical parameters setting error. This paper aims to fill the gap. We model the process and measurement noise of each state and measurement component based on their own stochastic properties, so that, the influence of measurement outliers can be incorporated into stochastic model. The unknown noise statistical parameters are regarded as random variables and the variational Bayesian approximation method is used to estimate them along with the navigation state. The corresponding robust adaptive single beacon navigation method is proposed. Moreover, the simulation and field data demonstrate that the proposed method is superior to the existing state-of-art navigation methods in the presence of measurement outliers and unknown noise statistical parameters.

A Novel Robust Student’s t-Based Cubature Information Filter with Heavy-Tailed Noises

In this paper, a novel robust Student’s t-based cubature information filter is proposed for a nonlinear multisensor system with heavy-tailed process and measurement noises. At first, the predictive probability density function (PDF) and the likelihood PDF are approximated as two different Student’s t distributions. To avoid the process uncertainty induced by the heavy-tailed process noise, the scale matrix of the predictive PDF is modeled as an inverse Wishart distribution and estimated dynamically. Then, the predictive PDF and the likelihood PDF are transformed into a hierarchical Gaussian form to obtain the approximate solution of posterior PDF. Based on the variational Bayesian approximation method, the posterior PDF is approximated iteratively by minimizing the Kullback-Leibler divergence function. Based on the posterior PDF of the auxiliary parameters, the predicted covariance and measurement noise covariance are modified. And then the information matrix and information state are updated by summing the local information contributions, which are computed based on the modified covariance. Finally, the state, scale matrix, and posterior densities are estimated after fixed point iterations. And the simulation results for a target tracking example demonstrate the superiority of the proposed filter.

Efficient unsupervised variational Bayesian image reconstruction using a sparse gradient prior

In this paper, we present an efficient unsupervised Bayesian approach and a prior distribution adapted to piecewise regular images. This approach is based on a hierarchical prior distribution promoting sparsity on image gradients. It is fully automatic since hyperparameters are estimated jointly with the image of interest. The estimation of all unknowns is performed efficiently thanks to a fast variational Bayesian approximation method. We highlight the good performance of the proposed approach through comparisons with state of the art approaches on an application to a diffraction tomographic problem.

An Improved Variational Inference Approach to Iterative OFDM Receiver Design for Superimposed Training-Based AF Relay Networks

Using variational Bayesian approximation (VBA), we propose a general framework for joint channel estimation, demodulation, and decoding in an orthogonal frequency-division multiplexing (OFDM) amplify-and-forward (AF) relay network. AF relaying is adopted due to its simplicity and ease of implementation, and both cases of fixed- and variable-gain AF protocols are addressed. Unlike the classical approach where a number of subcarriers are solely dedicated to pilots, partial data-dependent superimposed training is considered here, which preserves the spectral efficiency. Assuming a bit-interleaved coded modulation OFDM system at the source, a modified formulation of VBA is proposed, which is, in fact, a Bayesian framework of turbo data detection of OFDM under channel estimation errors. More precisely, by an appropriate exploitation of the inherent “soft” information on channel and data available from the VBA formalism, we derive a modified iterative receiver, which reduces the impact of channel uncertainty by averaging the “soft data decision” over the posterior distribution of the unknown channel at each decoding iteration. The proposed modified VBA approach is contrasted to the conventional VBA and to the mismatched VBA approach, where, in the latter, the unknown channel is just replaced by its imperfect estimate. We show that conventional and mismatched VBA are suboptimal and can be viewed as a particular case of the proposed VBA method. In addition, we show that the VBA approach makes a nice connection between the classical techniques such as pilot-only channel estimation and the expectation–maximization algorithm, which can be viewed as a special case of VBA. By comparison with state-of-the-art VBA-based estimators through numerical analysis, we highlight the superiority of our modified VBA method and demonstrate a notable performance improvement in terms of the bit error rate.

Efficient Variational Bayesian Method for Joint Channel Estimation and Signal Detection in OFDM-Based AF Relay Networks

We propose improved joint channel estimation and data detection in orthogonal frequency division multiplexing (OFDM) amplify-and-forward (AF) relay network in the case of imperfect partial channel knowledge at the destination. To improve the detection accuracy, we adopt the variational Bayesian approximation (VBA) which is an iterative Bayesian method for channel estimation. We derive a modified iterative estimator that takes into account the channel estimation errors at the previous iteration, to reduce the impact of channel uncertainty at the next iteration, and hence, to improve the overall detection accuracy. By comparison with state of the art VBA-based estimators through numerical analysis, we highlight the superiority of our new VBA method and demonstrate a notable improvement in terms of bit error rate and mean square error between the true and the estimated channel.

Efficient variational Bayesian approximation method based on subspace optimization.

Variational Bayesian approximations have been widely used in fully Bayesian inference for approximating an intractable posterior distribution by a separable one. Nevertheless, the classical variational Bayesian approximation (VBA) method suffers from slow convergence to the approximate solution when tackling large dimensional problems. To address this problem, we propose in this paper a more efficient VBA method. Actually, variational Bayesian issue can be seen as a functional optimization problem. The proposed method is based on the adaptation of subspace optimization methods in Hilbert spaces to the involved function space, in order to solve this optimization problem in an iterative way. The aim is to determine an optimal direction at each iteration in order to get a more efficient method. We highlight the efficiency of our new VBA method and demonstrate its application to image processing by considering an ill-posed linear inverse problem using a total variation prior. Comparisons with state of the art variational Bayesian methods through a numerical example show a notable improvement in computation time.

Adaptive filtering for jump Markov systems with unknown noise covariance

The paper proposed an adaptive filter for jump Markov systems with unknown measurement noise covariance. The filter is derived by treating covariance as a random matrix and an inverse-Wishart distribution is adopted as the conjugate prior. The variational Bayesian approximation method is employed to derive mode-conditioned estimates and mode-likelihood functions in the framework of interacting multiple model. A numerical example is provided to illustrate the performance of the proposed filter.

Variational Bayesian Approximation methods for inverse problems

Variational Bayesian Approximation (VBA) methods are recent tools for effective Bayesian computations. In this paper, these tools are used for inverse problems where the prior models include hidden variables and where where the estimation of the hyper parameters has also to be addressed. In particular two specific prior models (Student-t and mixture of Gaussian models) are considered and details of the algorithms are given.

Bayesian Robust PCA of Incomplete Data

We present a probabilistic model for robust factor analysis and principal component analysis in which the observation noise is modeled by Student-t distributions in order to reduce the negative effect of outliers. The Student-t distributions are modeled independently for each data dimensions, which is different from previous works using multivariate Student-t distributions. We compare methods using the proposed noise distribution, the multivariate Student-t and the Laplace distribution. Intractability of evaluating the posterior probability density is solved by using variational Bayesian approximation methods. We demonstrate that the assumed noise model can yield accurate reconstructions because corrupted elements of a bad quality sample can be reconstructed using the other elements of the same data vector. Experiments on an artificial dataset and a weather dataset show that the dimensional independency and the flexibility of the proposed Student-t noise model can make it superior in some applications.