Variational Bayesian Technique Research Articles

A nonlinear filter based on GSK for relative navigation using relative orbital elements

Due to the sensitivity of the relative orbital elements model to the measurement noise, the non-stationary heavy-tailed noise(NSHT) induced by the time-varying environment during the relative navigation usually leads to filter divergence. To address this problem, a new nonlinear filter based on Gaussian-Student's-Multivariate K(GSK) mixture distribution is proposed in this paper. A Dirichlet stochastic mixture vector fusing Gaussian, Student's t, and Multivariate K distributions is introduced, thus proposing a GSK mixture distribution modeling measurement likelihood; then the Kullback-Leibler Divergence (KLD) of the true posteriori probability density function(PDF) and the approximate posteriori PDF are minimized by a variational Bayesian(VB) technique to solve for the state and parameter approximate a posteriori estimations, and finally a new nonlinear filter based on the GSK mixture distribution is derived for angles-only relative navigation in time-varying environments. Simulation outcomes indicate that the filter can realize state estimation in non-stationary states effectively with 45.16% higher estimation accuracy than the existing advanced filters.

Irregular extended target tracking with unknown measurement noise covariance

To address the challenge of tracking irregular extended target while adapting to unknown measurement noise, this paper presents a novel tracking algorithm for irregular extended target. Considering the flexibility of the Gaussian process (GP) model for representing shape, in the proposed algorithm, we utilize the GP model to delineate the radial function of the tracked extended target’s contour. Then, we incorporate the GP model into the variational Bayesian technique to formulate the posterior distribution for both the unknown measurement noise and the shape state of the tracked target. Subsequently, the square-root cubature Kalman filter is employed to tackle the nonlinearity filtering in the problem of irregular extended target tracking (ETT), enabling simultaneous estimation of intricate shape details and the unknown measurement noise covariance. Through extensive experiments using both simulated and real-time lidar data, the proposed algorithm demonstrates superior performance compared to the existing ETT algorithms for extended target tracking in both accuracy and robustness.

A Design Methodology for Fault-Tolerant Neuromorphic Computing Using Bayesian Neural Network.

Memristor crossbar arrays are a promising platform for neuromorphic computing. In practical scenarios, the synapse weights represented by the memristors for the underlying system are subject to process variations, in which the programmed weight when read out for inference is no longer deterministic but a stochastic distribution. It is therefore highly desired to learn the weight distribution accounting for process variations, to ensure the same inference performance in memristor crossbar arrays as the design value. In this paper, we introduce a design methodology for fault-tolerant neuromorphic computing using a Bayesian neural network, which combines the variational Bayesian inference technique with a fault-aware variational posterior distribution. The proposed framework based on Bayesian inference incorporates the impacts of memristor deviations into algorithmic training, where the weight distributions of neural networks are optimized to accommodate uncertainties and minimize inference degradation. The experimental results confirm the capability of the proposed methodology to tolerate both process variations and noise, while achieving more robust computing in memristor crossbar arrays.

Multi-Normal-Inverse Wishart mixture distribution based nonlinear filter with applications

In Kalman filter-based spacecraft autonomous celestial navigation, Inertial/Celestial integrated navigation system and asynchronous multi-sensor data fusion, complex environments and sensor alignment errors are likely to lead to inaccurate statistical priors for the noise covariance matrices and the non-zero measurement noise mean vector (MNMV). To address this issue, this paper firstly proposes a Multi-Normal-Inverse Wishart (MNIW) mixture distribution modelling the joint probability density function (PDF) for one-step prediction and measure likelihood, then the MNIW mixture distribution is decomposed into a Gaussian hierarchy, and finally the posterior estimates of the state and variables are obtained using variational Bayesian technique. In this paper, a Multi-Normal-Inverse Wishart mixture distribution-based variational Bayesian extended Kalman filter (VB-EKF) is proposed, in which a first-order Taylor expansion is used to address the non-linear problem. The proposed filter can be used to address non-linear filtering problem with inaccurate noise covariance matrices and measurement bias. Simulations of spacecraft autonomous celestial navigation validate the effectiveness and superiority of the proposed filter.

Modified Strong Tracking Slide Window Variational Adaptive Kalman Filter With Unknown Noise Statistics

The filter performance will be degraded in the measurements with time-varying and unknown noise statistics. To combat the above challenges, a modified strong tracking slide window variational adaptive Kalman filter algorithm is proposed in this article. Firstly, the multiple fading factors are integrated into the proposed algorithm to adjust the error covariance. Next, an improved adaptive slide window method is designed for variational Bayesian Kalman filtering by adaptively adjusting the slide window size and correcting the previous state according to the later state, which improves the estimation accuracy and computational efficiency. Finally, the inverse Wishart distribution is considered for modeling process and measurement noise, and the state vector, as well as noise statistics, are inferred via the variational Bayesian technique without prior noise covariance information. Simulation results demonstrate that the proposed filter algorithm is more robust than existing filters in counteracting measurement and process noise uncertainties.

A Pearson-Type VII Distribution With Adaptive Parameters Selection-Based Interacting Multiple Model Kalman Filter

Existing robust filters under generalized nonstationary noises conditions are difficult to choose suitable prior parameters, this brief proposed a Pearson-type VII distribution with adaptive parameters selection based interacting multiple model (IMM) Kalman filter (IMMPVII KF). In the model conditional filtering process, both the one-step prediction and the measurement likelihood are modeled as Pearson-type VII distributions. They are decomposed into Gaussian-Gamma Hierarchies (GGH), which are then matched to the time-varying heavy-tailed properties of the noises by pre-selecting the sets of shape and rate parameters and the variational Bayesian (VB) technique. Finally, a new model probability update method for filter under non-Gaussian conditions is derived. Simulation results show that the filter proposed in this brief has better robustness and adaptability to generalized non-stationary noises than the existing filters.

Bayesian Inference for State-Space Models With Student-t Mixture Distributions.

This article proposes a robust Bayesian inference approach for linear state-space models with nonstationary and heavy-tailed noise for robust state estimation. The predicted distribution is modeled as the hierarchical Student- t distribution, while the likelihood function is modified to the Student- t mixture distribution. By learning the corresponding parameters online, informative components of the Student- t mixture distribution are adapted to approximate the statistics of potential uncertainties. Then, the obstacle caused by the coupling of the updated parameters is eliminated by the variational Bayesian (VB) technique and fixed-point iterations. Discussions are provided to show the reasons for the achieved advantages analytically. Using the Newtonian tracking example and a three degree-of-freedom (DOF) hover system, we show that the proposed inference approach exhibits better performance compared with the existing method in the presence of modeling uncertainties and measurement outliers.

A variational Bayesian inference technique for model updating of structural systems with unknown noise statistics

Dynamic models of structural and mechanical systems can be updated to match the measured data through a Bayesian inference process. However, the performance of classical (non-adaptive) Bayesian model updating approaches decreases significantly when the pre-assumed statistical characteristics of the model prediction error are violated. To overcome this issue, this paper presents an adaptive recursive variational Bayesian approach to estimate the statistical characteristics of the prediction error jointly with the unknown model parameters. This approach improves the accuracy and robustness of model updating by including the estimation of model prediction error. The performance of this approach is demonstrated using numerically simulated data obtained from a structural frame with material non-linearity under earthquake excitation. Results show that in the presence of non-stationary noise/error, the non-adaptive approach fails to estimate unknown model parameters, whereas the proposed approach can accurately estimate them.

Implicit Kalman filtering method for remaining useful life prediction of rolling bearing with adaptive detection of degradation stage transition point

Remaining useful life (RUL) prediction is a vital task in rolling bearing prognostics and health management (PHM) process. Kalman filtering (KF) is one of the hot spots in the research area of RUL prediction. However, three dispiriting shortcomings in KF methods are still unavoidable, including: (1) difficulty in tracking the unknown time-varying noise information, (2) the subjectivity for setting time to start prediction (TSP), and (3) short-term accuracy of the predicting results based on linear predictors. To improve the capability of KF methods, this work adopts the variational Bayesian technique to adaptively describe noise information and considers linear and nonlinear factors of multi-channel signals to recognize the degradation stage transition point of bearing as TSP. Moreover, this work proposes an implicit Kalman filtering method to predict the RUL. The effectiveness of the proposed method is validated on XJTU-SY and IMS-Rexnord bearing data. Results show that the proposed method can recognize the TSP and improve the long-term accuracy of the prediction result during the accelerated degradation stage.

Robust interacting multiple model cubature Kalman filter for nonlinear filtering with unknown non-Gaussian noise

Estimating noise covariance matrices and suppressing outliers simultaneously is a huge challenge because they are coupled. In order to solve the nonlinear filtering problem with unknown non-Gaussian noise, the maximum correntropy criterion (MCC) is used to suppress the outliers, while the variational Bayesian (VB) technique is used to estimate the one-step prediction error covariance matrix (PECM) and the measurement noise covariance matrix (MNCM), and the nonlinear problem is solved by the third-order spherical radial cubature rule. In this paper, nominal measurement is constructed to eliminate the influence of outliers on the recursive estimation of MNCM. Through these operations, the variational Bayesian and maximum correntropy based cubature Kalman filter (VBMCCKF) and the maximum correntropy cubature Kalman filter (MCCKF) are derived. The interacting multiple model (IMM) fusion framework is used to promote the collaborative work of VBMCCKF and MCCKF, thus a novel filter is proposed in this paper. The simulation verifies the validity and universal applicability of the proposed filter. The estimation error of the proposed filter in spacecraft autonomous celestial navigation is about 30% less than that of the existing filter with the best performance.

Design of robust Gaussian approximate filter and smoother with latency probability identification

The Kalman filter is a linear optimal estimator when it can accurately receive measurements and the noise obeys a Gaussian distribution. However, the measurement noise may have heavy-tailed characteristics because of outlier interference. Actually, real-world measurement information may have one-step randomly delayed measurements (ORDMs) with unknown latency probability (LP) due to exogenous disturbances. Moreover, systems are usually not linear. To overcome the influence of the state estimator of nonlinear systems from heavy-tailed measurement noise (HMN) and the unknown LP, this paper focuses on the robust Gaussian approximate (GA) filter and smoother for the above issues. First, the one-step predicted probability density function (PDF) is modeled as a Gaussian distribution, and the likelihood PDF is modeled as a Normal–Gamma–Beta (NGBM) distribution. Furthermore, novel robust Gaussian approximate filter and smoother are proposed based on the variational Bayesian technique. They provide general estimation frameworks for estimating state of nonlinear systems with HMN and ORDMs, and different nonlinear estimators can be implemented using different numerical techniques. In addition, the computational complexity of the proposed algorithms is calculated. Finally, taking a univariate nonstationary growth model, a passive ranging problem, and target tracking as examples, the effectiveness and superiority of the proposed robust algorithms contrasted with the existing algorithms are shown. The outcomes show that the proposed methods have higher estimation precision; however, the computational burden is marginally increased.

High-Resolution Fast-Rotating Sound Localization Based on Modal Composition Beamforming and Bayesian Inversion

Rotating source beamforming techniques have been effective means of noise localization on rotary machines. In this letter, we derive an alternative expression for modal composition beamforming (MCB) and subsequently consider the equivalent source assumption and cyclostationarity of the constant angular-speed rotating sound source so that a rotating sound source power (RSP) propagation model is derived. By estimating a suitable solution for the RSP model using the subspace variational Bayesian (SVB) technique with sparsity and total variation (TV) priors, the validity of the RSP model was established. According to the simulation results, the proposed RSP-SVB method leads to a significantly higher resolution than the MCB method. It can localize multiple fast-rotating sound sources accurately, rapidly, and effectively in environments with strong background noise interference. Therefore, our proposed RSP-SVB can offer a reliable solution for identifying fast-rotating blade noise.

Variational Bayesian Based Localization for Intelligent Vehicle Using Lidar and GPS Data Fusion: Algorithm and Experiments

Accurate localization is crucial for safe operation of intelligent vehicle (IV). In practice, global positioning system (GPS) signals sometimes may be contaminated or lost, resulting in inaccurate positions of IV. In this article, the distance of IV’s position between previous frame and current frame is derived from Lidar point cloud registration. A novel slide window variational Bayesian (VB) based localization method is proposed for IV with multiple dynamics by fusing GPS and Lidar. In the proposed method, the state of IV, the motion model identity of IV, the measurement loss identity, and the loss probability of measurement are jointly estimated by the VB technique. The effectiveness of the proposed localization method is validated by simulations and field experiments.

Return of the normal distribution: Flexible deep continual learning with variational auto-encoders

Learning continually from sequentially arriving data has been a long standing challenge in machine learning. An emergent body of deep learning literature suggests various solutions, through introduction of significant simplifications to the problem statement. As a consequence of a growing focus on particular tasks and their respective benchmark assumptions, these efforts are thus becoming increasingly tailored to specific settings. Whereas approaches that leverage Variational Bayesian techniques seem to provide a more general perspective of key continual learning mechanisms, they however entail their own caveats. Inspired by prior theoretical work on solving the prevalent mismatch between prior and aggregate posterior in deep generative models, we return to a generic variational auto-encoder based formulation and investigate its utility for continual learning. Specifically, we propose to adapt a two-stage training framework towards a context conditioned variant for continual learning, where we then formulate mechanisms to alleviate catastrophic forgetting through choices of generative rehearsal or well-motivated extraction of data exemplar subsets. Although the proposed generic two-stage variational auto-encoder is not tailored towards a particular task and allows for flexible amounts of supervision, we empirically demonstrate it to surpass task-tailored methods in both supervised classification, as well as unsupervised representation learning.

Improved state estimator for linear-Gaussian systems subject to initialization errors

This paper proposes an improved state estimator for linear-Gaussian systems subject to initialization errors. A one-step prediction function depicted by the Student-t is reconstructed artificially by inserting an auxiliary variable into the original Gaussian distribution. The variational Bayesian (VB) technique is then employed to obtain the approximated posterior joint distribution of the additional variable and the state. A fixed-point iteration is used for recursions to calculate the necessary moments of state with the updated distribution of the auxiliary variable. Simulation and experiment verify the performance of the proposed algorithm. It shows that the proposed method yields significant improvements over the existing initialization approaches, such as the practical Kalman filter (KF) and the recently-developed Bayesian initialization algorithm.

VB-Kalman Based Localization for Connected Vehicles With Delayed and Lost Measurements: Theory and Experiments

Traditionally, connected vehicles (CVs) share their own sensor data that relies on the satellite with their surrounding vehicles by vehicle-to-vehicle (V2V) communication. However, the satellite-based signal sometimes may be lost due to environmental factors. Time-delays and packet dropouts may occur randomly by V2V communication. To ensure the reliability and accuracy of localization for CVs, a novel variational Bayesian (VB)-Kalman method is developed for unknown and time varying probabilities of delayed and lost measurements. In this VB-Kalman localization method, two random variables are introduced to indicate whether a measurement is delayed and available, respectively. A hierarchical model is then formulated and its parameters and state are simultaneously estimated by the VB technique. Experimental results validate the proposed method for the localization of CVs in practice.

A new Gaussian–Student's t mixing distribution‐based Kalman filter with unknown measurement random delay rate

SummaryIn this article, a new Gaussian–Student's t mixing distribution‐based Kalman filter is presented to investigate the filtering issue for linear stochastic system with unknown measurement random delay rate and non‐stationary heavy‐tailed measurement noise. Firstly, by employing a Bernoulli distributed variable and introducing system state extension method, the form of measurement likelihood function of double measurement noise distributions is converted from the weighted sum to an exponential product. Secondly, the non‐stationary heavy‐tailed measurement noises of current time and last time are modeled as Gaussian–Student's t mixing distributions by introducing extra Bernoulli distributed variables. Thirdly, the variational Bayesian technique is utilized to jointly infer the system state, the Bernoulli distributed variables of measurement noises, distribution mixing probabilities, the intermediate random variables, the Bernoulli distributed variable of measurement delay and the unknown measurement random delay rate. Finally, the effectiveness of the proposed Kalman filter is demonstrated by a target tracking simulation experiment.

Multi-Extended Target Tracking Algorithm Based on VBEM-CPHD

Considering that the extended targets tracking problem of the measurement is a glint noise with an unknown inverse covariance, a new algorithm from a multi-extended target tracking based on the Variational Bayesian Expectation Maximization (VBEM) is proposed. To improve the variational Bayesian technique, a modeled Student’s t distribution is proposed based on multiple Gaussian mixture terms in order to replace the probability hypothesis density (PHD) intensity. The extended target was modeled with a Student’s t distribution with the glint noise, using Gauss-Gamma distribution combining the variational Bayesian technique to obtain an approximate distribution and applying the expectation-maximization algorithm for iterative estimation. The two experiments are compared and analyzed with the VBEM-CPHD algorithm and the traditional extended target tracking algorithm. Experiment 1 estimated the trajectory of the target, compared algorithms of VBEM-CPHD and GM-CPHD with OSPA distance, varied glint noise with the three different measurement noise standard deviations. Experiment 2 completed the examination of the tracking performance and stability of the proposed method and performed to compare the VBEM-CPHD algorithm proposed with the VB-based GM-CPHD (GM-VBCPHD) algorithm under an unknown measurement noise covariance. The experimental results indicate that the algorithm of VBEM-CPHD has high tracking accuracy, good adaptability, and a strong antijamming ability for multiple extended targets under glint noise conditions.

An adaptive IMM filter for jump Markov systems with inaccurate noise covariances in the presence of missing measurements

This paper derives a Kullback-Leibler average (KLA) based adaptive interacting multiple model (IMM) filter for jump Markov systems with inaccurate noise covariances in the presence of missing measurements. Firstly, a switching error model (SEM) is constructed to model the probability density function of measurement likelihood. The SEM can automatically choose appropriate error model for the real measurement or pure measurement noise by estimating a binary indicator. Secondly, by assigning conjugate priors on inaccurate noise covariances and binary indicator, the state, noise covariances and binary indicator are jointly estimated based on variational Bayesian (VB) technique. Finally, by using the KLA fusion scheme to fuse the conditioned estimates from every mode in the mixing and output phase, an adaptive IMM filter is derived. Simulation results demonstrate that the proposed filter achieves better performance than existing typical algorithms.

Collective Variational Inference for Personalized and Generative Physiological Modeling: A Case Study on Hemorrhage Resuscitation.

Individual physiological experiments typically provide useful but incomplete information about a studied physiological process. As a result, inferring the unknown parameters of a physiological model from experimental data is often challenging. The objective of this paper is to propose and illustrate the efficacy of a collective variational inference (C-VI) method, intended to reconcile low-information and heterogeneous data from a collection of experiments to produce robust personalized and generative physiological models. To derive the C-VI method, we utilize a probabilistic graphical model to impose structure on the available physiological data, and algorithmically characterize the graphical model using variational Bayesian inference techniques. To illustrate the efficacy of the C-VI method, we apply it to a case study on the mathematical modeling of hemorrhage resuscitation. In the context of hemorrhage resuscitation modeling, the C-VI method could reconcile heterogeneous combinations of hematocrit, cardiac output, and blood pressure data across multiple experiments to obtain (i) robust personalized models along with associated measures of uncertainty and signal quality, and (ii) a generative model capable of reproducing the physiological behavior of the population. The C-VI method facilitates the personalized and generative modeling of physiological processes in the presence of low-information and heterogeneous data. The resulting models provide a solid basis for the development and testing of interpretable physiological monitoring, decision-support, and closed-loop control algorithms.