Variational Bayesian Algorithm Research Articles

Robotic cross-modal generative adversarial network based on variational Bayesian Gaussian mixture noise model

Multimodal fusion is essential for robots to fully perceive the external environment. Single modal information limits the ability of robots to recognize and grasp objects. Meanwhile, traditional cross-modal data generation methods produce poor images, resulting in the bad effects of multimodal fusion. To solve the problem of the poor image effects of multimodal generation and the lack of data for multimodal fusion, this study proposes a variational Bayesian Gaussian mixture-conditional generative adversarial network (BGM-CGAN) for generating diverse cross-modal noise data. With the variational Bayesian Gaussian mixture algorithm, a uniform distributed random noise group is generated into a single mixed variable. The generated mixed variable is then generated through a Gaussian mixture model to generate a series of Gaussian mixed noise groups. The generated mixed noise group is randomly selected into a single Gaussian noise and imported into a modal image and fused with the modal image to successfully generate a high-resolution heterogeneous modal image. The method restores the heterogeneous modal information and solves the problem of the insufficient information and poor quality of generated images under a single modal. In the end, we use a variety of evaluation indexes (IS, FID, SSIM, and PSNR) to compare the proposed BGM-CGAN and other algorithms for cross-modal image generation capabilities. Results show the effectiveness and feasibility of the proposed algorithm. In addition, the BGM-CGAN algorithm has wide application prospects and can be extended to cross-modal material retrieval, cross-modal texture recognition, and other fields.

An Adaptive Vector Tracking Scheme for High-Orbit Degraded GNSS Signal

Global navigation satellite system (GNSS) receivers meet numerous challenges in a high-orbit environment, including weak and discontinuous signal, and time-varying strength. To resolve these issues and enhance reliability, an innovative adaptive vector tracking loop (VTL) scheme is proposed. Non-linear models of the VTL filter are established to calculate code phase and carrier frequency errors accurately. Based on this, a deep analysis has been developed on the measurement noise. To reduce the impact of the interdependent noises among channels in VTL, an adaptive VTL algorithm assisted by the variational Bayesian (VB) learning network is proposed to estimate the measurement noise and maintain the error convergence in the time-varying noise or signal outage conditions. Further, the implementation steps of the adaptive algorithm have been designed in detail. In particular, the carrier-to-noise power ratio (C/N0) estimation method is further employed to update the a prior probability density in case of change of tracking satellite. The simulation results indicate that the proposed VTL scheme with VB algorithm is a promising method to improve the accuracy and reliability of GNSS receivers significantly under a high-orbit degraded signal environment.

Multi-Vehicle Cooperative Target Tracking with Time-Varying Localization Uncertainty via Recursive Variational Bayesian Inference.

Cooperative target tracking by multiple vehicles connected through inter-vehicle communication is a promising way to improve the estimation of target state. The effectiveness of cooperative tracking closely depends on the accuracy of relative localization between host and cooperative vehicles. However, the localization signal usually provided by the satellite-based navigation system is rather susceptible to dynamic driving environment, thus influencing the effectiveness of cooperative tracking. In order to implement reliable cooperative tracking, especially when the statistical characteristic of the relative localization noise is time-varying and uncertain, this paper presents a recursive Bayesian framework which jointly estimates the state of the target and the cooperative vehicle as well as the localization noise parameter. An online variational Bayesian inference algorithm is further developed to achieve efficient recursive estimate. The simulation results verify that our proposed algorithm can effectively boost the accuracy of target tracking when the localization noise dynamically changes over time.

A Variational Bayesian Framework for Cluster Analysis in a Complex Network

A complex network is a network with non-trivial topological structures. It contains not just topological information but also attribute information available in the rich content of nodes. Concerning the task of cluster analysis in a complex network, model-based algorithms are preferred over distance-based ones, as they avoid designing specific distance measures. However, their models are only applicable to complex networks where the attribute information is composed of attributes in binary form. To overcome this disadvantage, we introduce a three-layer node-attribute-value hierarchical structure to describe the attribute information in a flexible and interpretable manner. Then, a new Bayesian model is proposed to simulate the generative process of a complex network. In this model, the attribute information is generated by following the hierarchical structure while the links between pairwise nodes are generated by a stochastic blockmodel. To solve the corresponding inference problem, we develop a variational Bayesian algorithm called TARA, which allows us to identify functionally meaningful clusters through an iterative procedure. Our extensive experiment results show that TARA can be an effective algorithm for cluster analysis in a complex network. Moreover, the parallelized version of TARA makes it possible to perform efficiently at its tasks when applied to large complex networks.

An adaptive variational Bayesian filter for nonlinear multi-sensor systems with unknown noise statistics

An improved adaptive variational Bayesian cubature information fusion algorithm for nonlinear multi-sensor systems with uncertain noise statistics is proposed in this paper. Aiming to estimate uncertain process and measurement noise covariances in nonlinear systems, the variational Bayesian theory is combined with the inverse Wishart distribution. System states and uncertain noise covariances are jointly estimated for nonlinear systems by means of cubature sampling, deriving the variational Bayesian cubature Kalman filter (VBCKF-QR). In addition, a variational Bayesian Cubature Information filter (VBCIF-QR) is proposed, and a distributed information feedback fusion algorithm is also derived for multi-sensor systems with unknown noise statistics. Simulation results demonstrate that the proposed VBCKF-QR/VBCIF-QR outperform conventional cubature Kalman/information filters.

Label constrained convolutional factor analysis for classification with limited training samples

This paper mainly addresses the statistical classification robust to small training data size. We develop a label constrained convolutional factor analysis (LCCFA) model, which unifies the factor analysis (FA), convolution operation and supervised learning. In the LCCFA model, each dictionary atom is used as a small-sized convolution kernel with the goal of learning the observations’ basic structures, which have highly shared characteristics among all observed data. This property enables the proposed method to describe data with fewer dictionary atoms than the FA model and reduces the model complexity. Consequently, the classification performance of the LCCFA model can be improved in the case of limited training samples. Meanwhile, the proposed model also projects the weight vectors of dictionary atoms to their class labels to constrain the learning of parameters. The difference in weight vectors from different classes increases due to the label constraint, thereby offering the potential to enhance the inter-class separability of statistical models. Additionally, the efficient parameter estimation is implemented via variational Bayesian (VB) algorithm. Experimental results on several benchmark datasets and measured radar high-resolution range profile (HRRP) data show that our method outperforms other related models in terms of small sample classification.

Improving S-MSCKF With Variational Bayesian Adaptive Nonlinear Filter

In recent years, multi-sensor fusion has been paid a great attention and is widely applied to simultaneous localization and mapping (SLAM), e.g. Visual-Inertial Odometry (VIO). However, in the most of previous works the observation noise of sensor is assumed to be Gaussian with invariant constant mean and covariance, which is not the case actually. In this paper, a variational Bayesian adaptive algorithm is proposed to improve the performance of visual-inertial odometry, where covariance of observation noise is modeled as inverse-Wishart distribution with harmonic mean so that observation noise is non-Gaussian. Besides, unscented transformation is utilized to handle the nonlinear system problem. The proposed algorithm is applied to the framework of stereo multi-state constraint Kalman filter system (S-MSCKF), which is a state-of-the-art VIO system with low computational cost and good robustness. Extensive experiments are performed on simulation data and open source datasets. The results demonstrate that our proposed algorithm is more robust and accurate compared with S-MSCKF.

Clustering Analysis in the Wireless Propagation Channel with a Variational Gaussian Mixture Model

In this paper, the Gaussian mixture model (GMM) is introduced to implement channel multipath clustering. The GMM incorporates the covariance structure and the mean information of the channel multipaths, thus it can effectively reveal the similarity of the channel multipaths. First, the expectation-maximization (EM) algorithm is utilized to search for the posterior estimation of the GMM parameters. Then, the variational Bayesian (VB) algorithm is employed to optimize the GMM parameters to enhance the searching ability of EM and further to determine the optimal number of Gaussian distributions without resorting to cross-validation. Finally, a compact index (CI) is proposed to validate the clustering results reasonably. Thanks to the proposed CI, it is possible to find a close relationship among the GMM clustering mechanism, the multipath propagation characteristics and the CI evaluation index. Experiments with synthetic data and outdoor-to-indoor (O2I) channel data are presented to demonstrate the effectiveness of the proposed method.

An Adaptive Target Tracking Algorithm Based on EKF for AUV with Unknown Non-Gaussian Process Noise

An adaptive target tracking method based on extended Kalman filter (TT-EKF) is proposed to simultaneously estimate the state of an Autonomous Underwater Vehicle (AUV) and an mobile recovery system (MRS) with unknown non-Gaussian process noise in homing process. In the application scenario of this article, the process noise includes the measurement noise of AUV heading and forward speed and the estimation error of MRS heading and forward speed. The accuracy of process noise covariance matrix (PNCM) can affect the state estimation performance of the TT-EKF. The variational Bayesian based algorithm is applied to estimate the process noise statistics. We use a Gaussian mixture distribution to model the non-Gaussian noisy forward speed of AUV and MRS. We use a von-Mises distribution to model the noisy heading of AUV and MRS. The variational Bayesian algorithm is applied to estimate the parameters of these distributions, and then the PNCM can be calculated. The prediction error of TT-EKF is online compensated by using a multilayer neural network, and the neural network is online trained during the target tracking process. Matlab simulation and experimental data analysis results verify the effectiveness of the proposed method.

Remote Sensing Image Denoising via Low-Rank Tensor Approximation and Robust Noise Modeling

Noise removal is a fundamental problem in remote sensing image processing. Most existing methods, however, have not yet attained sufficient robustness in practice, due to more or less neglecting the intrinsic structures of remote sensing images and/or underestimating the complexity of realistic noise. In this paper, we propose a new remote sensing image denoising method by integrating intrinsic image characterization and robust noise modeling. Specifically, we use low-Tucker-rank tensor approximation to capture the global multi-factor correlation within the underlying image, and adopt a non-identical and non-independent distributed mixture of Gaussians (non-i.i.d. MoG) assumption to encode the statistical configurations of the embedded noise. Then, we incorporate the proposed image and noise priors into a full Bayesian generative model and design an efficient variational Bayesian algorithm to infer all involved variables by closed-form equations. Moreover, adaptive strategies for the selection of hyperparameters are further developed to make our algorithm free from burdensome hyperparameter-tuning. Extensive experiments on both simulated and real multispectral/hyperspectral images demonstrate the superiority of the proposed method over the compared state-of-the-art ones.

Joint variational Bayesian based localization estimation algorithm on distributed gas source sensor network

According to the non-linear and unknown diffusion distribution model characteristics of gas leakage diffusion in the real environment, a distributed joint optimal estimation algorithm for gas source localization and parameters estimation was proposed in sensor networks. Firstly, the gas source localization framework was constructed based on the compressed sensing theory, Secondly, the joint sparse estimation method of the state distribution and unknown parameters of gas diffusive source is proposed based on the variational Bayesian inference algorithm. In which, an adaptive grid division strategy is given to improve the accuracy and performance of the joint estimation method and to balance the relationship between energy consumption and network resources. Finally, simulation results show that the proposed algorithm could effectively achieve a joint estimation of the diffusive source state and parameters, which could achieve higher estimation accuracy in a shorter time and meet the real-time requirements in complex environment compared with the traditional CS method.

Variational Bayesian weighted complex network reconstruction

Complex network reconstruction is a hot topic in many fields. Currently, the most popular data-driven reconstruction framework is based on lasso. However, it is found that, in the presence of noise, lasso loses efficiency for weighted networks. This paper builds a new framework to cope with this problem. The key idea is to employ a series of linear regression problems to model the relationship between network nodes, and then to use an efficient variational Bayesian algorithm to infer the unknown coefficients. The numerical experiments conducted on both synthetic and real data demonstrate that the new method outperforms lasso with regard to both reconstruction accuracy and running speed.

Soft Symbol Decoding in Sweep-Spread-Carrier Underwater Acoustic Communications: A Novel Variational Bayesian Algorithm and Its Analysis

Sweep spread carrier (S2C) based underwater acoustic (UWA) communications is a practically attractive but less explored modulation scheme in the published literature. In this paper, we present a rigorous treatment of the S2C communication receiver design and propose a data detection scheme that can handle challenging UWA channels. State-of-the-art S2C receivers based on the gradient heterodyne processing are only effective when the path delay and Doppler spread are moderate. We develop a new variational soft symbol decoding (VSSD) algorithm based on the principle of variational Bayes’ inference for a general linear channel model. In channels with moderate delay and Doppler spreads, we show that the VSSD algorithm is equivalent to the existing gradient heterodyne receivers for S2C communications. We apply the VSSD algorithm to the i.i.d. Gaussian multiple-input multiple-output channel and show, through numerical simulations, that it far outperforms the minimum mean squared error (MMSE) data detection. We illustrate the dramatic improvement in the performance of the VSSD based S2C receiver in two different models of simulated UWA channels and two contrasting measured UWA environments publicly available in the Watermark channel dataset. The proposed VSSD algorithm recovers data symbols at a signal-to-noise ratio (SNR) which is at least 10 dB (8 dB) lower than the MMSE decoder for uncoded (rate 2/3 LDPC coded) communications over UWA channels where the existing receivers either fail completely or must compromise on the data rate to maintain the bit error rate (BER) performance.

An Improved Real-Time Transfer Alignment Algorithm Based on Adaptive Noise Estimation for Distributed POS

Distributed position and orientation system (POS) plays an important role in the fields of aerial remote sensing, which serves the sensors by precise motion information. For distributed POS, the slave systems consist of low accuracy inertial sensors, which must depend on the high accuracy motion information of the master to proceed transfer alignment (TA) to improve accuracy. Generally, the TA filtering algorithms perform superior performance when the noise statistical characteristic is known and accurate, however like gust, engine vibration and other external disturbances both will cause the inertial sensors output with unknown, varying noises and performance decline. Aiming at this, a variational Bayesian central difference Kalman filtering algorithm for distributed POS real-time TA is developed to suppress the effect of external noise, incorporating the central difference Kalman filtering (CDKF) algorithm and variational Bayesian (VB) adaption theory. In detail, the algorithm is to apply the noise estimation by VB adaption to the CDKF algorithm to achieve real-time TA accuracy enhancement. Distributed POS flight test is devoted for the algorithm validation, by the comparison, evident progress in TA accuracy has been presented.

Detecting topic-level influencers in large-scale scientific networks

Scientific networks play an increasingly important role in facilitating knowledge and technique diffusion. In such networks, highly influential nodes (scientists or literatures) are prone to stimulate other researchers in the generation of innovative ideas. The objective of this study is to detect topic-level influencers from a large collection of links between nodes and textual contents in scientific networks. For this purpose, we propose a sparse link topic model (SLTM) that introduces a “Spike and Slab” prior to achieve sparsity in node-topic distribution. Compared with previous approaches, our model assumes that a node usually focuses on several salient topics instead of a wide range of topics, which is useful in learning topic-level influencers in scientific networks. In addition, a collapsed variational Bayesian (CVB) inference algorithm is designed for large-scale applications. Our experiments are conducted on a large scientific collaboration network. The results reveal that the proposed model significantly improves the precision of topic-level detection. Our analysis also reflects that SLTM can explicitly model the sparse topical structure of each node in the network.

Robust Variational Bayesian Adaptive Cubature Kalman Filtering Algorithm for Simultaneous Localization and Mapping with Heavy-Tailed Noise

Simultaneous localization and mapping (SLAM) has been applied across a wide range of areas from robotics to automatic pilot. Most of the SLAM algorithms are based on the assumption that the noise is timeinvariant Gaussian distribution. In some cases, this assumption no longer holds and the performance of the traditional SLAM algorithms declines. In this paper, we present a robust SLAM algorithm based on variational Bayes method by modelling the observation noise as inverse-Wishart distribution with “harmonic mean”. Besides, cubature integration is utilized to solve the problem of nonlinear system. The proposed algorithm can effectively solve the problem of filtering divergence for traditional filtering algorithm when suffering the time-variant observation noise, especially for heavy-tailed noise. To validate the algorithm, we compare it with other traditional filtering algorithms. The results show the effectiveness of the algorithm.

Variational Bayesian Iterative Estimation Algorithm for Linear Difference Equation Systems

Many basic laws of physics or chemistry can be written in the form of differential equations. With the development of digital signals and computer technology, the research on discrete models has received more and more attention. The estimates of the unknown coefficients in the discretized difference equation can be obtained by optimizing certain criterion functions. In modern control theory, the state-space model transforms high-order differential equations into first-order differential equations by introducing intermediate state variables. In this paper, the parameter estimation problem for linear difference equation systems with uncertain noise is developed. By transforming system equations into state-space models and on the basis of the considered priors of the noise and parameters, a variational Bayesian iterative estimation algorithm is derived from the observation data to obtain the parameter estimates. The unknown states involved in the variational Bayesian algorithm are updated by the Kalman filter. A numerical simulation example is given to validate the effectiveness of the proposed algorithm.

Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise.

Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous noise. A few of the matrix decomposition methods have been extended for such multi-view data integration and pattern discovery while only a few methods were designed to consider the heterogeneity of noise in such multi-view data for data integration explicitly. To this end, in this article, we propose a joint matrix decomposition framework (BJMD), which models the heterogeneity of noise by the Gaussian distribution in a Bayesian framework. We develop two algorithms to solve this model: one is a variational Bayesian inference algorithm, which makes full use of the posterior distribution; and another is a maximum a posterior algorithm, which is more scalable and can be easily paralleled. Extensive experiments on synthetic and real-world datasets demonstrate that BJMD is superior or competitive to the state-of-the-art methods.

Parametric Bayesian estimation of point-like pollution sources of groundwater layers

This paper considers the problem of estimating point-like pollution sources of groundwater layers. To cope with the ill-posed character of this problem, a parametric Bayesian framework has been recently established. In this framework, where the priors for the source parameters are either uniform or Gaussian and the observation noise is homogeneous, a stochastic Markov Chain Monte Carlo (MCMC) algorithm has been proposed to compute the posterior distribution of both source parameters and noise variance. Here, we consider a more general model with truncated-Gaussian priors for pollution quantity and spreading time parameter, which gathers advantages of uniform and Gaussian choices, and an inhomogeneous noise, which accounts for the spatial diversity among sensors. For this model, we extend the existing stochastic algorithm, then propose a concurrent deterministic algorithm based on the variational Bayesian (VB) approach. This approach designs an approximation of the posterior law based on a separable from. The proposed MCMC and VB algorithms target the exact posterior and the approximated posterior, respectively. It is further shown that the former is more accurate, while the latter is computationally more efficient. Results of numerical experiments conducted using an experimental platform to compare the performances of the proposed schemes are presented.

Variational Bayesian Exploration-Based Active Sarsa Algorithm

We proposed an improved variational Bayesian exploration-based active Sarsa (VBE-ASAR) algorithm, which tries to balance the exploration and exploitation dilemma, and speeds up the convergence rate. First, in the learning process, variational Bayesian method is adopted to measure the information gain, which is used as an exploration factor to construct an internal reward function for heuristic exploration. In addition, before the learning process, in order to improve the exploration performance, transfer learning is used to initialize the value function, where Bisimulation metric is introduced to measure the distance between two states from the source MDP and the target MDP, respectively. Finally, we apply the proposed algorithm to the cliff walking problem, and compare with the Sarsa algorithm, the Q-Learning algorithm, the VFT-Sarsa algorithm and the Bayesian Sarsa (BS) algorithm. Experimental results show that the VBE-ASAR algorithm has a faster learning rate.