Variational Bayesian Inference Framework Research Articles

Robust equalization for single-carrier underwater acoustic communication in sparse impulsive interference environment

Impulse noise significantly distorts the background noise within communication systems, causing a notable departure from the Gaussian distribution. This distortion, in turn, exerts a detrimental influence on the performance of conventional single-carrier frequency domain equalization techniques. In this paper, we introduce a novel algorithm for joint channel, interference, and symbol estimation within single-carrier systems. This algorithm is grounded in the framework of variational Bayesian inference and possesses the unique capability of simultaneously accounting for the sparsity inherent in the channel and impulse interference. In the course of processing, it leverages multi-parameter probability distributions to effectively model the intricate posteriori probabilities associated with impulse interference vectors and channel vectors. This, in turn, facilitates the computation of maximum a posteriori estimates and subsequently enables the estimation and elimination of interference. We validate the efficacy of our proposed method through both numerical simulations and the processing of experimental data. Our results unequivocally demonstrate the robustness and manageable computational complexity of our approach within the context of impulse interference-prone environments.

Density-adaptive registration of pointclouds based on Dirichlet Process Gaussian Mixture Models

We propose an algorithm for rigid registration of pre- and intra-operative patient anatomy, represented as pointclouds, during minimally invasive surgery. This capability is essential for development of augmented reality systems for guiding such interventions. Key challenges in this context are differences in the point density in the pre- and intra-operative pointclouds, and potentially low spatial overlap between the two. Solutions, correspondingly, must be robust to both of these phenomena. We formulated a pointclouds registration approach which considers the pointclouds after rigid transformation to be observations of a global non-parametric probabilistic model named Dirichlet Process Gaussian Mixture Model. The registration problem is solved by minimizing the Kullback-Leibler divergence in a variational Bayesian inference framework. By this means, all unknown parameters are recursively inferred, including, importantly, the optimal number of mixture model components, which ensures the model complexity efficiently matches that of the observed data. By presenting the pointclouds as KDTrees, both the data and model are expanded in a coarse-to-fine style. The scanning weight of each point is estimated by its neighborhood, imparting the algorithm with robustness to point density variations. Experiments on several datasets with different levels of noise, outliers and pointcloud overlap show that our method has a comparable accuracy, but higher efficiency than existing Gaussian Mixture Model methods, whose performance is sensitive to the number of model components.

Robust sparse Bayesian learning based on the Bernoulli-Gaussian model of impulsive noise

Traditional sparse reconstruction methods suffer from performance degradation in the presence of impulsive noise. In this paper, we adopt the Bernoulli-Gaussian distribution to model the impulsive noise and introduce the hidden state variables to indicate the state of the noise. Based on the new hierarchical Bayesian model, a robust sparse reconstruction algorithm called Bernoulli-Gaussian robust sparse Bayesian learning (BG-RSBL) is developed through the variational Bayesian inference framework. Different from existing robust reconstruction methods, BG-RSBL does not make sparse assumptions about the impulses and outliers. The employment of the Bernoulli-Gaussian model enables BG-RSBL to adapt to environments with a wider range of impulse densities. Comparative analysis shows that the reconstruction performance can be improved by including the outliers in posterior inference with appropriate weightings. Simulations performed under both Bernoulli-Gaussian and symmetric α-stable distributed impulsive noise demonstrate the robust and outstanding reconstruction performance of the proposed algorithm.

Variational Bayesian Inference Time Delay Estimation for Passive Sonars

In passive sonars, distance and depth estimation of underwater targets is often limited by the accuracy of time delay estimations. The estimation accuracy of the existing methods of time delay estimation is limited by the uniform discrete grid (signal sampling rate). When a true time delay is out of the grid, the estimation accuracy deteriorates due to the mismatch between the real-time delay and the discrete grid. This paper proposes a new method for time delay estimation, which realizes the time delay estimation under the framework of variational Bayesian inference. The proposed method is grid-less, that is, continuous in the time domain. Unlike the popular grid-less compressive time delay estimation method, this method does not require parameter adjustment, and can automatically estimate the number of time delays, noise variance, and amplitude variance. The simulation results showed that the performance of the proposed method was superior to the reference state-of-the-art time delay estimation methods.

Anisotropic Generalized Bayesian Coherent Point Drift for Point Set Registration

Registration is highly demanded in many real-world scenarios such as robotics and automation. Registration is challenging partly due to the fact that the acquired data is usually noisy and has many outliers. In addition, in many practical applications, one point set (PS) usually only covers a partial region of the other PS. Thus, most existing registration algorithms cannot guarantee theoretical convergence. This article presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets (PSs) are aligned and the anisotropic positional noise is considered. The high-dimensional point sets typically consist of the positional vectors and normal vectors. On one hand, with the normal vectors, the proposed method is more robust to noise and outliers, and the point correspondences can be found more accurately. On the other hand, incorporating the registration into the BCPD framework will guarantee the algorithm’s theoretical convergence. Our contributions in this article are three folds. First, the problem of rigidly aligning two general PSs with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm’s iterations are given in closed-form or with iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD. Note to Practitioners—This paper was motivated by the problem of 3D rigid PSR for computer-assisted surgery (CAS), especially in orthopedic applications. The proposed algorithm is also suitable for other scenarios where the initial coarse registration is conducted. The traditional registration methods are susceptible to noise (especially anisotropic noise), outliers, and incomplete partial data. This paper generalizes the recently proposed BCPD method to the six-dimensional scenario where anisotropic positional noise is considered and normal vectors are incorporated. The proposed noise model is decomposed into three parts to be solved alternately: the membership probability of mixture distributions, the soft correspondence estimation, and the model parameters (i.e., the rotation matrix, translation vector, the covariance matrix with the anisotropic positional error, and the concentration parameter with the estimation of the normal vectors). Especially, the convergence is guaranteed at the theoretical level using the variational inference theory. The experimental results demonstrate the superiority of our algorithm on registration accuracy, convergence speed, and robustness to noise, outliers, and partial data.

Posterior estimation using deep learning: a simulation study of compartmental modeling in dynamic positron emission tomography.

In medical imaging, images are usually treated as deterministic, while their uncertainties are largely underexplored. This work aims at using deep learning to efficiently estimate posterior distributions of imaging parameters, which in turn can be used to derive the most probable parameters as well as theiruncertainties. Our deep learning-based approaches are based on a variational Bayesian inference framework, which is implemented using two different deep neural networks based on conditional variational auto-encoder (CVAE), CVAE-dual-encoder, and CVAE-dual-decoder. The conventional CVAE framework, that is, CVAE-vanilla, can be regarded as a simplified case of these two neural networks. We applied these approaches to a simulation study of dynamic brain PET imaging using a reference region-based kineticmodel. In the simulation study, we estimated posterior distributions of PET kinetic parameters given a measurement of the time-activity curve. Our proposed CVAE-dual-encoder and CVAE-dual-decoder yield results that are in good agreement with the asymptotically unbiased posterior distributions sampled by Markov Chain Monte Carlo (MCMC). The CVAE-vanilla can also be used for estimating posterior distributions, although it has an inferior performance to both CVAE-dual-encoder andCVAE-dual-decoder. We have evaluated the performance of our deep learning approaches for estimating posterior distributions in dynamic brain PET. Our deep learning approaches yield posterior distributions, which are in good agreement with unbiased distributions estimated by MCMC. All these neural networks have different characteristics and can be chosen by the user for specific applications. The proposed methods are general and can be adapted to otherproblems.

Uncertainty‐aware convolutional neural network for explainable artificial intelligence‐assisted disaster damage assessment

Accurate damage assessment is a critical step in post-disaster risk assessment, mitigation, and recovery. Current practices performed by experts and reconnaissance teams in the form of field evaluation require considerable time and resources. Recent advances in remote sensing imagery, artificial intelligence (AI), and computer vision have enhanced automated and rapid disaster damage assessment. Recent literature has shown promising progress in AI-assisted aerial damage assessment. However, accounting for the uncertainty in the outcome for improved quantification of confidence and enhanced model explainability for human decision-makers remains one of the key challenges. Overlooking uncertainty can lead to erroneous decisions, especially in highly-consequential tasks such as damage assessment. The aim of this study is to develop uncertainty-aware deep learning models for the assessment of post-disaster damage using aerial imaging. Within the framework of variational Bayesian inference, Monte Carlo dropout sampling technique is used to propagate epistemic uncertainty in model predictions. With this stochastic setting, the model produces damage prediction labels with softmax as random variables, which helps quantify confidence in the model outcome using appropriate measures of uncertainty. Two networks are implemented and trained separately on two different disaster damage datasets consisting of unmanned aerial vehicle building footage as well as satellite-captured post-disaster imagery. The first network attains 59.4% accuracy in building classification, and the second network gives an accuracy of 55.1%. Results from uncertainty analysis, model confidence quantification, and analyzing model attention zone can lead to more explainable and risk-informed automated damage assessment outcomes using AI technology.

An R package VIGoR for joint estimation of multiple linear learners with variational Bayesian inference.

SummaryAn R package that can implement multiple linear learners, including penalized regression and regression with spike and slab priors, in a single model has been developed. Solutions are obtained with fast minorize-maximization algorithms in the framework of variational Bayesian inference. This package helps to incorporate multimodal and high-dimensional explanatory variables in a single regression model.Availability and implementationThe R package VIGoR (Variational Bayesian Inference for Genome-wide Regression) is available at the Comprehensive R Archive Network (CRAN) (https://cran.r-project.org/) and at GitHub (https://github.com/Onogi/VIGoR).Supplementary information Supplementary data are available at Bioinformatics online.

A Hierarchical Bayesian Inversion Method for Electromagnetic Imaging of Inhomogeneous Objects With Piecewise Homogeneities

Electromagnetic imaging of inhomogeneous objects is a nonlinear inverse scattering problem which can be effectively handled by means of linear inversion methods or nonlinear optimization methods. In addition to being nonlinear, inverse scattering problems are also ill-posed which can be overcome through a regularization scheme by taking into account some kind of a priori information. However, it is not straightforward to incorporate such inhomogeneity prior information into conventional inversion methods. In this work, we propose an innovative method to solve the problem of imaging inhomogeneous objects with piecewise homogeneity. Under Born approximation (BA), this problem is cast into a linear inversion one which is then addressed by means of a hierarchical Bayesian inversion method. To account for the piecewise homogeneity of the image, a Gaussian mixture model (GMM) is developed to enforce the feature. Specifically, a hidden label variable which is associated with each pixel is introduced for the class of materials and the contrast of each pixel is modeled as a mixture of independent Gaussian priors. Then, the proposed hierarchical Bayesian inversion method incorporates the GMM to the variational Bayesian inference framework to approximate the posterior distribution of all the unknowns (the unknown contrast, hidden label variable, and other model parameters). In addition, the damped generalized approximate message passing (GAMP) is utilized to circumvent the matrix inversion, making the overall process very computationally efficient. Extensive numerical examples will be shown to demonstrate the effectiveness of the proposed method.

Combining a DSGE Model with Neural Networks for Forecasting Economic Variables

This paper examines the possibility of combining a DSGE model and neural networks to supplement each other, in order to yield a good performance in out-of-sample forecasts for economic variables. The novel neural-net structure of TDVAE (Temporal Difference Variational Auto-Encoder) proposed by Gregor et.al [2019] helps to realize this idea. TDVAE virtually replicates a stochastic state-space model through combinations of neural networks within the framework of variational Bayesian inference. Because a DSGE model provides theoretical restrictions on the state transition matrices of a state-space model, I choose to transplant those DSGE-oriented matrices into the probabilistic model of state transition in TDVAE. This TDVAE-DSGE approach certainly achieved the superior performance in the task of out-of-sample forecasts on Japan's major macroeconomic variables during 1Q/2011 and 4Q/2019.

Enhancing the Accuracy and Robustness of a Compressive Sensing Based Device-Free Localization by Exploiting Channel Diversity.

As an emerging and promising technique, device-free localization (DFL) estimates target positions by analyzing their shadowing effects. Most existing compressive sensing (CS)-based DFL methods use the changes of received signal strength (RSS) to approximate the shadowing effects. However, in changing environments, RSS readings are vulnerable to environmental dynamics. The deviation between runtime RSS variations and the data in a fixed dictionary can significantly deteriorate the performance of DFL. In this paper, we introduce ComDec, a novel CS-based DFL method using channel state information (CSI) to enhance localization accuracy and robustness. To exploit the channel diversity of CSI measurements, the DFL problem is formulated as a joint sparse recovery problem that recovers multiple sparse vectors with common support. To solve this problem, we develop a joint sparse recovery algorithm under the variational Bayesian inference framework. In this algorithm, dictionaries are parameterized based on the saddle surface model. To adapt to the environmental changes and different channel characteristics, dictionary parameters are modelled as tunable parameters. Simulation results verified the superior performance of ComDec as compared with other state-of-the-art CS-based DFL methods.

Variational Bayesian Inference-Based Counting and Localization for Off-Grid Targets With Faulty Prior Information in Wireless Sensor Networks

The benefits of compressive sensing (CS) on counting and localization in wireless sensor networks have already been demonstrated in previous works. However, most existing solutions usually become infeasible in the presence of off-grid targets, which is inevitable for the traditional CS-based localization methods. To mitigate the error caused by off-grid targets, in this paper, a more accurate sparse approximation model is applied where the true and unknown sparsifying dictionary is approximated with its first-order Taylor expansion around a known dictionary. Based on the model, the counting and localization problem is formulated as a sparse reconstruction problem that recovers two sparse signals with related supports. To solve the problem, we develop an iterative algorithm under a variational Bayesian inference framework. Moreover, in practice, some faulty prior information (e.g., coarse positions) is usually available. To incorporate the information, a three-level hierarchical prior model is introduced and imposed on the sparse signals to be recovered, where the supports of sparse signals are learned in the third level. Additionally, a grid pruning mechanism and matrix inversion lemma are leveraged to accelerate the inference process. Extensive simulation results demonstrate that, compared with state-of-the-art algorithms, the proposed algorithm can achieve counting and localization with higher accuracy as well as lower cost.

Multiframe super-resolution based on half-quadratic prior with artifacts suppress

The multiframe super-resolution (SR) technique aims to obtain a high-resolution (HR) image by using a set of observed low-resolution (LR) images. In the reconstruction process, artifacts may be possibly produced due to the noise, especially in presence of stronger noise. In order to suppress artifacts while preserving discontinuities of images, in this paper a multiframe SR method is proposed by involving the reconstruction properties of the half-quadratic prior model together with the quadratic prior model using a convex combination. Moreover, by analyzing local features of the underlined HR image, these two prior models are combined by using an automatically calculated weight function, making both smooth and discontinuous pixels handled properly. A variational Bayesian inference (VBF) based algorithm is designed to efficiently and effectively seek the solution of the proposed method. With the VBF framework, motion parameters and hyper-parameters are all determined automatically, leading to an unsupervised SR method. The efficiency of the hybrid prior model is demonstrated theoretically and practically, which shows that our SR method can obtain better results from LR images even with stronger noise. Extensive experiments on several visual data have demonstrated the efficacy and superior performance of the proposed algorithm, which can not only preserve image details but also suppress artifacts.

On Symbol-Wise Variational Bayesian CSI Estimation and Detection for Distributed Antenna Systems Subjected to Multiple Unknown Jammers

This letter develops a novel iterative joint channel state information estimation (CSIE), jammer estimation (JE) and data detection (DD) algorithm for orthogonal frequency division multiplexing (OFDM)-based distributed antenna systems in the presence of multiple unknown jammers. In contrast to existing methods that only heuristically combine CSIE, JE and DD, the proposed algorithm rigorously casts the three problems under a unified variational Bayesian inference framework. Simulation results demonstrate that the proposed algorithm converges rapidly, and after convergence, the bit-error-rate performance of the proposed algorithm is very close to that of the ideal case which supposes perfect knowledge of CSI and jammer. In addition, it is shown that the proposed algorithm significantly outperforms other state-of-the-art methods.

A Fast and Accurate Reconstruction Algorithm for Compressed Sensing of Complex Sinusoids

The standard compressed sensing (CS) theory reconstructs a signal by recovering a sparse representation of the signal over a pre-specified dictionary. For CS of complex sinusoids, this dictionary is usually set to be a DFT matrix corresponding to a uniform frequency grid. However, such a setting can make conventional CS reconstruction methods degrade considerably, since component frequencies of practical signals do not necessarily align with the specified grid. To deal with this problem, we apply a linear approximation to the true unknown dictionary and establish a more accurate model for sparse approximation of practical complex sinusoids. Based on this model, signal reconstruction is reformulated as a problem that recovers two sparse coefficient vectors over two known dictionaries under the constraint that the vectors share the same support. To solve such a problem, we develop a fast iterative algorithm under a variational Bayesian inference framework. Results of extensive numerical experiments demonstrate that the algorithm can achieve CS of complex sinusoids with low computational cost as well as high reconstruction accuracy.

Signal Detection for OFDM-Based Virtual MIMO Systems under Unknown Doubly Selective Channels, Multiple Interferences and Phase Noises

In this paper, the challenging problem of signal detection under severe communication environment that plagued by unknown doubly selective channels (DSCs), multiple narrowband interferences (NBIs) and phase noises (PNs) is investigated for orthogonal frequency division multiplexing based virtual multiple-input multiple-output (OFDM-V-MIMO) systems. Based on the Variational Bayesian Inference framework, a novel iterative algorithm for joint signal detection, DSC, NBI and PN estimations is proposed. Simulation results demonstrate quick convergence of the proposed algorithm, and after convergence, the bit-error-rate performance of the proposed signal detection algorithm is very close to that of the ideal case which assumes perfect channel state information, no PN, and known positions and powers of NBIs plus additive white Gaussian noise. Furthermore, simulation results show that the proposed signal detection algorithm outperforms other state-of-the-art methods.