Variational Bayesian Inference Algorithm Research Articles

Cooperative Spectrum Sensing for Internet of Things Using Modeling of Power-Spectral-Density Estimation Errors

The application of cognitive radio (CR) technology to the Internet of Things (IoT) network can effectively solve the bottleneck problem of spectrum scarcity. As one of the key steps in CR-IoT, the research over spectrum sensing is of great importance. The utilization of the spectrum resource can be described by the power spectral density (PSD) which can be estimated by the IoT cognitive radio (CR) nodes cooperatively. The PSD-estimation induced error (PEIE) at each node can decrease the performance of spectrum sensing seriously, but it often be treated as Gaussian distribution easily in the published works which is not the fact in real systems. This paper tries to propose a general modeling method of PEIE to address this problem. After that, an optimization function is constructed over the PSD. Then the prior model which can lead to sparsity-inducing penalization terms is built by introducing two auxiliary parameters to take full advantage of the sparse property. Also, the variational Bayesian inference algorithm using the factor graph is used to find the iterative solution. Finally, compared with the traditional approach, the simulation results demonstrate the superior performance of the proposed algorithm.

Variational Bayesian Orthogonal Nonnegative Matrix Factorization Over the Stiefel Manifold.

Nonnegative matrix factorization (NMF) is one of the best-known multivariate data analysis techniques. The NMF uniqueness and its rank selection are two major open problems in this field. The solutions uniqueness issue can be addressed by imposing the orthogonality condition on NMF. This constraint yields sparser part-based representations and improved performance in clustering and source separation tasks. However, existing orthogonal NMF algorithms rely mainly on non-probabilistic frameworks that ignore the noise inherent in real-life data and lack variable uncertainties. Thus, in this work, we investigate a new probabilistic formulation of orthogonal NMF (ONMF). In the proposed model, we impose the orthogonality through a directional prior distribution defined on the Stiefel manifold called von Mises-Fisher distribution. This manifold consists of a set of directions that comply with the orthogonality condition that arises in many applications. Moreover, our model involves an automatic relevance determination (ARD) prior to address the model order selection issue. We devised an efficient variational Bayesian inference algorithm to solve the proposed ONMF model, which allows fast processing of large datasets. We evaluated the proposed model, called VBONMF, on the task of blind decomposition of real-world multispectral images of ancient documents. The numerical experiments demonstrate its efficiency and competitiveness compared to the state-of-the-art approaches.

A Vector Processor for Mean Field Bayesian Channel Estimation

Physical layer signal processing algorithms in the wireless domain are seeing increased use of machine learning algorithms, especially Bayesian methods. This work presents the hardware architecture and implementation of a vector processor for one such application, Bayesian channel estimation (CE) (BCE). The BCE vector processor supports a generic instruction set with a supplement of specialized instructions to realize Bayesian algorithms in the signal processing context. The vector processor is designed to work as an accelerator in a system-on-chip (SoC) with an AHB/AXI bus interface or as stand-alone unit. The vector processor achieves more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> improvement in performance when compared with a traditional CE algorithm running on a commercial vector processor. To the best of authors knowledge, this is a first known hardware implementation of a variational Bayesian inference algorithm for a wireless communication application.

Multi-Emitter Localization via Concurrent Variational Bayesian Inference in UAV-Based WSN

Applying Compressive Sensing (CS) to Received Signal Strength (RSS) based multi-emitter localization using Unmanned Arial Vehicles (UAVs) attracts much attention for its simplicity and efficiency. However, the RSS-based CS approach is vulnerable to the noise in a practical scenario. To mitigate this, we propose a robust localization framework for multiple emitters in UAV-based Wireless Sensor Network (WSN). We first approximate the lognormal noise’s influence on the dictionary by a two-layer hierarchical prior model. Then, by exploiting multi-frequency measurements, the multi-emitter localization is transformed into the joint estimation for multiple sparse vectors and noise level. Finally, the joint estimation problem is solved by a Concurrent Variational Bayesian Inference (CVBI) algorithm, where an adaptive grid pruning mechanism is designed. The merits of the proposed framework are testified by numerical simulations.

Variational Bayesian Compressive Multipolarization Indoor Radar Imaging

This article introduces a probabilistic Bayesian model for addressing the problem of compressive multipolarization through-wall radar imaging (TWRI). The proposed approach formulates the task of wall-clutter mitigation and multipolarization image reconstruction as a Bayesian inference problem for a joint distribution between observed radar measurements and latent wall-clutter matrix and indoor target images. The joint probability distribution incorporates three prior beliefs: low-dimensional structure of the wall reflections, group sparsity structure of the target images, and joint sparsity among the polarization images. These signal attributes are modeled through hierarchical priors, whose parameters and hyperparameters are treated with a full Bayesian formulation. Furthermore, this article presents a variational Bayesian inference algorithm that estimates wall-clutter and multipolarization images as posterior distributions and optimizes the model parameters and hyperparameters simultaneously. Experimental results on simulated and real radar data show that the proposed model is very effective at removing wall clutter and enhancing target localization even when the radar measurements are significantly reduced.

Angular-Domain Selective Channel Tracking and Doppler Compensation for High-Mobility mmWave Massive MIMO

In this paper, we consider a mmWave massive multiple-input multiple-output (MIMO) communication system with one static base station (BS) serving a fast-moving user, both equipped with a very large array. The transmitted signal arrives at the user through multiple paths, each with a different angle-of-arrival (AoA) and hence Doppler frequency offset (DFO), thus resulting in a fast time-varying multipath fading MIMO channel. In order to mitigate the Doppler-induced channel aging for reduced pilot overhead, we propose a new angular-domain selective channel tracking and Doppler compensation scheme at the user side. Specifically, we formulate the joint estimation of partial angular–domain channel and DFO parameters as a dynamic compressive sensing (CS) problem. Then we propose a Doppler-aware-dynamic variational Bayesian inference (DD-VBI) algorithm to solve this problem efficiently. Finally, we propose a practical DFO compensation scheme which selects the dominant paths of the fast time-varying channel for DFO compensation and thereby converts it into a slow time-varying effective channel. Compared with the existing methods, the proposed scheme can enjoy the huge array gain provided by the massive MIMO and also balance the tradeoff between the CSI signaling overhead and spatial multiplexing gain. Simulation results verify the advantages of the proposed scheme over various baseline schemes.

Multi-Vehicle Cooperative Target Tracking with Time-Varying Localization Uncertainty via Recursive Variational Bayesian Inference.

Cooperative target tracking by multiple vehicles connected through inter-vehicle communication is a promising way to improve the estimation of target state. The effectiveness of cooperative tracking closely depends on the accuracy of relative localization between host and cooperative vehicles. However, the localization signal usually provided by the satellite-based navigation system is rather susceptible to dynamic driving environment, thus influencing the effectiveness of cooperative tracking. In order to implement reliable cooperative tracking, especially when the statistical characteristic of the relative localization noise is time-varying and uncertain, this paper presents a recursive Bayesian framework which jointly estimates the state of the target and the cooperative vehicle as well as the localization noise parameter. An online variational Bayesian inference algorithm is further developed to achieve efficient recursive estimate. The simulation results verify that our proposed algorithm can effectively boost the accuracy of target tracking when the localization noise dynamically changes over time.

Joint variational Bayesian based localization estimation algorithm on distributed gas source sensor network

According to the non-linear and unknown diffusion distribution model characteristics of gas leakage diffusion in the real environment, a distributed joint optimal estimation algorithm for gas source localization and parameters estimation was proposed in sensor networks. Firstly, the gas source localization framework was constructed based on the compressed sensing theory, Secondly, the joint sparse estimation method of the state distribution and unknown parameters of gas diffusive source is proposed based on the variational Bayesian inference algorithm. In which, an adaptive grid division strategy is given to improve the accuracy and performance of the joint estimation method and to balance the relationship between energy consumption and network resources. Finally, simulation results show that the proposed algorithm could effectively achieve a joint estimation of the diffusive source state and parameters, which could achieve higher estimation accuracy in a shorter time and meet the real-time requirements in complex environment compared with the traditional CS method.

Variational Bayesian Inference for Channel Estimation and User Activity Detection in C-RAN

Cloud radio access network (C-RAN) is recognized as a key technology for the fifth generation (5G) wireless communication systems, where machine-type communication (MTC) is considered to support devices’ connectivity. In this letter, we study the user activity detection (UAD) and channel estimation (CE) problems in C-RAN for MTC. Based on the user activity sparsity and signal spatial sparsity in C-RAN, we build a two-layer prior distribution graphical model to exploit the sparsity property and analyze the problem with variational Bayesian inference (VBI). We find that the width of prior distribution has a considerable impact on the algorithm performance and propose a modified VBI algorithm. Simulation results are presented to show that the proposed algorithm can achieve better performance with lower complexity than the existing approaches.

Detecting topic-level influencers in large-scale scientific networks

Scientific networks play an increasingly important role in facilitating knowledge and technique diffusion. In such networks, highly influential nodes (scientists or literatures) are prone to stimulate other researchers in the generation of innovative ideas. The objective of this study is to detect topic-level influencers from a large collection of links between nodes and textual contents in scientific networks. For this purpose, we propose a sparse link topic model (SLTM) that introduces a “Spike and Slab” prior to achieve sparsity in node-topic distribution. Compared with previous approaches, our model assumes that a node usually focuses on several salient topics instead of a wide range of topics, which is useful in learning topic-level influencers in scientific networks. In addition, a collapsed variational Bayesian (CVB) inference algorithm is designed for large-scale applications. Our experiments are conducted on a large scientific collaboration network. The results reveal that the proposed model significantly improves the precision of topic-level detection. Our analysis also reflects that SLTM can explicitly model the sparse topical structure of each node in the network.

User Location Tracking in Massive MIMO Systems via Dynamic Variational Bayesian Inference

Accurate user location tracking is the key to enable location-based services and assist communications in 5 G networks. The massive multiple-input multiple-output (MIMO) technology employed in 5 G networks could potentially provide accurate user localization due to increased spectral efficiency and high directivity. In this paper, we propose an efficient user location tracking algorithm in massive MIMO systems. Firstly, we propose a temporal Markov group-sparse (TMGS) model based on a grid reference to capture the probabilistic temporal correlation and group sparsity of the massive MIMO channels jointly. Then we propose a dynamic variational Bayesian inference (D-VBI) algorithm to handle the TMGS priors under ill-conditioned measurement matrix in the location tracking problem. The proposed D-VBI can jointly recover the user's coarse location in the grid reference and refine the off-grid points to automatically learn the user's exact location to high accuracy. Moreover, the TMGS-based D-VBI algorithm can provide prior information about the user's next location and the possible arriving directions of the future channels to the consecutive time slot to improve the location tracking accuracy. Finally, we verify the superior performance of the proposed location tracking algorithm by extensive simulations.

Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise.

Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous noise. A few of the matrix decomposition methods have been extended for such multi-view data integration and pattern discovery while only a few methods were designed to consider the heterogeneity of noise in such multi-view data for data integration explicitly. To this end, in this article, we propose a joint matrix decomposition framework (BJMD), which models the heterogeneity of noise by the Gaussian distribution in a Bayesian framework. We develop two algorithms to solve this model: one is a variational Bayesian inference algorithm, which makes full use of the posterior distribution; and another is a maximum a posterior algorithm, which is more scalable and can be easily paralleled. Extensive experiments on synthetic and real-world datasets demonstrate that BJMD is superior or competitive to the state-of-the-art methods.

One-Bit Quantized Massive MIMO Detection Based on Variational Approximate Message Passing

One-bit quantization can significantly reduce the massive multiple-input and multiple-output (MIMO) system hardware complexity, but at the same time it also brings great challenges to the system algorithm design. Specifically, it is difficult to recover information from the highly distorted samples as well as to obtain accurate channel estimation without increasing the number of pilots. In this paper, a novel inference algorithm called variational approximate message passing (VAMP) for one-bit quantized massive MIMO receiver is developed, which attempts to exploit the advantages of both the variational Bayesian inference algorithm and the bilinear generalized approximated message passing algorithm to accomplish joint channel estimation and data detection in a closed form with first-order complexity. Asymptotic state evolution analysis indicates the fast convergence rate of VAMP and also provides a lower bound for the data detection error. Moreover, through extensive simulations, we show that VAMP can achieve excellent detection performance with low pilot overhead in a wide range of scenarios.

Sparse Variational Bayesian Inference for Water Pipeline Systems With Parameter Uncertainties

In this paper, multi-leak identification based on a transient wave model with physical parameter uncertainties for smart water supply systems is studied. We formulate multi-leak identification under uncertain parameters (such as friction factor, wave speed, and source–end discharge oscillation) as a sparse signal recovery problem with inaccurate parameters in the measurement matrix, by using spatial sampling in leak location space. A stochastic sparse variational Bayesian inference (SSVBI) algorithm to jointly learn the spatial samples, sparse signal, and uncertain parameters is proposed for multi-leak identification. In addition, we establish the convergence of the SSVBI algorithm to an approximate minimum means squared estimate. The proposed approach can be applied to an arbitrary number of leaks, and its computational complexity is insensitive to the number of leaks. This is a significant technical improvement over existing approaches. Finally, simulations show that the SSVBI-based joint learning of uncertain parameters and sparse model can achieve a huge performance gain over existing methods.

Bayesian Learning for Dynamic Feature Extraction With Application in Soft Sensing

Data-driven techniques such as principal component analysis (PCA) have been widely used to derive predictive models from historical data and applied for quality prediction in industry. Motivated by reducing data collinearity and extracting informative driving forces behind data, latent variable models are explored to facilitate the prediction by regressing data on a set of extracted features. In this paper, a novel learning strategy is proposed to build dynamic features under a full Bayesian framework, incorporating data information and prior knowledge of process dynamics. Unlike the traditional PCA that extracts features based on variances explained, in this paper, the latent features are extracted with the guidance of preferred velocities of nominal variations. By applying Bayesian learning algorithms, parameters are estimated with probability distributions accounting for corresponding uncertainties, and the number of latent features can be automatically determined by the variational Bayesian inference algorithm. The effectiveness and practicability of this Bayesian dynamic feature regression are demonstrated through simulated examples as well as an industrial case study.

Neuron's eye view: Inferring features of complex stimuli from neural responses.

Experiments that study neural encoding of stimuli at the level of individual neurons typically choose a small set of features present in the world—contrast and luminance for vision, pitch and intensity for sound—and assemble a stimulus set that systematically varies along these dimensions. Subsequent analysis of neural responses to these stimuli typically focuses on regression models, with experimenter-controlled features as predictors and spike counts or firing rates as responses. Unfortunately, this approach requires knowledge in advance about the relevant features coded by a given population of neurons. For domains as complex as social interaction or natural movement, however, the relevant feature space is poorly understood, and an arbitrary a priori choice of features may give rise to confirmation bias. Here, we present a Bayesian model for exploratory data analysis that is capable of automatically identifying the features present in unstructured stimuli based solely on neuronal responses. Our approach is unique within the class of latent state space models of neural activity in that it assumes that firing rates of neurons are sensitive to multiple discrete time-varying features tied to the stimulus, each of which has Markov (or semi-Markov) dynamics. That is, we are modeling neural activity as driven by multiple simultaneous stimulus features rather than intrinsic neural dynamics. We derive a fast variational Bayesian inference algorithm and show that it correctly recovers hidden features in synthetic data, as well as ground-truth stimulus features in a prototypical neural dataset. To demonstrate the utility of the algorithm, we also apply it to cluster neural responses and demonstrate successful recovery of features corresponding to monkeys and faces in the image set.

Sparse Bayesian linear regression with latent masking variables

Extracting a small number of relevant features for the task, i.e., feature selection, is often a crucial step in supervised learning problems. Sparse linear regression provides a fast and convenient option for feature selection, where regularization facilitates reducing the weight parameters of irrelevant features. However, the regularization also induces undesirable shrinkage in the weights of relevant features.Here, we propose Bayesian masking (BM) in order to resolve the trade-off problem between sparsity and shrinkage. Our strategy is not to directly impose any regularization on the weights; instead, BM introduces binary latent variables, called masking variables, into a regression model to keep the sparsity; each feature and sample has a binary variable whose value determines if the feature is masked or not at the sample. We derive a variational Bayesian inference algorithm for the augmented model based on the factorized information criterion (FIC), a recently-proposed asymptotic approximation of the marginal log-likelihood. We analyze the one-dimensional estimators of Lasso, automatic relevance determination (ARD), and BM, and thus show the superiority of BM in terms of the sparsity-shrinkage trade-off. Finally, we confirm our theoretical analyses through experiments and, demonstrate that BM achieves higher feature selection accuracy compared with Lasso and ARD.

One-bit compressed sensing reconstruction for block sparse signals

Data compression is crucial for resource-constrained signal acquisition and wireless transmission applications with limited data bandwidth. In such applications, wireless data transmission dominates the energy consumption, and the limited telemetry bandwidth could be overwhelmed by the large amount of data generated from multiple sensors. Conventional data compression techniques are computationally intensive, consume large silicon area and offset the energy benefits from reduced data transmission. Recently, compressed sensing (CS) has shown potential in achieving compression performance comparable to previous methods but it has simpler hardware. Especially, one-bit CS theory proves that the signs of compressed measurements contain sufficient information about signal reconstruction, gives that the signals are sparse or compressible in specific dictionaries, thus demonstrating its potential in energy-constrained signal recording and wireless transmission applications. However, the sparsity assumption is too restrictive in many actual scenarios, especially when it is difficult to seek sparse representation for signals. In this paper, a novel one-bit CS method is proposed to reconstruct the signals that are difficult to represent with traditional sparse models. It is capable of recovering signal with comparable compression ratio but avoiding the dictionary selection procedure.The proposed method consists of two parts. 1) The block sparse model is adopted to enforce the structured sparsity of the signals. It not only overcomes the drawbacks of conventional sparse models but also enhances the signal representation accuracy. 2) The probabilistic model of one-bit CS procedure is constructed. Because of the existence of logistic function in probabilistic model of one-bit CS, the Bayesian inference cannot be used to proceed, and the variational Bayesian inference algorithm is developed to reconstruct the original signals from one-bit measurements.Various experiments on different quantities of compressed measurements and iterations are carried out to evaluate the recovery performance of the proposed approach. The photoplethysmography (PPG) signals recorded from subject wrist (dorsal locations) by using PPG sensors built in a wristband are selected as the validation data because they are difficult to represent with traditional sparse dictionaries. The experimental results reveal that the proposed approach outperforms the state-of-the-art one-bit CS method in terms of both reconstruction accuracy and convergence rate.Compared with prior method on one-bit CS, the proposed method shows competitive or superior performance in three aspects. Firstly, by adopting the block sparse model, the proposed method improves the capability to compress signals that are difficult to represent with traditional sparse models, thus making it more practical for long term and real applications. Secondly, by embedding the statistical properties of the one-bit measurements into the recovery algorithm, the proposed method outperforms other one-bit CS methods in terms of both reconstruction performance and convergence speed. Finally, energy and computational efficiency of the proposed method make it an ideal candidate for resource-constrained, large scale, multiple channel signal acquisition and transmission applications.

Asymmetric biclustering with constrained von Mises-Fisher models

As a probability distribution on the high-dimensional sphere, the von Mises-Fisher (vMF) distribution is widely used for directional statistics and data analysis methods based on correlation. We consider a constrained vMF distribution for block modeling, which provides a probabilistic model of an asymmetric biclustering method that uses correlation as the similarity measure of data features. We derive the variational Bayesian inference algorithm for the mixture of the constrained vMF distributions. It is applied to a multivariate data visualization method implemented with enhanced parallel coordinate plots.

Bayesian Classifier for Sparsity-Promoting Feature Selection

A Bayesian classifier for sparsity-promoting feature selection is developed in this paper, where a set of nonlinear mappings for the original data is performed as a pre-processing step. The linear classification model with such mappings from the original input space to a nonlinear transformation space can not only construct the nonlinear classification boundary, but also realize the feature selection for the original data. A zero-mean Gaussian prior with Gamma precision and a finite approximation of Beta process prior are used to promote sparsity in the utilization of features and nonlinear mappings in our model, respectively. We derive the Variational Bayesian (VB) inference algorithm for the proposed linear classifier. Experimental results based on the synthetic data set, measured radar data set, high-dimensional gene expression data set, and several benchmark data sets demonstrate the aggressive and robust feature selection capability and comparable classification accuracy of our method comparing with some other existing classifiers.