Sparse Bayesian Learning Framework Research Articles

Sparse Bayesian Learning With Dynamic Filtering for Inference of Time-Varying Sparse Signals

Many signal processing applications require estimation of time-varying sparse signals, potentially with the knowledge of an imperfect dynamics model. In this paper, we propose an algorithm for dynamic filtering of time-varying sparse signals based on the sparse Bayesian learning (SBL) framework. The key idea underlying the algorithm, termed SBL-DF, is the incorporation of a signal prediction generated from a dynamics model and estimates of previous time steps into the hyperpriors of the SBL probability model. The proposed algorithm is online, robust to imperfect dynamics models (due to the propagation of dynamics information through higher-order statistics), robust to certain undesirable dictionary properties such as coherence (due to properties of the SBL framework), allows the use of arbitrary dynamics models, and requires the tuning of fewer parameters than many other dynamic filtering algorithms do. We also extend the fast marginal likelihood SBL inference procedure to the informative hyperprior setting to create a particularly efficient version of the SBL-DF algorithm. Numerical simulations show that SBL-DF converges much faster and to more accurate solutions than standard SBL and other dynamical filtering algorithms. In particular, we show that SBL-DF outperforms state of the art algorithms when the dictionary contains the challenging coherence and column scaling structure found in many practical applications.

Direct sequence spread spectrum communication narrowband interference cancelation in compressed domain

AbstractDue to the broad bandwidth nature of the direct sequence spread spectrum (DSSS) signal, the application of narrowband interference (NBI) cancelation algorithms based on the Nyquist‐Shannon sampling theory are limited by the high sampling rate. To solve the sampling and processing difficulty in the traditional DSSS communication interference cancelation method, we used the compressive sensing (CS) to reduce the sampling rate of the DSSS communication system, and further proposed a compressed domain NBI elimination method in this study. We proved that NBI with a certain bandwidth and DSSS signal are both sparse in specific dictionaries and proposed the corresponding sparse dictionary construction method. Detailed analysis of the compressed domain features of NBI and the DSSS signal demonstrated that it is possible to extract the strong NBI components from the compressed measurements. Two compressed domain NBI cancelation algorithms were proposed based on this idea. The extracted NBI components were filtered out from the compressed measurements directly in the first algorithm, and then we achieved the DSSS signal demodulation in compressed domain using the separated DSSS signal components with the orthogonal matching pursuit (OMP) algorithm. A proposed extraction filter block sparse Bayesian learning (EFBSBL) framework algorithm was utilized to estimate the NBI from the compressed measurements in the second algorithm. The estimated NBI was eliminated from the received signal, and then we recompressed the DSSS signal after NBI cancelation and realized the DSSS signal demodulation in compressed domain with the OMP algorithm. Simulation results support our theoretical analysis results and reveal that the two proposed algorithms are both effective for NBI cancelation and significant interference cancelation performance are achieved.

Scalable Mean-Field Sparse Bayesian Learning

Sparse Bayesian learning is a powerful framework for expressing compressed sensing problems in the language of Bayesian inference, but its principal algorithm—the variational relevance vector machine—is matrix-bound and scales poorly to larger problem sizes. Recent approaches to construct more computationally efficient methods of variational inference in the sparse Bayesian learning framework either scale poorly with the problem size or fail to minimize the correct variational objective. This work demonstrates an efficient matrix-free algorithm for sparse Bayesian learning that scales well to large problems, converges at a rate competetive with existing fast methods, and provably descends on the correct mean-field objective.

Sparse Bayesian learning for network structure reconstruction based on evolutionary game data

Network structure reconstruction is a fundamental problem for understanding, predicting and controlling the behaviors of complex networked systems and has received growing attention due to the potentials in a wide range of fields. Recent years have witnessed dramatic advances in the field of network structure reconstruction, especially the famous compressed sensing-based methods. However, some neglected disadvantages still exist in the existing works, such as the high measurement correlation existing in the solution matrix, reconstruction behaviors subject to model-based constraints and pure point estimate of the reconstruction results without credibility, which inevitably drag down the reconstruction performance. To address these problems, we propose a new framework of sparse Bayesian learning for network structure reconstruction based on evolutionary game data from the perspective of Bayesian and statistics. Specifically, we formulate the problem of network structure reconstruction as a Bayesian compressed sensing problem. Then, a hierarchical prior model is invoked for conjugated Bayesian inference to obtain the posterior distribution of the reconstructed result, including the reconstructed mean and covariance. Finally, the parameters in the reconstructed results are updated by an iterative estimation procedure. Results from numerical experiments have demonstrated applicability and efficiency of the proposed method and presented superiority over other reconstruction methods.

On the Convergence of a Bayesian Algorithm for Joint Dictionary Learning and Sparse Recovery

Dictionary learning (DL) is a well-researched problem, where the goal is to learn a dictionary from a finite set of noisy training signals, such that the training data admits a sparse representation over the dictionary. While several solutions are available in the literature, relatively little is known about their convergence and optimality properties. In this paper, we make progress on this problem by analyzing a Bayesian algorithm for DL. Specifically, we cast the DL problem into the sparse Bayesian learning (SBL) framework by imposing a hierarchical Gaussian prior on the sparse vectors. This allows us to simultaneously learn the dictionary as well as the parameters of the prior on the sparse vectors using the expectation-maximization algorithm. The dictionary update step turns out to be a non-convex optimization problem, and we present two solutions, namely, an alternating minimization (AM) procedure and an Armijo line search (ALS) method. We analytically show that the ALS procedure is globally convergent, and establish the stability of the solution by characterizing its limit points. Further, we prove the convergence and stability of the overall DL-SBL algorithm, and show that the minima of the cost function of the overall algorithm are achieved at sparse solutions. As a concrete example, we consider the application of the SBL-based DL algorithm to image denoising, and demonstrate the efficacy of the algorithm relative to existing DL algorithms.

Exploring the Laplace Prior in Radio Tomographic Imaging with Sparse Bayesian Learning towards the Robustness to Multipath Fading.

Radio tomographic imaging (RTI) is a technology for target localization by using radio frequency (RF) sensors in a wireless network. The change of the attenuation field caused by the target is represented by a shadowing image, which is then used to estimate the target’s position. The shadowing image can be reconstructed from the variation of the received signal strength (RSS) in the wireless network. However, due to the interference from multi-path fading, not all the RSS variations are reliable. If the unreliable RSS variations are used for image reconstruction, some artifacts will appear in the shadowing image, which may cause the target’s position being wrongly estimated. Due to the sparse property of the shadowing image, sparse Bayesian learning (SBL) can be employed for signal reconstruction. Aiming at enhancing the robustness to multipath fading, this paper explores the Laplace prior to characterize the shadowing image under the framework of SBL. Bayesian modeling, Bayesian inference and the fast algorithm are presented to achieve the maximum-a-posterior (MAP) solution. Finally, imaging, localization and tracking experiments from three different scenarios are conducted to validate the robustness to multipath fading. Meanwhile, the improved computational efficiency of using Laplace prior is validated in the localization-time experiment as well.

Learning the Structured Sparsity: 3-D Massive MIMO Channel Estimation and Adaptive Spatial Interpolation

This paper addresses the channel estimation problem for three-dimensional (3-D) massive multiple-input multiple-output (MIMO) systems, where the base station (BS) is equipped with a two-dimensional uniform planar array (UPA) to serve a number of user equipments (UEs). To implement with low hardware complexity, the number of available radio-frequency (RF) chains at BS is constrained to be much smaller than the number of antennas. The theoretical analysis of sparse property for 3-D massive MIMO channel reveals that there exists two kinds of sparse structures in beam-domain channel vector, namely the common sparsity structure among sub-arrays of UPA and block sparsity structure per sub-array. Based on this property, a novel structured sparse Bayesian learning framework is proposed to estimate the channel between BS and UE reliably. Moreover, an adaptive spatial interpolation scheme is proposed to further reduce the number of required RF chains at BS while maintaining the estimation performance. The simulation results show that the proposed scheme provides stable estimation performance for a variety of scenarios with different numbers of RF chains, transmit signal-to-noise ratios, Rician factors, and angular spreads, and outperforms the reference schemes significantly.

Seismic spectral sparse reflectivity inversion based on SBL-EM: experimental analysis and application

Abstract In this paper, we propose a new method of seismic spectral sparse reflectivity inversion that, for the first time, introduces Expectation-Maximization-based sparse Bayesian learning (SBL-EM) to enhance the accuracy of stratal reflectivity estimation based on the frequency spectrum of seismic reflection data. Compared with the widely applied sequential algorithm-based sparse Bayesian learning (SBL-SA), SBL-EM is more robust to data noise and, generally, can not only find a sparse solution with higher precision, but also yield a better lateral continuity along the final profile. To investigate the potential of SBL-EM in a seismic spectral sparse reflectivity inversion, we evaluate the inversion results by comparing them with those of a SBL-SA-based approach in multiple aspects, including the sensitivity to different frequency bands, the robustness to data noise, the lateral continuity of the final profiles and so on. Furthermore, we apply the mean square error (MSE), residual variance (RV) of seismograms and residual energy (RE) between the frequency spectra of the true and inverted reflectivity model to highlight the advantages of the proposed method over the SBL-SA-based approach in terms of spectral sparse reflectivity inversion within a sparse Bayesian learning framework. Multiple examples, including both numerical and field experiments, are carried out to validate the proposed method.

Improved Bayesian ISAR Imaging by Learning the Local Structures of the Target Scene

In inverse synthetic aperture radar (ISAR) imaging, some weak scatterers might be missing in the results obtained by conventional compressive sensing (CS)-based methods. This paper proposes a novel method to preserve the weak scatterers by exploiting the local structures of the target scene statistically in the imaging process. Particularly, we partition the target scene into many overlapping patches, and the local structure in each patch is represented by the covariance matrix, which is learned from the measurements. Simultaneously, the Dirichlet process (DP) priors are exploited to accomplish the clustering of the local structures. Under the sparse Bayesian learning framework, the scatterers are modeled hierarchically, and we tailor the variational Bayesian inference (VBI) to incorporate the structural information into the reconstruction of the target image, where the weak scatterers can be preserved. Comprehensive experiments based on synthetic data, electromagnetic (EM) simulated data, and real data have validated the performance of the proposed method.

BSBL-Based DOA and Polarization Estimation with Linear Spatially Separated Polarization Sensitive Array

The problem of multiple incident signals’ direction of arrival (DOA) and polarization estimation with linear spatially separated polarization sensitive array (SS-PSA) is investigated in sparse Bayesian learning (SBL) framework. SS-PSA is widely studied due to its low mutual coupling compared with the spatially collocated polarization sensitive array. On the one hand, the sparse representation of data model of proposed array can be expressed skillfully as a block-sparse representation. On the other hand, block sparse Bayesian learning (BSBL) algorithm has excellent performance in recovering the block-sparse signals. Therefore, in this paper, BSBL algorithm is extended to SS-PSA for DOA and polarization estimation. Firstly, a sparse representation model without parameters coupling is established, where the sparse coefficient vector is a block-sparse signal. Secondly, the expectation–maximization method in BSBL framework, i.e., BSBL-EM algorithm, is proposed to recover the block-sparse signal. Lastly, angles estimation can be obtained from the recovered sparse signal according to the intra-block correlation. Simulation results demonstrate that the BSBL-EM algorithm used in DOA and polarization estimation with SS-PSA exhibits better performance compared with Block Orthogonal Matching Pursuit algorithm and Group Basis Pursuit.

A new surrogate modeling method combining polynomial chaos expansion and Gaussian kernel in a sparse Bayesian learning framework

SummarySurrogate modeling techniques have been increasingly developed for optimization and uncertainty quantification problems in many engineering fields. The development of surrogates requires modeling high‐dimensional and nonsmooth functions with limited information. To this end, the hybrid surrogate modeling method, where different surrogate models are combined, offers an effective solution. In this paper, a new hybrid modeling technique is proposed by combining polynomial chaos expansion and kernel function in a sparse Bayesian learning framework. The proposed hybrid model possesses both the global characteristic advantage of polynomial chaos expansion and the local characteristic advantage of the Gaussian kernel. The parameterized priors are utilized to encourage the sparsity of the model. Moreover, an optimization algorithm aiming at maximizing Bayesian evidence is proposed for parameter optimization. To assess the performance of the proposed method, a detailed comparison is made with the well‐established PC‐Kriging technique. The results show that the proposed method is superior in terms of accuracy and robustness.

Discrimination of SSVEP responses using a kernel based approach.

Brain Computer Interfaces based on Steady State Visual Evoked Potentials have gained increased attention due to their low training requirements and higher information transfer rates. In this work, a method based on sparse kernel machines is proposed for the discrimination of Steady State Visual Evoked Potentials responses. More specifically, a new kernel based on Partial Least Squares is introduced to describe the similarities between EEG trials, while the estimation of regression weights is performed using the Sparse Bayesian Learning framework. The experimental results obtained on two benchmarking datasets, have shown that the proposed method provides significantly better performance compared to state of the art approaches of the related literature.

An efficient and robust adaptive sampling method for polynomial chaos expansion in sparse Bayesian learning framework

Sparse polynomial chaos expansion has been widely used to tackle problems of function approximation in the field of uncertain quantification. The accuracy of PCE depends on how to construct the experimental design. Therefore, adaptive sampling methods of designs of experiment are raised. Classic designs of experiment for PCE are based on least-square minimization techniques, where the design space is only defined by the inputs without involving the responses of the system. To overcome this limitation, a novel adaptive sampling method is introduced in sparse Bayesian learning framework. The design point is enriched sequentially by maximizing a generalized expectation of loss function criterion which allows an effective use of all the information available, on which two adaptive strategies are derived to get a balance between the global exploration and the local exposition via the error information from the previous iteration. The numerical results show that the proposed method is superior to classic design of experiment in terms of efficiency and robustness.

A novel reconstruction approach for high-resolution demodulation spectrum with group-sparsity characteristics

There exists a distinct envelope modulation of the radiated noise of underwater acoustic target because of the propeller beat. These parameters related with envelop modulation imply a wealth of physical information such as the speed of the propeller. Therefore, the detection of envelope modulation on noise (DEMON) characteristics is critical for classifying and recognizing the underwater acoustic target. In this study, a novel high-resolution reconstruction approach of DEMON is proposed by exploiting its group sparsity across subbands compared with the drawback in Fourier transform (FT)-based DEMON. On the one hand, the estimation of sparse DEMON is converted into solving underdetermined equation in the inverse Fourier basis by exploiting the sparsity of DEMON; on the other hand, the reconstruction method can be improved and developed using the exploitation of the group sparsity across subbands. Unlike the non-sparse FT-based DEMON, which requires a further detection of line spectrum for classification or recognition, our proposed DEMON is sparse and directly provides the line spectrum. It effectively avoids the threshold choice in the detection and artificial interference in the feature extraction. Furthermore, the proposed method is developed in the non-parametrical sparse Bayesian learning framework, so it has the capability of learning the sparsity of DEMON automatically.

Narrowband Interference Separation for Synthetic Aperture Radar via Sensing Matrix Optimization-Based Block Sparse Bayesian Learning

High-resolution synthetic aperture radar (SAR) operating with a large bandwidth is subject to impacts from various kinds of narrowband interference (NBI) in complex electromagnetic environments. Recently, many radio frequency interference (RFI) suppression approaches for SAR based on sparse recovery have been proposed and demonstrated to outperform traditional ones in preserving the signal of interest (SOI) while suppressing the interference by exploiting their intrinsic structures. In particular, the joint recovery strategy of SOI and NBI with a cascaded dictionary, which eliminates the steps of NBI reconstruction and time-domain cancellation, can further reduce unnecessary system complexity. However, these sparsity-based approaches hardly work effectively for signals from an extended target or NBI with a certain bandwidth, since neither of them is sparse in a prescient domain. Moreover, sub-dictionaries corresponding to different components in the cascaded matrix are not strictly independent, which severely limits the performance of separated reconstruction. In this paper, we present an enhanced NBI separation algorithm for SAR via sensing matrix optimization-based block sparse Bayesian learning (SMO-BSBL) to solve these problems above. First, we extend the block sparse Bayesian learning framework to a complex-valued domain for the convenience of radar signal processing with lower computation complexity and modify it to deal with the separation problem of NBI in the contaminated echo. For the sake of improving the separated reconstruction performance, we propose a new block coherence measure by defining the external and internal block structure, which is used for optimizing the observation matrix. The optimized observation matrix is then employed to reconstruct SOI and NBI simultaneously under the modified BSBL framework, given a known and fixed cascaded dictionary. Numerical simulation experiments and comparison results demonstrate that the proposed SMO-BSBL is effective and superior to other advanced algorithms in NBI suppression for SAR.

Underdetermined DOA estimation using coprime array via multiple measurement sparse Bayesian learning

Underdetermined direction of arrival (DOA) estimation with coprime array is discussed in the framework of multiple measurement sparse Bayesian learning (MSBL). Exploiting the extended difference coarray, a larger number of degrees of freedom can be obtained for locating more sources than sensors. A linear operation and a prewhitening procedure are incorporated into the sparse signal recovery model to eliminate the influence of noise. Then, MSBL employs an empirical Bayesian strategy to resolve $$l_{0}$$ minimization problem. Simulation results show the superiority of the MSBL algorithm in underdetermined DOA detection performance, resolution ability and estimation accuracy when there are multiple measurement vectors for on-grid and off-grid sources, respectively.

Massive MIMO Channel Estimation Over the mmWave Systems Through Parameters Learning

In this letter, we formulate an off-grid channel model to characterize spatial sample mismatching in the discrete Fourier transform (DFT) based massive multiple-input-multiple-output (MIMO) channel estimation over the millimeter-wave (mmWave) band. Then, we decompose the off-grid mmWave massive MIMO channel estimation into the learning of model parameters and virtual channel estimation. Specifically, an expectation maximization (EM) based sparse Bayesian learning framework is first developed to learn the model parameters, such as bias parameters and spatial signatures, with unknown noise. With the learned model parameters, we resort to the linear minimum mean square error method to estimate the instantaneous virtual channel with less pilot overhead. Finally, we corroborate the validity of the proposed method through numerical simulations.

Sparse Bayesian Learning for the Time-Varying Massive MIMO Channels: Acquisition and Tracking

The low-rank property of the channel covariances can be adopted to reduce the overhead of the channel training in massive MIMO systems. In this paper, with the help of the virtual channel representation, we apply such property to both time-division duplex and frequency-division duplex systems, where the time-varying channel scenarios are considered. First, we formulate the dynamic massive MIMO channel as one sparse signal model. Then, an expectation maximization-based sparse Bayesian learning framework is developed to learn the model parameters of the sparse virtual channel. Specifically, the Kalman filter (KF) and the Rauch–Tung–Striebel smoother are utilized to track the model parameters of the uplink (UL) spatial sparse channel in the expectation step. During the maximization step, a fixed-point theorem-based algorithm and a low-complex searching method are constructed to recover the temporal varying characteristics and the spatial signatures, respectively. With the angle reciprocity, we recover the downlink (DL) model parameters from the UL ones. After that, the KF with the reduced dimension is adopt to fully exploit the channel temporal correlations to enhance the DL/UL virtual channel tracking accuracy. A monitoring scheme is also designed to detect the change of model parameters and trigger the relearning process. Finally, we demonstrate the efficacy of the proposed schemes through the numerical simulations.

Joint Channel Estimation and Impulsive Noise Mitigation Method for OFDM Systems Using Sparse Bayesian Learning

The impulsive noise can deteriorate sharply the performance of orthogonal frequency division multiplexing (OFDM) systems. In this paper, we propose a novel joint channel impulse response estimation and impulsive noise mitigation algorithm based on compressed sensing theory. In this algorithm, both the channel impulse response and the impulsive noise are treated as a joint sparse vector. Then, the sparse Bayesian learning framework is adopted to jointly estimate the channel impulse response, the impulsive noise, and the data symbols, in which the data symbols are regarded as unknown parameters. The Cramer–Rao Bound is derived for the benchmark. Unlike the previous impulsive noise mitigation methods, the proposed algorithm utilizes all subcarriers without any a priori information of the channel and impulsive noise. The simulation results show that the proposed algorithm achieves significant performance improvement on the channel estimation and bit error rate performance.

Sparse Bayesian Approach for DOD and DOA Estimation With Bistatic MIMO Radar

This study addresses the problem of joint direction-of-departure (DOD) and direction-of-arrival (DOA) estimation with bistatic multiple-input multiple-output (MIMO) radar. To the best of our knowledge, a limited number of sparse Bayesian learning (SBL)-based methods exist that can be applied to joint DOD and DOA estimation. This is because of the heavy computational load and strong correlation between the nearby basis. To overcome these challenges, we present a new coarse non-uniformly sampled 2D grid and propose an improved SBL-based method for joint estimation of the DOD and DOA in MIMO radar. With the new grid, the computational load can be significantly reduced, and the nearby 2D grid points can provide a low correlation basis. To handle the modeling error derived from the coarse grid, we also introduce a modified linear approximation method into the SBL framework in which the locations of grid points are considered as adjustable parameters, and the grid points can be updated recursively. Finally, a block majorization-minimization algorithm is applied to perform Bayesian inference. Experimental results indicate that our method can improve the joint DOD and DOA estimation performance, particularly in the case of low signal-noise-ratio, limited snapshots, or correlated signals.