Sparse Bayesian Inference Research Articles

A Sparse Bayesian Learning Method for Structural Equation Model-Based Gene Regulatory Network Inference

Gene regulatory networks (GRNs) are underlying networks identified by interactive relationships between genes. Reconstructing GRNs from massive genetic data is important for understanding gene functions and biological mechanism, and can provide effective service for medical treatment and genetic research. A series of artificial intelligence based methods have been proposed to infer GRNs from both gene expression data and genetic perturbations. The accuracy of such algorithms can be better than those models that just consider gene expression data. A structural equation model (SEM), which provides a systematic framework integrating both types of gene data conveniently, is a commonly used model for GRN inference. Considering the sparsity of GRNs, in this paper, we develop a novel sparse Bayesian inference algorithm based on Normal-Equation-Gamma (NEG) type hierarchical prior (BaNEG) to infer GRNs modeled with SEMs more accurately. First, we reparameterize an SEM as a linear type model by integrating the endogenous and exogenous variables; Then, a Bayesian adaptive lasso with a three-level NEG prior is applied to deduce the corresponding posterior mode and estimate the parameters. Simulations on synthetic data are run to compare the performance of BaNEG to some state-of-the-art algorithms, the results demonstrate that the proposed algorithm visibly outperforms the others. What’s more, BaNEG is applied to infer underlying GRNs from a real data set composed of 47 yeast genes from Saccharomyces cerevisiae to discover potential relationships between genes.

Infrared small target tracking algorithm based on temporal-spatial structure sparse Bayesian estimation

The tracking of infrared small target may be unstable due to the interference of background clutter and imaging noise. Moreover, the changing target appearance could degrade the tracking robustness. How to estimate the target spatial-temporal structure is the key to enhance the tracking stability for appearance changing small target under heavy clutter. This paper proposes a robust target tracking algorithm based on sparse representation and Bayesian inference that can estimate and predict the target spatial-temporal structure. Firstly, the small target signal is sparsely decomposed on the generalized Gaussian target over-complete dictionary (GGTOD). In this way the spatial structure information of small target is extracted from the noised and clutter contaminated infrared image. Then, according to the long term observations the long term temporal-spatial structure distribution of the target is established. Meanwhile the short term target temporal-spatial structure distribution is built according to the short term observations. Finally, the long term structure distribution and the short term structure distribution are combined by Bayesian inference to estimate and predict the target temporal-spatial structure in the next frame. By estimating the temporal change of target spatial structure, the proposed method achieves outstanding adaptability to the changing small target and robustness to clutter disturbance. Experiments on various infrared sequences show that the proposed method not only can estimate and predict the temporal-spatial structure of small target accurately but also can track the appearance changing small target stably under the inference of heavy clutter and noise.

T–S Fuzzy Model Identification with Sparse Bayesian Techniques

This paper introduces a novel method for fuzzy modeling based on sparse Bayesian techniques. The sparse representation problems in the Takagi–Sugeno (T–S) fuzzy system identification are studied, which is to establish a T–S fuzzy system with a adaptive number of fuzzy rules and simultaneously have a minimal number of nonzero consequent parameters. The proposed method is called sparse Bayesian fuzzy inference systems (B-sparseFIS). There are two main procedures in the paper. Firstly, initial fuzzy rule of antecedent part is extracted automatically by an AP clustering method. By using the algorithm of adaptive block orthogonal matching pursuit, the number of rules is computed statistically then the main important fuzzy rules can be selected. In the algorithm, the redundant rules are eliminated for better model accuracy and generalization performance; secondly, an adaptive B-sparseFIS is exploited. The consequence of fuzzy system is identified and simplified with sparse Bayesian techniques such that many consequent parameters will approximate to zero. Four examples are provided to illustrate the effectiveness of the proposed algorithm. Furthermore, the performances of the algorithm are validated through the results of statistical analyses including parameter estimate error, MSE, NRMSE, etc.

Structural damage identification using improved Jaya algorithm based on sparse regularization and Bayesian inference

Structural damage identification can be considered as an optimization problem, by defining an appropriate objective function relevant to structural parameters to be identified with optimization techniques. This paper proposes a new heuristic algorithm, named improved Jaya (I-Jaya) algorithm, for structural damage identification with the modified objective function based on sparse regularization and Bayesian inference. To improve the global optimization capacity and robustness of the original Jaya algorithm, a clustering strategy is employed to replace solutions with low-quality objective values and a new updated equation is used for the best-so-far solution. The objective function that is sensitive and robust for effective and reliable damage identification is developed through sparse regularization and Bayesian inference and used for optimization analysis with the proposed I-Jaya algorithm. Benchmark tests are conducted to verify the improvement in the developed algorithm. Numerical studies on a truss structure and experimental validations on an experimental reinforced concrete bridge model are performed to verify the developed approach. A limited quantity of modal data, which is distinctively less than the number of unknown system parameters, are used for structural damage identification. Significant measurement noise effect and modelling errors are considered. Damage identification results demonstrate that the proposed method based on the I-Jaya algorithm and the modified objective function based on sparse regularization and Bayesian inference can provide accurate and reliable damage identification, indicating the proposed method is a promising approach for structural damage detection using data with significant uncertainties and limited measurement information.

Three-Dimensional Imaging Method for Array ISAR Based on Sparse Bayesian Inference.

The problem of synthesis scatterers in inverse synthetic aperture radar (ISAR) make it difficult to realize high-resolution three-dimensional (3D) imaging. Radar array provides an available solution to this problem, but the resolution is restricted by limited aperture size and number of antennas, leading to deterioration of the 3D imaging performance. To solve these problems, we propose a novel 3D imaging method with an array ISAR system based on sparse Bayesian inference. First, the 3D imaging model using a sparse linear array is introduced. Then the elastic net estimation and Bayesian information criterion are introduced to fulfill model order selection automatically. Finally, the sparse Bayesian inference is adopted to realize super-resolution imaging and to get the 3D image of target of interest. The proposed method is used to process real radar data of a Ku band array ISAR system. The results show that the proposed method can effectively solve the problem of synthesis scatterers and realize super-resolution 3D imaging, which verify the practicality of our proposed method.

DOA Estimation for a Mixture of Uncorrelated and Coherent Sources Based on Hierarchical Sparse Bayesian Inference with a Gauss-Exp-Chi2 Prior

Direction of arrival (DOA) estimation algorithms based on sparse Bayesian inference (SBI) can effectively estimate coherent sources without recurring to extra decorrelation techniques, and their estimation performance is highly dependent on the selection of sparse prior. Specifically, the specified sparse prior is expected to concentrate its mass on the zero and distribute with heavy tails; otherwise, these algorithms may suffer from performance degradation. In this paper, we introduce a new sparse-encouraging prior, referred to as “Gauss-Exp-Chi2” prior, and develop an efficient DOA estimation algorithm for a mixture of uncorrelated and coherent sources under a hierarchical SBI framework. The Gauss-Exp-Chi2 prior distribution exhibits a sharp peak at the origin and heavy tails, and this property makes it an appropriate prior to encourage sparse solutions. A three-layer hierarchical sparse Bayesian model is established. Then, by exploiting variational Bayesian approximation, the model parameters are estimated by alternately updating until Kullback-Leibler (KL) divergence between the true posterior and the variational approximation becomes zero. By constructing the source power spectra with the estimated model parameters, the number and locations of the highest peaks are extracted to obtain source number and DOA estimates. In addition, some implementation details for algorithm optimization are discussed and the Cramér-Rao bound (CRB) of DOA estimation is derived. Simulation results demonstrate the effectiveness and favorable performance of the proposed algorithm as compared with the state-of-the-art sparse Bayesian algorithms.

Sparse Method for Direction of Arrival Estimation Using Denoised Fourth-Order Cumulants Vector.

Fourth-order cumulants (FOCs) vector-based direction of arrival (DOA) estimation methods of non-Gaussian sources may suffer from poor performance for limited snapshots or difficulty in setting parameters. In this paper, a novel FOCs vector-based sparse DOA estimation method is proposed. Firstly, by utilizing the concept of a fourth-order difference co-array (FODCA), an advanced FOCs vector denoising or dimension reduction procedure is presented for arbitrary array geometries. Then, a novel single measurement vector (SMV) model is established by the denoised FOCs vector, and efficiently solved by an off-grid sparse Bayesian inference (OGSBI) method. The estimation errors of FOCs are integrated in the SMV model, and are approximately estimated in a simple way. A necessary condition regarding the number of identifiable sources of our method is presented that, in order to uniquely identify all sources, the number of sources K must fulfill . The proposed method suits any geometry, does not need prior knowledge of the number of sources, is insensitive to associated parameters, and has maximum identifiability , where M is the number of sensors in the array. Numerical simulations illustrate the superior performance of the proposed method.

Underdetermined Wideband DOA Estimation for Off-Grid Sources with Coprime Array Using Sparse Bayesian Learning.

Sparse Bayesian learning (SBL) is applied to the coprime array for underdetermined wideband direction of arrival (DOA) estimation. Using the augmented covariance matrix, the coprime array can achieve a higher number of degrees of freedom (DOFs) to resolve more sources than the number of physical sensors. The sparse-based DOA estimation can deteriorate the detection and estimation performance because the sources may be off the search grid no matter how fine the grid is. This dictionary mismatch problem can be well resolved by the SBL using fixed point updates. The SBL can automatically choose sparsity and approximately resolve the non-convex optimizaton problem. Numerical simulations are conducted to validate the effectiveness of the underdetermined wideband DOA estimation via SBL based on coprime array. It is clear that SBL can obtain good performance in detection and estimation compared to least absolute shrinkage and selection operator (LASSO), simultaneous orthogonal matching pursuit least squares (SOMP-LS) , simultaneous orthogonal matching pursuit total least squares (SOMP-TLS) and off-grid sparse Bayesian inference (OGSBI).

Sparse Variational Bayesian Inference for Water Pipeline Systems With Parameter Uncertainties

In this paper, multi-leak identification based on a transient wave model with physical parameter uncertainties for smart water supply systems is studied. We formulate multi-leak identification under uncertain parameters (such as friction factor, wave speed, and source–end discharge oscillation) as a sparse signal recovery problem with inaccurate parameters in the measurement matrix, by using spatial sampling in leak location space. A stochastic sparse variational Bayesian inference (SSVBI) algorithm to jointly learn the spatial samples, sparse signal, and uncertain parameters is proposed for multi-leak identification. In addition, we establish the convergence of the SSVBI algorithm to an approximate minimum means squared estimate. The proposed approach can be applied to an arbitrary number of leaks, and its computational complexity is insensitive to the number of leaks. This is a significant technical improvement over existing approaches. Finally, simulations show that the SSVBI-based joint learning of uncertain parameters and sparse model can achieve a huge performance gain over existing methods.

Direction of Arrival Estimation in Linear Arrays With Intersubarray Displacement Errors Using Sparse Bayesian Inference

Linear arrays that are composed of multiple well-calibrated subarrays are widely used to improve the angular resolution in array signal processing. However, intersubarray displacement errors (IDEs) that are ubiquitous in practical applications and degrade the performance of the direction of arrival (DOA) estimation. To address this problem, we build a DOA estimation model that considers manifold mismatch due to the IDEs. The model is solved in the framework of sparse Bayesian inference with the variational inference methodology. The root-mean-square-errors of the DOA estimates of the proposed method are smaller in comparison with the sparsity-cognizant total least-squares approach. The improved method is applied to partially calibrate a virtual linear array that is constructed via the extended towed array measurement method in the case of velocity mismatch. We assume that the IDEs of a virtual linear array lead to a biased DOA estimate. Simulations and experimental results demonstrate the validity of the proposed method.

High-Resolution Electromagnetic Vortex Imaging Based on Sparse Bayesian Learning

The vortex electromagnetic (EM) wave carrying orbital angular momentum (OAM) has been found great potential to improve radar imaging performance. However, to achieve high-resolution image, a large number of OAM modes should be applied in the existing EM vortex imaging methods, which seriously limits the improvement of resolution in practice. The orthogonality of OAM eigenmodes enables the reconstruction of targets with limited measurements. Thus, to improve the imaging resolution and reduce the number of measurements, the sparse Bayesian learning (SBL) is introduced to reconstruct the targets for EM vortex imaging. The sparse representation model for EM vortex imaging is first derived using the stepped frequency signal. Subsequently, based on the SBL framework, the enhanced SBL (ESBL) and variational sparse Bayesian inference (VSBI) are utilized to reconstruct the target. Simulation results show that the SBL-based reconstruction algorithms can achieve much higher resolution than conventional fast Fourier transform-based methods. The imaging results considering the influences of noise and radiation pattern optimization are presented and analyzed. Moreover, the off-grid imaging model is also derived and the target is reconstructed based on a modified VSBI algorithm. Finally, a practical application processing the real-world data from a proof-of-concept experiment in an anechoic chamber is provided.

Holographic SAR Tomography Image Reconstruction by Combination of Adaptive Imaging and Sparse Bayesian Inference

In this letter, we propose an imaging algorithm for the holographic synthetic aperture radar tomography in the circumstance of sparse and nonuniform elevation circular passes. Considering the anisotropic behavior of scatterers and the off-grid effect of sparse signal recovery, the algorithm combines the 2-D adaptive imaging method for circular SAR and the sparse Bayesian inference-based method for elevation reconstruction. For each circular pass, the azimuth-range 2-D image can be formed by the adaptive imaging method, which depends on the preretrieved maximum azimuth response angle and the azimuth persistence width. To deal with the off-grid effect in elevation reconstruction, which is caused by the deviation between the true scatterers and the discretized imaging grids, the off-grid sparse Bayesian inference method jointly estimates the scatterers and elevation off-grid error by applying their hierarchical priors. Compared with the conventional compressive sensing method that does not concern the off-grid effect, the proposed algorithm can provide more accurate 3-D reconstruction for pointlike targets, which is verified by the real-data experiments.

RFI localization in synthetic aperture interferometric radiometers based on sparse Bayesian inference

ABSTRACTThe presence of radio frequency interference (RFI) sources emitting in the L-band, which is reserved for passive measurements by International Telecommunications Union (ITU) regulations, has seriously deteriorated the data quality of many brightness temperature (BT) snapshots in the Soil Moisture and Ocean Salinity (SMOS) project. In order to obviate the Gibbs-like contamination on the BT maps, one effective way is to locate the positions of RFI sources and switch them off. This article discusses a new method for RFI localization that is tailored to the scenario of synthetic aperture interferometric radiometry. The novel aspect lies in addressing the problem of RFI localization from a probabilistic viewpoint. By introducing the sparsity of RFI distribution in the spatial domain as a priori knowledge, we have employed the sparse Bayesian inference (SBI) strategy to estimate the locations of RFI sources. In addition, we have also tested the proposed method using numerical simulations and actual SMOS data. The results indicate that the proposed method has advantages in both accuracy and resolution of RFI source localization over the conventional direction-of-arrival (DOA) methods used in the beamforming technique.

An analytical subspace-based robust sparse Bayesian inference estimator for off-grid TDOA localization

To locate multiple sources through time-difference-of-arrival (TDOA) measurements, existing algorithms generally require the matching relationship between measurements and the corresponding sources. In this paper, we propose a new Bayesian learning method for cases where the matching relationship is not given and off-grid error is considered. To achieve this, first we propose a new basis generator, which casts the localization problem within the Bayesian learning scheme. Then, we modify the existing sparse Bayesian inference (SBI) approaches and explore the priors on fingerprinting weights, resulting in two intermediate algorithms. On these foundations, a subspace-based robust SBI (SRSBI) algorithm is proposed as the core of this paper. SRSBI is highlighted by its ability to work free from iteration when estimating off-grid targets. What' more, SRSBI offers considerable robustness against initial guesses of hyper-parameters. Numerical simulations demonstrate the superiority of SRSBI in terms of accuracy, robustness and speed, compared to the other reported ones.

A novel Bayesian inference soft sensor for real‐time statistic learning modeling for industrial polypropylene melt index prediction

ABSTRACTA novel real‐time soft sensor based on a sparse Bayesian probabilistic inference framework is proposed for the prediction of melt index in industrial polypropylene process. The Bayesian framework consists of a relevance vector machine for predicting melt index and a particle filtering algorithm for soft sensor optimization. An online correcting strategy is also developed for improving the performance of real‐time melt index prediction. The method takes advantages of the probabilistic inference and using prior statistical knowledge of polymerization process. Developed soft sensors are validated with ten public databases from UCI machine learning repository and real data from industrial polypropylene process. Experimental results indicate the effectiveness of proposed method and show the improvement in both prediction precision and generalization capability compared with the reported models in literatures. © 2017 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2017, 134, 45384.

Sparse Bayesian Inference and the Temperature Structure of the Solar Corona

Abstract Measuring the temperature structure of the solar atmosphere is critical to understanding how it is heated to high temperatures. Unfortunately, the temperature of the upper atmosphere cannot be observed directly, but must be inferred from spectrally resolved observations of individual emission lines that span a wide range of temperatures. Such observations are “inverted” to determine the distribution of plasma temperatures along the line of sight. This inversion is ill posed and, in the absence of regularization, tends to produce wildly oscillatory solutions. We introduce the application of sparse Bayesian inference to the problem of inferring the temperature structure of the solar corona. Within a Bayesian framework a preference for solutions that utilize a minimum number of basis functions can be encoded into the prior and many ad hoc assumptions can be avoided. We demonstrate the efficacy of the Bayesian approach by considering a test library of 40 assumed temperature distributions.

Off-grid DOA estimation in real spherical harmonics domain using sparse Bayesian inference

When true targets do not locate exactly on discretized sampling grids, sparse reconstruction methods cannot estimate direction-of-arrival (DOA) accurately due to angular differences. This DOA estimation problem can be solved by off-grid sparse Bayesian inference (OGSBI). However, this method brings high computational complexity when estimating 2-D off-grid DOAs with spherical arrays. In order to solve 2-D off-grid DOA estimation, we adopt two steps to reduce computations and meanwhile maintain good performance. First, a real-valued off-grid model is constructed in real spherical harmonics domain. It models angular differences by exploiting the multivariable Taylor expansion to construct a matching matrix. Second, a projection-based basis selection sparse Bayesian learning combining with least squares (PSBL-LS) algorithm is proposed to estimate 2-D off-grid DOAs. This method reduces computations in learning both posterior of sparse signals and angular differences. The PSBL-LS uses the potential basis functions selected from the matching matrix to learning the posterior distribution of sparse signals. At the same time, the angular differences are estimated by least squares method based on the selected basis functions. Simulations show our proposed method improves accuracy and reduces computational load.

Measurement Matrix Optimization and Mismatch Problem Compensation for DLSLA 3-D SAR Cross-Track Reconstruction.

With a short linear array configured in the cross-track direction, downward looking sparse linear array three-dimensional synthetic aperture radar (DLSLA 3-D SAR) can obtain the 3-D image of an imaging scene. To improve the cross-track resolution, sparse recovery methods have been investigated in recent years. In the compressive sensing (CS) framework, the reconstruction performance depends on the property of measurement matrix. This paper concerns the technique to optimize the measurement matrix and deal with the mismatch problem of measurement matrix caused by the off-grid scatterers. In the model of cross-track reconstruction, the measurement matrix is mainly affected by the configuration of antenna phase centers (APC), thus, two mutual coherence based criteria are proposed to optimize the configuration of APCs. On the other hand, to compensate the mismatch problem of the measurement matrix, the sparse Bayesian inference based method is introduced into the cross-track reconstruction by jointly estimate the scatterers and the off-grid error. Experiments demonstrate the performance of the proposed APCs’ configuration schemes and the proposed cross-track reconstruction method.

Sparse Bayesian Inference of White Matter Fiber Orientations from Compressed Multi-resolution Diffusion MRI.

The RubiX [1] algorithm combines high SNR characteristics of low resolution data with high spacial specificity of high resolution data, to extract microstructural tissue parameters from diffusion MRI. In this paper we focus on estimating crossing fiber orientations and introduce sparsity to the RubiX algorithm, making it suitable for reconstruction from compressed (under-sampled) data. We propose a sparse Bayesian algorithm for estimation of fiber orientations and volume fractions from compressed diffusion MRI. The data at high resolution is modeled using a parametric spherical deconvolution approach and represented using a dictionary created with the exponential decay components along different possible directions. Volume fractions of fibers along these orientations define the dictionary weights. The data at low resolution is modeled using a spatial partial volume representation. The proposed dictionary representation and sparsity priors consider the dependence between fiber orientations and the spatial redundancy in data representation. Our method exploits the sparsity of fiber orientations, therefore facilitating inference from under-sampled data. Experimental results show improved accuracy and decreased uncertainty in fiber orientation estimates. For under-sampled data, the proposed method is also shown to produce more robust estimates of fiber orientations.

Fast Markov chain Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors

Sparsity has become a key concept for solving of high-dimensional inverse problems using variational regularization techniques. Recently, using similar sparsity-constraints in the Bayesian framework for inverse problems by encoding them in the prior distribution has attracted attention. Important questions about the relation between regularization theory and Bayesian inference still need to be addressed when using sparsity promoting inversion. A practical obstacle for these examinations is the lack of fast posterior sampling algorithms for sparse, high-dimensional Bayesian inversion. Accessing the full range of Bayesian inference methods requires being able to draw samples from the posterior probability distribution in a fast and efficient way. This is usually done using Markov chain Monte Carlo (MCMC) sampling algorithms. In this paper, we develop and examine a new implementation of a single component Gibbs MCMC sampler for sparse priors relying on L1-norms. We demonstrate that the efficiency of our Gibbs sampler increases when the level of sparsity or the dimension of the unknowns is increased. This property is contrary to the properties of the most commonly applied Metropolis–Hastings (MH) sampling schemes. We demonstrate that the efficiency of MH schemes for L1-type priors dramatically decreases when the level of sparsity or the dimension of the unknowns is increased. Practically, Bayesian inversion for L1-type priors using MH samplers is not feasible at all. As this is commonly believed to be an intrinsic feature of MCMC sampling, the performance of our Gibbs sampler also challenges common beliefs about the applicability of sample based Bayesian inference.