Variational Bayesian Algorithm Research Articles

An adaptive convolutional neural network based on transmissibility grayscale image for online identification of offshore platform damage pattern

In response to the challenges in offshore platform damage pattern recognition, where traditional methods lack adaptability owing to relying on feature extraction and expert knowledge, a novel method for online recognition of offshore platform damage patterns using an improved convolutional neural network (CNN) based on transmissibility grayscale image is proposed. The original transmissibility function is converted into a two-dimensional grayscale image to capture the essential features in the original signal, which is then used as the input for the deep learning model. Subsequently, an attention mechanism is introduced to enable the CNN to focus on key areas of the image, emphasize important feature channels, dynamically adjust feature representations, and enhance its overall performance. Meanwhile, the model is optimized by virtue of variational Bayesian algorithm and early stopping criteria, and finally, the trained offline model is applied to online damage pattern recognition of offshore platforms. Through on indoor vibration monitoring experiments, seven different damage patterns of the platform model are recognized accurately, which show that the accuracy of the improved CNN model is increased by an average of 8.5 % as compared to traditional intelligent methods. Furthermore, the proposed method is applied to the online damage identification of offshore platforms in the field, and the corresponding prediction accuracy reaches 98.9 %, demonstrating the feasibility and generalizability of the proposed method in adaptive feature extraction and precise identification of damage patterns.

Attitude solving aided UWB/IMU integrated algorithm with real-time NLOS suppression

In response to the challenges posed by non-line-of-sight (NLOS) errors and inadequate attitude estimation accuracy in ultra-wideband and inertial measurement unit (UWB/IMU) integrated navigation algorithms in the complex environment, a robust UWB/IMU integrated positioning scheme is proposed. On one hand, the utilization of the robust local weighted regression algorithm (RLWR) is employed to mitigate the impact of NLOS errors on UWB data. RLWR incorporates information from nodes with known pseudo-range within local time intervals into the regression model, enhancing the identification of NLOS errors and improving positioning accuracy. On the other hand, the variational Bayesian filter algorithm based on adaptive conjugate gradient descent (ACGD) is proposed to improve the accuracy of IMU attitude calculation. The algorithm leverages an ACGD approach to optimize the attitude output of the accelerometer and magnetometer. The output is then incorporated into the variational Bayesian filtering system alongside the gyroscopic attitude output compensated by integrated positioning. Compared to conventional quaternion calculation and gradient descent linear filtering methods, the approach exhibits superior precision and stability. The experimental findings demonstrate that the amalgamation of the proposed NLSO identification suppression algorithm and the enhanced attitude computation algorithm confers significant advantages in terms of both localization accuracy and attitude estimation precision in complex environments. Moreover, the robust solution presented in the paper ensures the preservation of filter performance in the event of UWB measurement failure.

Functional form selection and calibration of macroscopic fundamental diagrams

Macroscopic fundamental diagram (MFD) is widely applied in network-level traffic control and management with most applications necessitating a well-calibrated MFD. With various data sources, more and more empirical MFDs are documented, while the MFD functional form is predetermined by traffic engineers based on their prior experiences. To our best, no generally accepted functional form has been identified. An automatic functional form selection method is yet to be devised. To meet this, a two-step MFD calibration framework is proposed to enable both the functional form selection and the estimation of parameters in this paper. A math program problem is first developed to identify a proper functional form from a set of candidate functions via random sampling of the measurement data. A mean-field variational Bayesian (MFVB) algorithm is then proposed to estimate the parameters of the selected MFD functions using the full measurement dataset. Both calibrations with and without the MFD dynamics are evaluated. The comparison between these calibration results highlights that the calibration considering the MFD dynamics can better characterize network traffic dynamics subject to dynamic travel demand and traffic control measures. Leveraging functional form selection and the computational advantages of the MFVB method, the two-step framework can significantly reduce the computational burden. Results using simulated data and empirical data validate the effectiveness and efficiency of the two-step framework. Furthermore, different functional forms are identified for different cities, highlighting the importance of functional form selection in the MFD calibration.

Fast Bayesian Inference of Reparameterized Gamma Process With Random Effects

In the field of reliability engineering, the gamma process plays an important role in modeling degradation processes. However, extracting lifetime information from product degradation observations has long been suffering from both ineffective modeling techniques and inefficient statistical inference methods. To overcome these challenges, we propose a reparameterized gamma process with random effects in this article. Compared with the classical gamma process, the proposed model has a more intuitive physical interpretation. In addition, statistical inference for the model can be readily done through the variational Bayesian algorithm. Combining with the Gauss–Hermite quadrature and the Laplace approximation, the algorithm yields closed-form variational posteriors for the proposed model. Its superiority over two other inference methods (expectation maximization and Monte Carlo Markov Chain) in terms of computational efficiency and estimation accuracy is demonstrated by simulation.

Bayesian Dictionary Learning on Robust Tubal Transformed Tensor Factorization.

The recent study on tensor singular value decomposition (t-SVD) that performs the Fourier transform on the tubes of a third-order tensor has gained promising performance on multidimensional data recovery problems. However, such a fixed transformation, e.g., discrete Fourier transform and discrete cosine transform, lacks being self-adapted to the change of different datasets, and thus, it is not flexible enough to exploit the low-rank and sparse property of the variety of multidimensional datasets. In this article, we consider a tube as an atom of a third-order tensor and construct a data-driven learning dictionary from the observed noisy data along the tubes of the given tensor. Then, a Bayesian dictionary learning (DL) model with tensor tubal transformed factorization, aiming to identify the underlying low-tubal-rank structure of the tensor effectively via the data-adaptive dictionary, is developed to solve the tensor robust principal component analysis (TRPCA) problem. With the defined pagewise tensor operators, a variational Bayesian DL algorithm is established and updates the posterior distributions instantaneously along the third dimension to solve the TPRCA. Extensive experiments on real-world applications, such as color image and hyperspectral image denoising and background/foreground separation problems, demonstrate both effectiveness and efficiency of the proposed approach in terms of various standard metrics.

Variational bayesian clustering algorithm for unsupervised anomalous sound detection incorporating VH-BCL+

Variational bayesian clustering algorithm for unsupervised anomalous sound detection incorporating VH-BCL+

Iterative channel estimation and data detection algorithm for MIMO-OTFS systems

Channel estimation in high-mobility environments is a challenging problem for advanced mobile communication systems (5G and beyond). In this manuscript, we first propose an iterative algorithm for channel estimation and data detection in the delay-Doppler domain for multiple-input multiple-output orthogonal time frequency space system. Then, in order to increase the spectral efficiency of the system, we use a superimposed pilot pattern. The proposed algorithm takes advantage from the sparse nature of the channel in the delay-Doppler domain and iterates between message passing-aided data detection and data-aided channel estimation. For channel estimation, we propose two algorithms. The first one consists in estimating all channel parameters, including the number of path gains, delay taps, Doppler taps, and channel gains by using a mean-field approximation and the variational Bayesian expectation maximization algorithm. The second one, based on the fact that delay and Doppler taps remain unchanged for a rather long period of time, uses an MMSE approach combined with Cholesky decomposition to only estimate channel gains in each transmitted frame. For data detection, we adapt the message-passing algorithm proposed in the literature. We also derive a lower bound on the signal-to-interference-plus-noise ratio of the proposed scheme, and maximize it by optimally allocating power between pilots and data symbols. Finally, we compare the complexity and the performance in terms of normalized mean square error, bit error rate, and spectral efficiency against existing methods. Simulation results, conducted in high-mobility scenarios show that the proposed algorithm achieves a good compromise between complexity and performance.

Identification of Errors-in-Variable System With Heteroscedastic Noise and Partially Known Input Using Variational Bayesian

In this paper, an approach for identification of an errors-in-variable (EIV) system whose output is contaminated by heteroscedastic noise is developed. A Markov chain is applied to depict the correlation of the switching of heteroscedastic noise model. The estimation of model parameters adopts a variational Bayesian algorithm. The advantage of the Bayesian approach is the full probability description of the estimates while the classical expectation-maximization (EM) algorithm only provides point estimation. A simulated numerical example and an experimental study on a polyester fiber process are provided to demonstrate the effectiveness of the proposed method. Three performance indexes, NMAE, MRE and RMSE, are used to evaluate the performance of the proposed algorithm. Meanwhile, Monte Carlo cross validations are performed to demonstrate the effectiveness and superiority of the proposed algorithm.

Joint Latent Space Model for Social Networks with Multivariate Attributes.

In social, behavioral and economic sciences, researchers are interested in modeling a social network among a group of individuals, along with their attributes. The attributes can be responses to survey questionnaires and are often high dimensional. We propose a joint latent space model (JLSM) that summarizes information from the social network and the multivariate attributes in a person-attribute joint latent space. We develop a variational Bayesian expectation-maximization estimation algorithm to estimate the attribute and person locations in the joint latent space. This methodology allows for effective integration, informative visualization and prediction of social networks and attributes. Using JLSM, we explore the French financial elites based on their social networks and their career, political views and social status. We observe a division in the social circles of the French elites in accordance with the differences in their attributes. We analyze user networks and behaviors in multimodal social media systems like YouTube. A R package "jlsm" is developed to fit the models proposed in this paper and is publicly available from the CRAN repository https://cran.r-project.org/web/packages/jlsm/jlsm.pdf .

Variational Bayesian inference for bipartite mixed-membership stochastic block model with applications to collaborative filtering

A network-based method applied to collaborative filtering in recommender systems is introduced in this paper. Specifically, a novel mixed-membership stochastic block model with a conjugate prior from the exponential family is proposed for bipartite networks. The analytical expression of the model is derived, and a variational Bayesian algorithm that is computationally feasible for approximating the untractable posterior distributions is presented. Extensive simulations show that the proposed model provides more accurate inference than competing methods with the presence of outliers. The proposed model is also applied to a MovieLens dataset for a real data application.

Variational Bayesian Algorithms for Maneuvering Target Tracking with Nonlinear Measurements in Sensor Networks.

The variational Bayesian method solves nonlinear estimation problems by iteratively computing the integral of the marginal density. Many researchers have demonstrated the fact its performance depends on the linear approximation in the computation of the variational density in the iteration and the degree of nonlinearity of the underlying scenario. In this paper, two methods for computing the variational density, namely, the natural gradient method and the simultaneous perturbation stochastic method, are used to implement a variational Bayesian Kalman filter for maneuvering target tracking using Doppler measurements. The latter are collected from a set of sensors subject to single-hop network constraints. We propose a distributed fusion variational Bayesian Kalman filter for a networked maneuvering target tracking scenario and both of the evidence lower bound and the posterior Cramér-Rao lower bound of the proposed methods are presented. The simulation results are compared with centralized fusion in terms of posterior Cramér-Rao lower bounds, root-mean-squared errors and the 3σ bound.

Variational inference for detecting differential translation in ribosome profiling studies.

Translational efficiency change is an important mechanism for regulating protein synthesis. Experiments with paired ribosome profiling (Ribo-seq) and mRNA-sequencing (RNA-seq) allow the study of translational efficiency by simultaneously quantifying the abundances of total transcripts and those that are being actively translated. Existing methods for Ribo-seq data analysis either ignore the pairing structure in the experimental design or treat the paired samples as fixed effects instead of random effects. To address these issues, we propose a hierarchical Bayesian generalized linear mixed effects model which incorporates a random effect for the paired samples according to the experimental design. We provide an analytical software tool, "riboVI," that uses a novel variational Bayesian algorithm to fit our model in an efficient way. Simulation studies demonstrate that "riboVI" outperforms existing methods in terms of both ranking differentially translated genes and controlling false discovery rate. We also analyzed data from a real ribosome profiling experiment, which provided new biological insight into virus-host interactions by revealing changes in hormone signaling and regulation of signal transduction not detected by other Ribo-seq data analysis tools.

Variational Bayesian Inference for Robust Identification of PWARX Systems With Time-Varying Time-Delays.

This article presents a robust variational Bayesian (VB) algorithm for identifying piecewise autoregressive exogenous (PWARX) systems with time-varying time-delays. To alleviate the adverse effects caused by outliers, the probability distribution of noise is taken to follow a t -distribution. Meanwhile, a solution strategy for more accurately classifying undecidable data points is proposed, and the hyperplanes used to split data are determined by a support vector machine (SVM). In addition, maximum-likelihood estimation (MLE) is adopted to re-estimate the unknown parameters through the classification results. The time-delay is regarded as a hidden variable and identified through the VB algorithm. The effectiveness of the proposed algorithm is illustrated by two simulation examples.

Distributed variational Bayesian adaptive filtering for randomly delayed measurements and unknown noise statistics in multi-sensor networked systems

To deal with randomly delayed measurements and unknown noise statistics in multi-sensor networked systems, a distributed variational Bayesian adaptive cubature Kalman filtering algorithm (DVB_ACKF) is presented in this paper. To compensate for the effects induced by randomly delayed measurements, a refined measurement reorganization method is designed via a two-stage estimation strategy. Specifically, we first estimate the time delay step. The measurements are then rearranged based on the estimated results from the first stage. Then, a novel noise estimator with the sliding-window and variational Bayesian method is developed to estimate the mean and covariance of the unknown noise. The network topology attribute and measurement accuracy have been combined to eliminate the distributed fusion error. The tuning parameter and forgetting factor as well as their bounds are analyzed. Furthermore, numerical simulations are carried out to verify the superior performance of the developed algorithm.

A Variational Bayesian Blind Calibration Approach for Air Quality Sensor Deployments

Air pollution has become a global threat to urban environments and public health. Low-cost air quality sensor systems have been deployed to support fine-grained monitoring, and in-field calibration methods are necessary to assure the accuracy of sensor observation. Existing methods use in-field reference to train some specific calibration models. Nevertheless, collecting sufficient reference data after deployment is challenging. First, the synchronized data pair of reference observation is hard to obtain. Thus, many blind calibration approaches rely on some alternative reference like a historical reference before sensor observation. However, since the relationship model between alternative reference and sensor observation becomes more complicated, the amount of collected data is still insufficient to learn the complex model. To address the above challenge, we propose a Variational Bayesian Blind Calibration Algorithm. Our method introduces a supplement set that only involves historical reference and synchronized reference measurement without sensor observation to alleviate the few-reference pressure. Large amounts of the introduced supplement sets have been collected in different cities by accurate stations, which we adopt to conduct prediction tasks and build a Bayesian model to learn how to combine the formulated prediction task and target calibration task in a theoretically better way. Furthermore, we design a variational Bayesian framework to make good use of the easily obtained supplement set to train a better prediction model for improving calibration performance and relieving the pressure of collecting costly reference measurements after deployment. Evaluations of real-world and synthetic datasets show that the proposed approach has a better performance than previous baselines.

MAntRA: A framework for model agnostic reliability analysis

We propose a novel model-agnostic data-driven reliability analysis framework for time-dependent reliability analysis. The proposed approach – referred to as MAntRA – combines Bayesian inference and stochastic differential equations to evaluate the reliability of stochastically-driven dynamical systems for which the governing physics is a priori unknown. A two-stage approach is adopted: in the first stage, an efficient variational Bayesian equation discovery algorithm is developed to determine the governing physics of an underlying stochastic differential equation (SDE) from measured output-only data. The developed algorithm is efficient and accounts for epistemic uncertainty due to limited and noisy data and aleatoric uncertainty because of environmental effects and external excitation. In the second stage, the discovered SDE is solved using a stochastic integration scheme, and the probability of failure is computed. The efficacy of the proposed approach is illustrated in four numerical examples. The results obtained indicate the possible application of the proposed approach for reliability analysis of in-situ and heritage structures from on-site measurements.

Distributed Bayesian Inference Over Sensor Networks.

In this article, two novel distributed variational Bayesian (VB) algorithms for a general class of conjugate-exponential models are proposed over synchronous and asynchronous sensor networks. First, we design a penalty-based distributed VB (PB-DVB) algorithm for synchronous networks, where a penalty function based on the Kullback-Leibler (KL) divergence is introduced to penalize the difference of posterior distributions between nodes. Then, a token-passing-based distributed VB (TPB-DVB) algorithm is developed for asynchronous networks by borrowing the token-passing approach and the stochastic variational inference. Finally, applications of the proposed algorithm on the Gaussian mixture model (GMM) are exhibited. Simulation results show that the PB-DVB algorithm has good performance in the aspects of estimation/inference ability, robustness against initialization, and convergence speed, and the TPB-DVB algorithm is superior to existing token-passing-based distributed clustering algorithms.

Clustering Hidden Markov Models With Variational Bayesian Hierarchical EM.

The hidden Markov model (HMM) is a broadly applied generative model for representing time-series data, and clustering HMMs attract increased interest from machine learning researchers. However, the number of clusters (K) and the number of hidden states (S) for cluster centers are still difficult to determine. In this article, we propose a novel HMM-based clustering algorithm, the variational Bayesian hierarchical EM algorithm, which clusters HMMs through their densities and priors and simultaneously learns posteriors for the novel HMM cluster centers that compactly represent the structure of each cluster. The numbers K and S are automatically determined in two ways. First, we place a prior on the pair (K,S) and approximate their posterior probabilities, from which the values with the maximum posterior are selected. Second, some clusters and states are pruned out implicitly when no data samples are assigned to them, thereby leading to automatic selection of the model complexity. Experiments on synthetic and real data demonstrate that our algorithm performs better than using model selection techniques with maximum likelihood estimation.

Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution.

In this paper, we introduce a variational Bayesian algorithm (VBA) for image blind deconvolution. Our VBA generic framework incorporates smoothness priors on the unknown blur/image and possible affine constraints (e.g., sum to one) on the blur kernel, integrating the VBA within a neural network paradigm following an unrolling methodology. The proposed architecture is trained in a supervised fashion, which allows us to optimally set two key hyperparameters of the VBA model and leads to further improvements in terms of resulting visual quality. Various experiments involving grayscale/color images and diverse kernel shapes, are performed. The numerical examples illustrate the high performance of our approach when compared to state-of-the-art techniques based on optimization, Bayesian estimation, or deep learning.

A Novel Hybrid Method Based on Deep Learning for an Integrated Navigation System during DVL Signal Failure

The navigation performance of an autonomous underwater vehicle (AUV) as the main tool for exploring the ocean greatly affects its work efficiency. Under the circumstance that high-precision GNSS positioning signals cannot be obtained, the role of the Strapdown Inertial Navigation System/Doppler Velocity Log (SINS/DVL) integrated navigation system is becoming more prominent. Due to marine creatures or the seafloor topography, DVL is prone to outliers or even failures during measurement. To solve these problems, a LSTM/SVR-VBAKF algorithm aided integrated navigation system is proposed. First, under normal circumstances of DVL, the output information of SINS and DVL are used as training samples, and they train the Long Short-Term Memory (LSTM) model. To enhance the robustness and adaptability of the filter, a novel variational Bayesian adaptive filtering algorithm based on support vector regression is proposed. When the DVL formation is missing, the deep learning method adopted in this paper will be continuously output to ensure the effect of integrated navigation. The shipboard test data is verified from two aspects: filter performance and neural network-assisted integrated navigation system capability. The experimental results show that the new method proposed in this paper can effectively handle a situation where DVL output is not available.