Variational Bayesian Learning Research Articles

Nonlinear sparse variational Bayesian learning based model predictive control with application to PEMFC temperature control

The accuracy of the underlying model predictions is crucial for the success of model predictive control (MPC) applications. If the model is unable to accurately analyze the dynamics of the controlled system, the performance and stability guarantees provided by MPC may not be achieved. Learning-based MPC can learn models from data, improving the applicability and reliability of MPC. This study develops a nonlinear sparse variational Bayesian learning based MPC (NSVB-MPC) for nonlinear systems, where the model is learned by the developed NSVB method. Variational inference is used by NSVB-MPC to assess the predictive accuracy and make the necessary corrections to quantify system uncertainty. The suggested approach ensures input-to-state stability (ISS) and the feasibility of recursive constraints in accordance with the concept of an invariant terminal region. Finally, a PEMFC temperature control model experiment confirms the effectiveness of the NSVB-MPC method.

Dynamic fault detection and diagnosis of industrial alkaline water electrolyzer process with variational Bayesian dictionary learning

Alkaline Water Electrolysis (AWE) is one of the simplest green hydrogen production method using renewable energy. AWE system typically yields process variables that are serially correlated and contaminated by measurement uncertainty. A novel robust dynamic variational Bayesian dictionary learning (RDVDL) monitoring approach is proposed to improve the reliability and safety of AWE operation. RDVDL employs a sparse Bayesian dictionary learning to preserve the dynamic mechanism information of AWE process which allows the easy interpretation of fault detection results. To improve the robustness to measurement uncertainty, a low-rank vector autoregressive (VAR) method is derived to reliably extract the serial correlation from process variables. The effectiveness of the proposed approach is demonstrated with an industrial hydrogen production process, and RDVDL can efficiently detect and diagnose critical AWE faults.

Improved biometric authentication using surf based variational Bayesian extreme learning machine

Improved biometric authentication using surf based variational Bayesian extreme learning machine

Robust Tensor-Based DOA and Polarization Estimation in Conformal Polarization Sensitive Array with Bad Data.

Partially impaired sensor arrays pose a significant challenge in accurately estimating signal parameters. The occurrence of bad data is highly probable, resulting in random loss of source information and substantial performance degradation in parameter estimation. In this paper, a tensor variational sparse Bayesian learning (TVSBL) method is proposed for the estimate of direction of arrival (DOA) and polarization parameters jointly based on a conformal polarization sensitive array (CPSA), taking into account scenarios with the partially impaired sensor array. First, a sparse tensor-based received data model is developed for CPSAs that incorporates bad data. Then, a column vector detection method is proposed to diagnose the positions of the impaired sensors. In scenarios involving partially impaired sensor arrays, a low-rank matrix completion method is employed to recover the random loss of signal information. Finally, variational sparse Bayesian learning (VSBL) and minimum eigenvector methods are utilized sequentially to obtain the DOA and polarization parameters estimation, successively. Furthermore, the Cramér-Rao bound is given for the proposed method. Simulation results validated the effectiveness of the proposed method.

Variational Bayesian Learning With Reliable Likelihood Approximation for Accurate Process Quality Evaluation

The State Estimation (SE) method is troubled by heavy computational tasks and poor estimation tracking capability for the large-scale active distribution network. Given the aforementioned difficulty, this paper proposed a novel multi-area Forecasting Aided State Estimation (FASE) strategy to perceive the state of the system effectively. The proposed strategy begins with the implementation of an improved multi-area FASE model. The processing of multi-source measurement data, such as Micro Phasor Measurement Units (μPMUs) and Supervisory Control and Data Acquisition (SCADA), and equivalent load based information interaction reliably complete the FASE of multi areas. Especially, a 3rd degree dimensionality reduction SR-CKF algorithm is designed for local FASE model considering the influence of large-scale distribution networks data on the numerical stability of the estimator. The case study shows the advantages of the proposed strategy in estimation accuracy, efficiency, and numerical stability compared with the existing ones.

Variational Bayesian Learning Based Localization and Channel Reconstruction in RIS-aided Systems

Variational Bayesian Learning Based Localization and Channel Reconstruction in RIS-aided Systems

Bayesian Channel Estimation for Intelligent Reflecting Surface-Aided mmWave Massive MIMO Systems With Semi-Passive Elements

In this paper, we propose a Bayesian channel estimator for intelligent reflecting surface-aided (IRS-aided) millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems with semi-passive elements that can receive the signal in the active sensing mode. Ultimately, our goal is to minimize the channel estimation error using the received signal at the base station and additional information acquired from a small number of active sensors at the IRS. Unlike recent works on channel estimation with semi-passive elements that require both uplink and downlink training signals to estimate the UE-IRS and IRS-BS links, we only use uplink training signals to estimate all the links. To compute the minimum mean squared error (MMSE) estimates of all the links, we propose a novel variational inference-sparse Bayesian learning (VI-SBL) channel estimator that performs approximate posterior inference on the channel using VI with the mean-field approximation under the SBL framework. The simulation results show that VI-SBL outperforms the state-of-the-art baselines for IRS with passive reflecting elements in terms of the channel estimation accuracy and training overhead. Furthermore, VI-SBL with semi-passive elements is shown to be more spectral- and energy-efficient than the baselines with passive reflecting elements.

Variational Bayesian Learning Based Delay-Doppler Channel Estimator for Multi-User OTFS Systems

Variational Bayesian Learning Based Delay-Doppler Channel Estimator for Multi-User OTFS Systems

Statistical mechanics of continual learning: Variational principle and mean-field potential.

An obstacle to artificial general intelligence is set by continual learning of multiple tasks of a different nature. Recently, various heuristic tricks, both from machine learning and from neuroscience angles, were proposed, but they lack a unified theory foundation. Here, we focus on continual learning in single-layered and multilayered neural networks of binary weights. A variational Bayesian learning setting is thus proposed in which the neural networks are trained in a field-space, rather than a gradient-ill-defined discrete-weight space, and furthermore, weight uncertainty is naturally incorporated, and it modulates synaptic resources among tasks. From a physics perspective, we translate variational continual learning into a Franz-Parisi thermodynamic potential framework, where previous task knowledge serves as a prior probability and a reference as well. We thus interpret the continual learning of the binary perceptron in a teacher-student setting as a Franz-Parisi potential computation. The learning performance can then be analytically studied with mean-field order parameters, whose predictions coincide with numerical experiments using stochastic gradient descent methods. Based on the variational principle and Gaussian field approximation of internal preactivations in hidden layers, we also derive the learning algorithm considering weight uncertainty, which solves the continual learning with binary weights using multilayered neural networks, and performs better than the currently available metaplasticity algorithm in which binary synapses bear hidden continuous states and the synaptic plasticity is modulated by a heuristic regularization function. Our proposed principled frameworks also connect to elastic weight consolidation, weight-uncertainty modulated learning, and neuroscience-inspired metaplasticity, providing a theoretically grounded method for real-world multitask learning with deep networks.

Target location and velocity estimation with the multistatic MU‐MIMO‐OFDM modulation signal

AbstractIn passive radar and joint communication and radar sensing (JCRS), target sensing with the multiuser multiple input multiple output orthogonal frequency division multiplexing (MU‐MIMO‐OFDM) modulation signal is gaining increasing interest. Multiple transmit nodes emitting the MU‐MIMO‐OFDM modulation signals at the same carrier frequency and one receiver collecting the target‐reflected signals for target location and velocity estimation are considered. This is a typical scenario when using the fifth‐generation (5G) communication network signal for target sensing. In this scenario, the echo signals corresponding to different transmit nodes are not resolved in the receiver, and the modulated data symbols cannot be removed from the received signals. Most traditional parameter estimation methods in passive radar and JCRS may not be suitable here. A location and velocity estimation method with the received echo signals is proposed. Specifically, the location parameters are extracted directly from the received echo signals. The location estimation is cast into a block sparse vector reconstruction problem. The variational Bayesian sparsity learning (VBSL) method is exploited for the reconstruction of the block sparse vector. Accelerated VBSL methods are developed for improving the computational efficiency. Simulations verify the effectiveness of the proposed methods.

Covariance-Free Variational Bayesian Learning for Correlated Block Sparse Signals

We consider the problem of estimating channel in massive machine type communication (mMTC) systems. The sparse device activity in a mMTC system makes the channel block-sparse, with intra-block correlation. Block-sparse Bayesian learning (B-SBL) is a powerful framework for estimating such signals. The existing B-SBL algorithms become computationally expensive for high-dimensional problems, which is common in mMTC systems. This is because of large number of devices in a mMTC system, they invert a large-dimensional matrix to calculate the covariance matrix. To address this problem, we exploit variational Bayesian inference, and design a novel covariance-free variational B-SBL algorithm which inverts multiple small-sized block matrices, instead of inverting a complete big-sized matrix. The complexity is further reduced by avoiding explicit computation of the covariance matrix. The proposed algorithm, instead of performing costly matrix inversions, solves multiple linear systems to calculate an unbiased estimate of the posterior statistics. The proposed algorithm is numerically shown to estimate the mMTC channel with a much lesser complexity, and that too without compromising the reconstruction performance.

Variational Bayesian Network with Information Interpretability Filtering for Air Quality Forecasting

Air quality plays a vital role in people’s health, and air quality forecasting can assist in decision making for government planning and sustainable development. In contrast, it is challenging to multi-step forecast accurately due to its complex and nonlinear caused by both temporal and spatial dimensions. Deep models, with their ability to model strong nonlinearities, have become the primary methods for air quality forecasting. However, because of the lack of mechanism-based analysis, uninterpretability forecasting makes decisions risky, especially when the government makes decisions. This paper proposes an interpretable variational Bayesian deep learning model with information self-screening for PM2.5 forecasting. Firstly, based on factors related to PM2.5 concentration, e.g., temperature, humidity, wind speed, spatial distribution, etc., an interpretable multivariate data screening structure for PM2.5 forecasting was established to catch as much helpful information as possible. Secondly, the self-screening layer was implanted in the deep learning network to optimize the selection of input variables. Further, following implantation of the screening layer, a variational Bayesian gated recurrent unit (GRU) network was constructed to overcome the complex distribution of PM2.5 and achieve accurate multi-step forecasting. The high accuracy of the proposed method is verified by PM2.5 data in Beijing, China, which provides an effective way, with multiple factors for PM2.5 forecasting determined using deep learning technology.

Variational Bayesian Learning for the Modelling of Indoor Broadband Powerline Communication Impulsive Noise

Variational Bayesian Learning for the Modelling of Indoor Broadband Powerline Communication Impulsive Noise

A Variational Bayesian-Based Simultaneous Localization and Mapping Method for Autonomous Underwater Vehicle Navigation

Simultaneous Localization and Mapping (SLAM) is a well-known solution for mapping and realizing autonomous navigation of an Autonomous Underwater Vehicle (AUV) in unknown underwater environments. However, the inaccurate time-varying observation noise will cause filtering divergence and reduce the accuracy of localization and feature estimation. In this paper, VB-AUFastSLAM based on the unscented-FastSLAM (UFastSLAM) and the Variational Bayesian (VB) is proposed. The UFastSLAM combines unscented particle filter (UPF) and unscented Kalman filter (UKF) to estimate the AUV poses and features. In addition, to resist the unknown time-varying observation noise, the method of Variational Bayesian learning is introduced into the SLAM framework. Firstly, the VB method is used to estimate the joint posterior probability of the AUV path and observation noise. The Inverse-Gamma distribution is used to model the observation noise and real-time noise parameters estimation is performed to improve the AUV localization accuracy. Secondly, VB is reused to estimate the noise parameters in the feature update stage to enhance the performance of the feature estimation. The proposed algorithms are first validated in an open-source simulation environment. Then, an AUV SLAM system based on the Inertial Navigation System (INS), Doppler Velocity Log (DVL), and single-beam Sonar are also built to verify the effectiveness of the proposed algorithms in the marine environment. The accuracy of the proposed methods can reach 0.742% and 0.776% of the range, respectively, which is much better than 1.825% and 1.397% of the traditional methods.

Phase retrieval for block sparsity based on adaptive coupled variational Bayesian learning

Phase retrieval (PR) of block-sparse signals is a new branch of sparse PR that causes rising research, which focusses with methods owing a high successful rate. However, the recovery performances of existing methods for block sparsity are usually unfit for large-scale problems with unacceptable compute complexity. We derive an algorithm for PR of block sparsity via variational Bayesian learning with expectation maximisation to mitigate this drawback. In the proposed algorithm, the block-sparse structure is modelled by the hierarchical constructional priors with a novel adaptive coupled pattern, which provides a strong relationship between the neighbour blocks. Simulations indicate that the proposed algorithm outperforms the existing methods in success rate, noise-robustness, and signal detection rate in large-scale cases with acceptable computation complexity.

A joint DOA and polarization estimation method based on the conformal polarization sensitive array from the sparse reconstruction perspective

The conformal polarization sensitive array (CPSA) is formed by placing some vector sensors on the conformal array and it has a wide range of practical application in direction of arrival (DOA) and polarization parameters estimation. However, due to the diversity of each sensor’s direction, the performance of the conventional parameters estimation methods based on the CPSA would decrease greatly, especially under the low signal to noise ratio (SNR) and limited snapshots. In order to solve this problem, a unified framework and sparse reconstruction perspective for joint DOA and polarization estimation based on CPSA is proposed in this paper. Specifically, the array received signal model of the CPSA is formulated first and the two-dimensional spatial sparsity of the incident signals is then exploited. Subsequently, after employing the singular value decomposition method to reduce the dimension of array output matrix, the variational sparse Bayesian learning and orthogonal matching pursuit methods are utilized to solve the source DOA estimation, respectively. Finally, the polarization parameters are obtained by the minimum eigenvector method. Simulation results demonstrate that the novel approaches can provide improved estimation accuracy and resolution with low SNR and limited snapshots.

Uncertainty quantification in super-resolution guided wave array imaging using a variational Bayesian deep learning approach

Super-resolution guided wave array imaging has shown to be a feasible tool to detect and image sub-wavelength defects. However, research gaps remain in effectively quantifying and understanding the uncertainties in super-resolution imaging. In this study, we present a Bayesian deep learning approach to quantify and interpret various uncertainties in super-resolution guided wave array imaging. Specifically, we implement a Monte Carlo (MC) dropout scheme in the multi-scale deep learning models for approximate Bayesian inference to effectively quantify uncertainties in super-resolution subwavelength defect imaging. Furthermore, we decompose the total predictive uncertainty into distinct uncertainty sources: aleatoric uncertainty inherent in the data and epistemic uncertainty associated with the Bayesian deep learning model. From the experimental study, we observe that the two types of uncertainty (aleatoric and epistemic) in super-resolution guided wave array imaging can be successfully quantified using the multi-scale Bayesian deep learning approach. We further discuss the effectiveness of the Bayesian deep learning approach in small-data cases and compare the super-resolution imaging performance with a non-Bayesian (deterministic) deep learning approach.

Phase-resolved real-time ocean wave prediction with quantified uncertainty based on variational Bayesian machine learning

Phase-resolved wave prediction is of vital importance for the real-time control of wave energy converters. In this paper, a novel wave prediction method is proposed, which, to the authors’ knowledge, achieves the real-time nonlinear wave prediction with quantified uncertainty (including both aleatory and model uncertainties) for the first time. Moreover, the proposed method achieves the prediction of the predictable zone without assuming linear sea states, while all previous works on predictable zone determination were based on linear wave theory (which produces overly conservative estimations). The proposed method is developed based on the Bayesian machine learning approach, which can take advantage of machine learning model’s ability in tackling complex nonlinear problems, while taking various forms of uncertainties into account via the Bayesian framework. A set of wave tank experiments are carried out for evaluation of the method. The results show that the wave elevations at the location of interest are predicted accurately based on the measurements at sensor location, and the prediction uncertainty and its variations across the time horizon are well captured. The comparison with other wave prediction methods shows that the proposed method outperforms them in terms of both prediction accuracy and the length of the predictable zone. Particularly, for short-term wave forecasting, the prediction error by the proposed method is as much as 55.4% and 11.7% lower than the linear wave theory and deterministic machine learning approaches, and the predictable zone is expanded by the proposed method by as much as 74.6%.

Centralized and Decentralized Channel Estimation in FDD Multi-User Massive MIMO Systems

We design a centralized and a decentralized variational Bayesian learning (C- and D-VBL) algorithms for the base station (BS) of a frequency division duplex massive multiple input multiple output (mMIMO) cellular system, wherein users send compressed information for it to estimate their downlink channels. The BS in the decentralized algorithm consists of multiple processing units (PUs), and each PU separately estimates the channels of a group of users, by employing the proposed D-VBL algorithm. To reduce channel estimation error, the PUs exploit the structured sparsity inherent in multi-user mMIMO channels by exchanging information among themselves. We investigate the proposed C-VBL and low-complexity D-VBL algorithms and show that i) they substantially outperform the state-of-the-art centralized and decentralized algorithms in terms of the normalized mean squared error and the bit error rate. This is because they beneficially exploit the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inherent</i> channel sparsity, while the existing state-of-the-art solutions fail to do so. The proposed D-VBL is also robust to PU failures, and provides a similar performance as its centralized counterpart (C-VBL), but with a much reduced complexity.

A novel variational SBL approach for off-grid DOA detection under nonuniform noise

In this paper, a novel variational sparse Bayesian learning (SBL) approach is proposed for off-grid direction of arrival (DOA) estimation in the presence of nonuniform noise. In the proposed approach, a variational off-grid sparse model over the array covariance vector is first constructed, where the nonuniform noise covariance is modeled into a variational signal vector. At the same time, the grid refining procedure based on the variational off-grid sparse model is re-derived for reducing off-grid error, in which the spatial grid points are linear approximated and iteratively updated. Benefit from the constructed variational off-grid sparse model, the proposed approach does not require a separate estimation for the nonuniform noise variance. Numerous simulation experiments illustrate that the proposed approach has strong robustness to both off-grid error and nonuniform noise, and has superior performance on DOA detection with nonuniform noise.