Stein Variational Gradient Descent Research Articles

Physics‐reliable frugal local uncertainty analysis for full waveform inversion

AbstractFull waveform inversion stands at the forefront of seismic imaging technologies, pivotal in retrieving high‐resolution subsurface velocity models. Its application is especially profound when imaging complex geologies such as salt bodies, which are regions notoriously challenging, yet essential given their hydrocarbon potential. However, with the power of full waveform inversion comes the intrinsic challenge of estimating the associated uncertainties. Such uncertainties are crucial in understanding the reliability of subsurface models, particularly in terrains like subsalt regions. Addressing this, we advocate for a nuanced approach employing the Stein variational gradient descent algorithm. Through a judicious use of a limited number of velocity model particles and the integration of random field‐based perturbations, our methodology provides a local representation of the uncertainties inherent in full waveform inversion. Our evaluations, based on the Marmousi model, showcase the robustness of the proposed technique. Yet, it is our exploration into salt‐intensive terrains, leveraging data from the Sigsbee 2A synthetic model and the Gulf of Mexico, that emphasizes the method's versatility. Findings indicate pronounced uncertainties along salt boundaries and in the deeper subsalt sediments, contrasting the minimal uncertainties in non‐salt terrains. However, anomalies like salt canyons present unique challenges, potentially due to the interplay of multi‐scattering effects. Emphasizing the scalability and cost‐effectiveness of this approach, we highlight its potential for large‐scale industrial applications in full waveform inversion, while also underscoring the necessity for prudence when integrating these uncertainty insights into subsequent seismic‐driven geological and reservoir modelling.

Bayesian variational time-lapse full waveform inversion

Summary Time-lapse seismic full-waveform inversion (FWI) provides estimates of dynamic changes in the Earth’s subsurface by performing multiple seismic surveys at different times. Since FWI problems are highly non-linear and non-unique, it is important to quantify uncertainties in such estimates to allow robust decision making based on the results. Markov chain Monte Carlo (McMC) methods have been used for this purpose, but due to their high computational cost, those studies often require a pre-existing accurate baseline model and estimates of the locations of potential velocity changes, and neglect uncertainty in the baseline velocity model. Such detailed and accurate prior information is not always available in practice. In this study we use an efficient optimization method called stochastic Stein variational gradient descent (sSVGD) to solve time-lapse FWI problems without assuming such prior knowledge, and to estimate uncertainty both in the baseline velocity model and the velocity change over time. We test two Bayesian strategies: separate Bayesian inversions for each seismic survey, and a single join inversion for baseline and repeat surveys, and compare the methods with standard linearised double difference inversion. The results demonstrate that all three methods can produce accurate velocity change estimates in the case of having fixed (exactly repeatable) acquisition geometries. However, the two Bayesian methods generate significantly more accurate results when acquisition geometries changes between surveys. Furthermore, joint inversion provides the most accurate velocity change and uncertainty estimates in all cases tested. We therefore conclude that Bayesian time-lapse inversion using a joint inversion strategy may be useful to image and monitor subsurface changes, in particular where variations in the results would lead to different consequent decisions.

A generalized data assimilation architecture of digital twin for complex process industrial systems

As one of the critical cores of digital twin (DT), data assimilation (DA) can maintain consistency and synchronization between DT and physical system. Kalman filtering is a common DA method, but its estimation performance is deteriorated by factors such as model inaccuracy and time-varying noise covariance in practical applications. The errors caused by these multiple uncertainties are all coupled to the measurements, which augments the difficulty for DT to obtain physical system information. In order to tackle the DA problem with multiple uncertainties, this paper proposes a generalized DA architecture for DT in sophisticated process industry. First, by combining Stein variational gradient descent and nonlinear Bayesian filtering paradigm, a recursive estimation framework is established, which has higher accuracy in estimating the noise covariance compared to traditional methods. Second, to effectively deal with model inaccuracy by using filtering residuals containing time-varying noise, we propose a neural network and modified wavelet-based model error compensation (NNMW-MEC) block. Based on the modified wavelet technique, the filtering residual denoising built in NNMW-MEC can better cope with time-varying noise compared to existing wavelets, and extract the low-frequency signal involving model error information from noisy residual smoothly. In addition, because of the neural network-based state-compensation subblock, NNMW-MEC has more outstanding ability in compensating the state deviations with large changing range. Finally, we take the boiler system in a coal-fired power plant as an example to verify the effectiveness of our architecture. Experimental results show that the DA architecture proposed in this paper can improve the estimation performance of DT under inaccurate models and uncertain noise statistics.

Posterior sampling with convolutional neural network-based plug-and-play regularization with applications to poststack seismic inversion

Uncertainty quantification is a crucial component in any geophysical inverse problem, as it provides decision makers with valuable information about the inversion results. Seismic inversion is a notoriously ill-posed inverse problem, due to the band-limited and noisy nature of seismic data; as such, quantifying the uncertainties associated with the ill-posed nature of this inversion process is essential for qualifying the subsequent interpretation and decision-making processes. Selecting appropriate prior information is a crucial — yet nontrivial — step in probabilistic inversion because it influences the ability of sampling-based inference algorithms to provide geologically plausible posterior samples. However, the necessity to encapsulate prior knowledge into a probability distribution can greatly limit our ability to define expressive priors. To address this limitation and following in the footsteps of the plug-and-play (PnP) methodology for deterministic inversion, we develop a regularized variational inference framework that performs posterior inference by implicitly regularizing the Kullback-Leibler divergence loss — a measure of the distance between the approximated and target probabilistic distributions — with a convolutional neural network-based denoiser. We call this new algorithm PnP Stein variational gradient descent and determine its ability to produce high-resolution trustworthy samples that realistically represent subsurface structures. Our method is validated on synthetic and field poststack seismic data.

Implicit Posteriori Parameter Distribution Optimization in Reinforcement Learning.

Efficient and intelligent exploration remains a major challenge in the field of deep reinforcement learning (DRL). Bayesian inference with a distributional representation is usually an effective way to improve the exploration ability of the RL agent. However, when optimizing Bayesian neural networks (BNNs), most algorithms need to specify an explicit parameter distribution such as a multivariate Gaussian distribution. This may reduce the flexibility of model representation and affect the algorithm performance. Therefore, to improve sample efficiency and exploration based on Bayesian methods, we propose a novel implicit posteriori parameter distribution optimization (IPPDO) algorithm. First, we adopt a distributional perspective on the parameter and model it with an implicit distribution, which is approximated by generative models. Each model corresponds to a learned latent space, providing structured stochasticity for each layer in the network. Next, to make it possible to optimize an implicit posteriori parameter distribution, we build an energy-based model (EBM) with value function to represent the implicit distribution which is not constrained by any analytic density function. Then, we design a training algorithm based on amortized Stein variational gradient descent (SVGD) to improve the model learning efficiency. We compare IPPDO with other prevailing DRL algorithms on the OpenAI Gym, MuJoCo, and Box2D platforms. Experiments on various tasks demonstrate that the proposed algorithm can represent the parameter uncertainty implicitly for a learned policy and can consistently outperform competing approaches.

A real-time hierarchical control method for safe human–robot coexistence

In modern industrial environments, robots are expected to work close to human operators collaborating with them in completing various tasks, e.g., collaborative assembly or delivery of objects. However, it is difficult to take task optimization into account under the premise of ensuring safety in a human-involved dynamic environment. Therefore, a real-time hierarchical control method containing two hierarchical optimal controllers with complementary functions is proposed to tackle the above problems. The upper-layer Model Predictive Controller aims at performing primary tasks, such as end-effector pose tracking, singularity, and joint limit avoidance. It is formulated as a Bayesian Inference problem with a Gaussian process prior and an exponential likelihood function. The resulting maximum a posteriori estimation problem can be solved efficiently using the Matrix-Scaled Stein Variational Gradient Descent and GPU. The upper-layer optimal controller aims at performing primary tasks, such as end-effector pose tracking, singularity, and joint limit avoidance. The lower-layer safety-critical controller, formulated as a constrained quadratic programming problem, is responsible for tracking the output interpolation of the higher-layer controller while respecting the safety constraints constructed in the form of Stochastic Control Barrier Functions. Both of the optimal controllers run repeatedly but with different frequencies (upper-layer controller: 20Hz, lower-layer controller: 40Hz). The proposed method provides a solution to deal with both collision avoidance and task constraints. Lastly, we conducted real-time simulation and experimental tests on a Doosan robot, and the test results validated the effectiveness of the proposed method.

A rotorcraft in-flight ice detection framework using computational aeroacoustics and Bayesian neural networks

This work develops a novel ice detection framework specifically suitable for rotorcraft using computational aeroacoustics and Bayesian neural networks. In an offline phase of the work, the acoustic signature of glaze and rime ice shapes on an oscillating wing are computed. In addition, the aerodynamic performance indicators corresponding to the ice shapes are also monitored. These performance indicators include the lift, drag, and moment coefficients. A Bayesian neural network is subsequently trained using projected Stein variational gradient descent to create a mapping from the acoustic signature generated by the iced wings to predict their performance indicators along with quantified uncertainty that is highly important for time- and safety-critical decision-making scenarios. While the training is carried out fully offline, usage of the Bayesian neural network to make predictions can be conducted rapidly online allowing for an ice detection system that can be used in real time and in-flight.

DAMix: Exploiting Deep Autoregressive Model Zoo for Improving Lossless Compression Generalization

Deep generative models have demonstrated superior performance in lossless compression on identically distributed data. However, in real-world scenarios, data to be compressed are of various distributions and usually cannot be known in advance. Thus, commercially expected neural compression must have strong Out-of-Distribution (OoD) generalization capabilities. Compared with traditional compression methods, deep learning methods have intrinsic flaws for OoD generalization. In this work, we make the attempt to tackle this challenge via exploiting a zoo of Deep Autoregressive models (DAMix). We build a model zoo consisting of autoregressive models trained on data from diverse distributions. In the test phase, we select useful expert models by a simple model evaluation score and adaptively aggregate the predictions of selected models. By assuming the outputs from each expert model are biased in favor of their training distributions, a von Mises-Fisher based filter is proposed to recover the value of unbiased predictions that provides more accurate density estimations than a single model. We derive the posterior of unbiased predictions as well as concentration parameters in the filter, and a novel temporal Stein variational gradient descent for sequential data is proposed to adaptively update the posterior distributions. We evaluate DAMix on 22 image datasets, including in-distribution and OoD data, and demonstrate that making use of unbiased predictions has up to 45.6% improvement over the single model trained on ImageNet.

Variational inference accelerates accurate DNA mixture deconvolution

We investigate a class of DNA mixture deconvolution algorithms based on variational inference, and we show that this can significantly reduce computational runtimes with little or no effect on the accuracy and precision of the result. In particular, we consider Stein Variational Gradient Descent (SVGD) and Variational Inference (VI) with an evidence lower-bound objective. Both provide alternatives to the commonly used Markov-Chain Monte-Carlo methods for estimating the model posterior in Bayesian probabilistic genotyping. We demonstrate that both SVGD and VI significantly reduce computational costs over the current state of the art. Importantly, VI does so without sacrificing precision or accuracy, presenting an overall improvement over previously published methods.

Stein variational gradient descent with learned direction

Recently, as a particle-based Bayesian inference method, Stein variational gradient descent (SVGD) has attracted much attention due to its strong approximation power and efficiency compared to traditional variational inference and Markov chain Monte Carlo sampling methods. However, the original SVGD method assumes the descent direction to be within an reproducing kernel Hilbert space (RKHS), which on one hand limits its expressive power, and on the other hand suffers from the curse of dimensionality. To address this issue, we propose to parameterize the descent direction using a flexible neural network. With this parameterization strategy, the descent direction can be possibly optimized in each step for updating the particles, such that the target distribution can be expected to be better approximated. Experiments on both synthetic and real datasets demonstrate the effectiveness and efficiency of the proposed method, especially in the high-dimensional situations, as compared with the original SVGD.

Client Selection for Federated Bayesian Learning

Distributed Stein Variational Gradient Descent (DSVGD) is a non-parametric distributed learning framework for federated Bayesian learning, where multiple clients jointly train a machine learning model by communicating a number of non-random and interacting particles with the server. Since communication resources are limited, selecting the clients with most informative local learning updates can improve the model convergence and communication efficiency. In this paper, we propose two selection schemes for DSVGD based on Kernelized Stein Discrepancy (KSD) and Hilbert Inner Product (HIP). We derive the upper bound on the decrease of the global free energy per iteration for both schemes, which is then minimized to speed up the model convergence. We evaluate and compare our schemes with conventional schemes in terms of model accuracy, convergence speed, and stability using various learning tasks and datasets.

An improved Hammerstein system identification method using Stein Variational Inference and sampling technology

This paper considers the identification of the Hammerstein system with immeasurable process noise. The complexity of the Hammerstein system makes it difficult to obtain accurate mathematical expressions of the parameters, or even impossible to obtain accurate mathematical expressions at all. In this contribution, we cast the Hammerstein system parameter identification problem as a posterior parameter estimation problem and take a sampling and Stein variational inference viewpoint to solve it. Improved Stein variational gradient descent(ISVGD)algorithm is proposed in posterior parameter calculation. Compared with other methods, not only the prior distribution of parameters but also the proposal distribution of parameters is considered. In other words, The posterior information is enriched and the parameter identification accuracy is improved. At the same time, ISVGD and reversible jump markov chain monte carlo (RJMCMC) algorithms are used (called ISVGD-RJMCMC algorithm) in the structure detection problem. In the algorithm, the correct basis function number k can be found in the parameter estimation. It tends to be accurate but is slow to converge. Three simulation examples were given to demonstrate the proposed algorithm’s effectiveness. Furthermore, the performances of these approaches were analyzed, including parameter estimation accuracy and error, system order estimation and parameter convergence analysis.

3-D Bayesian variational full waveform inversion

SUMMARYSeismic full-waveform inversion (FWI) provides high resolution images of the subsurface by exploiting information in the recorded seismic waveforms. This is achieved by solving a highly non-linear and non-unique inverse problem. Bayesian inference is therefore used to quantify uncertainties in the solution. Variational inference is a method that provides probabilistic, Bayesian solutions efficiently using optimization. The method has been applied to 2-D FWI problems to produce full Bayesian posterior distributions. However, due to higher dimensionality and more expensive computational cost, the performance of the method in 3-D FWI problems remains unknown. We apply three variational inference methods to 3-D FWI and analyse their performance. Specifically, we apply automatic differential variational inference (ADVI), Stein variational gradient descent (SVGD) and stochastic SVGD (sSVGD), to a 3-D FWI problem and compare their results and computational cost. The results show that ADVI is the most computationally efficient method but systematically underestimates the uncertainty. The method can therefore be used to provide relatively rapid but approximate insights into the subsurface together with a lower bound estimate of the uncertainty. SVGD demands the highest computational cost, and still produces biased results. In contrast, by including a randomized term in the SVGD dynamics, sSVGD becomes a Markov chain Monte Carlo method and provides the most accurate results at intermediate computational cost. We thus conclude that 3-D variational FWI is practically applicable, at least in small problems, and can be used to image the Earth’s interior and to provide reasonable uncertainty estimates on those images.

Stein variational gradient descent: Many-particle and long-time asymptotics

Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics:variational inferenceandMarkov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify theStein-Fisher information(orkernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of $ \Gamma $-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.

Bayesian Physics-Informed Neural Networks for the Subsurface Tomography Based on the Eikonal Equation

The high cost of acquiring a sufficient amount of seismic data for training has limited the use of machine learning in seismic tomography. In addition, the inversion uncertainty due to the noisy data and data scarcity is less discussed in conventional seismic tomography literature. To mitigate the uncertainty effects and quantify their impacts in the prediction, the so-called Bayesian Physics-Informed Neural Networks (BPINNs) based on the eikonal equation are adopted to infer the velocity field and reconstruct the travel-time field. In BPINNs, two inference algorithms including Stein Variational Gradient Descent (SVGD) and Gaussian variational inference (VI) are investigated for the inference task. The numerical results of several benchmark problems demonstrate that the velocity field can be estimated accurately and the travel-time can be well approximated with reasonable uncertainty estimates by BPINNs. This suggests that the inferred velocity model provided by BPINNs may serve as a valid initial model for seismic inversion and migration.

Bayesian Seismic Tomography Based on Velocity-Space Stein Variational Gradient Descent for Physics-Informed Neural Network

Bayesian Seismic Tomography Based on Velocity-Space Stein Variational Gradient Descent for Physics-Informed Neural Network

Stein Variational Gradient Descent with Multiple Kernels

Bayesian inference is an important research area in cognitive computation due to its ability to reason under uncertainty in machine learning. As a representative algorithm, Stein variational gradient descent (SVGD) and its variants have shown promising successes in approximate inference for complex distributions. In practice, we notice that the kernel used in SVGD-based methods has a decisive effect on the empirical performance. Radial basis function (RBF) kernel with median heuristics is a common choice in previous approaches, but unfortunately, this has proven to be sub-optimal. Inspired by the paradigm of Multiple Kernel Learning (MKL), our solution to this flaw is using a combination of multiple kernels to approximate the optimal kernel, rather than a single one which may limit the performance and flexibility. Specifically, we first extend Kernelized Stein Discrepancy (KSD) to its multiple kernels view called Multiple Kernelized Stein Discrepancy (MKSD) and then leverage MKSD to construct a general algorithm Multiple Kernel SVGD (MK-SVGD). Further, MK-SVGD can automatically assign a weight to each kernel without any other parameters, which means that our method not only gets rid of optimal kernel dependence but also maintains computational efficiency. Experiments on various tasks and models demonstrate that our proposed method consistently matches or outperforms the competing methods.

A Bayesian Deep Learning Approach to Near‐Term Climate Prediction

AbstractSince model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state‐of‐the‐art decadal climate prediction efforts, we pursue a complementary machine‐learning‐based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre‐industrial control simulation of the Community Earth System Model. While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of machine learning models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

Stein Variational Gradient Descent on Infinite-Dimensional Space and Applications to Statistical Inverse Problems

In this paper, we propose an infinite-dimensional version of the Stein variational gradient descent (iSVGD) method for solving Bayesian inverse problems. The method can generate approximate samples from posteriors efficiently. Based on the concepts of operator-valued kernels and vector-valued reproducing kernel Hilbert spaces, a rigorous definition is given for the infinite-dimensional objects, e.g., the Stein operator, which are proved to be the limit of finite-dimensional ones. Moreover, a more efficient iSVGD with preconditioning operators is constructed by generalizing the change of variables formula and introducing a regularity parameter. The proposed algorithms are applied to an inverse problem of the steady state Darcy flow equation. Numerical results confirm our theoretical findings and demonstrate the potential applications of the proposed approach in the posterior sampling of large-scale nonlinear statistical inverse problems.

Stabilizing Training of Generative Adversarial Nets via Langevin Stein Variational Gradient Descent.

Generative adversarial networks (GANs), which are famous for the capability of learning complex underlying data distribution, are, however, known to be tricky in the training process, which would probably result in mode collapse or performance deterioration. Current approaches of dealing with GANs' issues almost utilize some practical training techniques for the purpose of regularization, which, on the other hand, undermines the convergence and theoretical soundness of GAN. In this article, we propose to stabilize GAN training via a novel particle-based variational inference-Langevin Stein variational gradient descent (LSVGD), which not only inherits the flexibility and efficiency of original SVGD but also aims to address its instability issues by incorporating an extra disturbance into the update dynamics. We further demonstrate that, by properly adjusting the noise variance, LSVGD simulates a Langevin process whose stationary distribution is exactly the target distribution. We also show that LSVGD dynamics has an implicit regularization, which is able to enhance particles' spread-out and diversity. Finally, we present an efficient way of applying particle-based variational inference on a general GAN training procedure no matter what loss function is adopted. Experimental results on one synthetic data set and three popular benchmark data sets-Cifar-10, Tiny-ImageNet, and CelebA-validate that LSVGD can remarkably improve the performance and stability of various GAN models.