Uncertainty Quantification In Inverse Problems Research Articles

Principal Uncertainty Quantification With Spatial Correlation for Image Restoration Problems.

Uncertainty quantification for inverse problems in imaging has drawn much attention lately. Existing approaches towards this task define uncertainty regions based on probable values per pixel, while ignoring spatial correlations within the image, resulting in an exaggerated volume of uncertainty. In this paper, we propose PUQ (Principal Uncertainty Quantification) - a novel definition and corresponding analysis of uncertainty regions that takes into account spatial relationships within the image, thus providing reduced volume regions. Using recent advancements in generative models, we derive uncertainty intervals around principal components of the empirical posterior distribution, forming an ambiguity region that guarantees the inclusion of true unseen values with a user-defined confidence probability. To improve computational efficiency and interpretability, we also guarantee the recovery of true unseen values using only a few principal directions, resulting in more informative uncertainty regions. Our approach is verified through experiments on image colorization, super-resolution, and inpainting; its effectiveness is shown through comparison to baseline methods, demonstrating significantly tighter uncertainty regions.

CUQIpy: II. Computational uncertainty quantification for PDE-based inverse problems in Python

Inverse problems, particularly those governed by Partial Differential Equations (PDEs), are prevalent in various scientific and engineering applications, and uncertainty quantification (UQ) of solutions to these problems is essential for informed decision-making. This second part of a two-paper series builds upon the foundation set by the first part, which introduced CUQIpy, a Python software package for computational UQ in inverse problems using a Bayesian framework. In this paper, we extend CUQIpy’s capabilities to solve PDE-based Bayesian inverse problems through a general framework that allows the integration of PDEs in CUQIpy, whether expressed natively or using third-party libraries such as FEniCS. CUQIpy offers concise syntax that closely matches mathematical expressions, streamlining the modeling process and enhancing the user experience. The versatility and applicability of CUQIpy to PDE-based Bayesian inverse problems are demonstrated on examples covering parabolic, elliptic and hyperbolic PDEs. This includes problems involving the heat and Poisson equations and application case studies in electrical impedance tomography and photo-acoustic tomography, showcasing the software’s efficiency, consistency, and intuitive interface. This comprehensive approach to UQ in PDE-based inverse problems provides accessibility for non-experts and advanced features for experts.

CUQIpy: I. Computational uncertainty quantification for inverse problems in Python

This paper introduces CUQIpy, a versatile open-source Python package for computational uncertainty quantification (UQ) in inverse problems, presented as Part I of a two-part series. CUQIpy employs a Bayesian framework, integrating prior knowledge with observed data to produce posterior probability distributions that characterize the uncertainty in computed solutions to inverse problems. The package offers a high-level modeling framework with concise syntax, allowing users to easily specify their inverse problems, prior information, and statistical assumptions. CUQIpy supports a range of efficient sampling strategies and is designed to handle large-scale problems. Notably, the automatic sampler selection feature analyzes the problem structure and chooses a suitable sampler without user intervention, streamlining the process. With a selection of probability distributions, test problems, computational methods, and visualization tools, CUQIpy serves as a powerful, flexible, and adaptable tool for UQ in a wide selection of inverse problems. Part II of the series focuses on the use of CUQIpy for UQ in inverse problems with partial differential equations.

Scaling Up Bayesian Uncertainty Quantification for Inverse Problems Using Deep Neural Networks

Scaling Up Bayesian Uncertainty Quantification for Inverse Problems Using Deep Neural Networks

Hyper-differential sensitivity analysis for inverse problems governed by ODEs with application to COVID-19 modeling

We consider inverse problems governed by systems of ordinary differential equations (ODEs) that contain uncertain parameters in addition to the parameters being estimated. In such problems, which are common in applications, it is important to understand the sensitivity of the solution of the inverse problem to the uncertain model parameters. It is also of interest to understand the sensitivity of the inverse problem solution to different types of measurements or parameters describing the experimental setup. Hyper-differential sensitivity analysis (HDSA) is a sensitivity analysis approach that provides tools for such tasks. We extend existing HDSA methods by developing methods for quantifying the uncertainty in the estimated parameters. Specifically, we propose a linear approximation to the solution of the inverse problem that allows efficiently approximating the statistical properties of the estimated parameters. We also explore the use of this linear model for approximate global sensitivity analysis. As a driving application, we consider an inverse problem governed by a COVID–19 model. We present comprehensive computational studies that examine the sensitivity of this inverse problem to several uncertain model parameters and different types of measurement data. Our results also demonstrate the effectiveness of the linear approximation model for uncertainty quantification in inverse problems and for parameter screening.

Randomized approaches to accelerate MCMC algorithms for Bayesian inverse problems

Markov chain Monte Carlo (MCMC) approaches are traditionally used for uncertainty quantification in inverse problems where the physics of the underlying sensor modality is described by a partial differential equation (PDE). However, the use of MCMC algorithms is prohibitively expensive in applications where each log-likelihood evaluation may require hundreds to thousands of PDE solves corresponding to multiple sensors; i.e., spatially distributed sources and receivers perhaps operating at different frequencies or wavelengths depending on the precise application. We show how to mitigate the computational cost of each log-likelihood evaluation by using several randomized techniques and embed these randomized approximations within MCMC algorithms. The resulting MCMC algorithms are computationally efficient methods for quantifying the uncertainty associated with the reconstructed parameters. We demonstrate the accuracy and computational benefits of our proposed algorithms on a model application from diffuse optical tomography where we invert for the spatial distribution of optical absorption.

Efficient Krylov subspace methods for uncertainty quantification in large Bayesian linear inverse problems

SummaryUncertainty quantification for linear inverse problems remains a challenging task, especially for problems with a very large number of unknown parameters (e.g., dynamic inverse problems) and for problems where computation of the square root and inverse of the prior covariance matrix are not feasible. This work exploits Krylov subspace methods to develop and analyze new techniques for large‐scale uncertainty quantification in inverse problems. In this work, we assume that generalized Golub‐Kahan‐based methods have been used to compute an estimate of the solution, and we describe efficient methods to explore the posterior distribution. In particular, we use the generalized Golub‐Kahan bidiagonalization to derive an approximation of the posterior covariance matrix, and we provide theoretical results that quantify the accuracy of the approximate posterior covariance matrix and of the resulting posterior distribution. Then, we describe efficient methods that use the approximation to compute measures of uncertainty, including the Kullback‐Liebler divergence. We present two methods that use the preconditioned Lanczos algorithm to efficiently generate samples from the posterior distribution. Numerical examples from dynamic photoacoustic tomography demonstrate the effectiveness of the described approaches.

Robotics, Vision and Control: Fundamental Algorithms in MATLAB, Second Edition [Bookshelf

The second edition of this book covers an impressively broad array of important topics in robotics and computer vision. The book’s presentation style is approachable and engaging, not dense nor intimidating, and the material should be accessible to most upper-level, undergraduate engineering and computer science students. Each chapter combines foundational material, historical vignettes, colorful figures, descriptions of current technology, and software examples to help the reader quickly grasp the context, main ideas, and applications; this approach makes the book an excellent reference source. A nice feature is that, for many sections, it is possible to flip directly to that section and grasp the key ideas without having to read all of the material that comes before it. Due to the breadth of the material covered, this book can be used as a textbook for multiple courses in robotics and computer vision. The text sacrifices some mathematical details and derivations in favor of an example-driven, learn-by-doing treatment. Therefore, the instructor may wish to supplement the book with other sources that go into more detail on the fundamentals. This book is an introduction to both computational inverse problems and uncertainty quantification (UQ) for inverse problems. The text discusses more advanced material on Bayesian methods and UQ, including Markov chain Monte Carlo sampling methods for UQ in inverse problems. Each chapter contains the Matlab code, which implements the algorithms and generates the figures as well as a large number of exercises. This material is intended for graduate students, researchers, and applied scientists. It is appropriate for use with courses on computational inverse problems, Bayesian methods for inverse problems, and UQ methods for inverse problems.

Uncertainty quantification for inverse problems with weak partial-differential-equation constraints

In statistical inverse problems, the objective is a complete statistical description of unknown parameters from noisy observations to quantify uncertainties in unknown parameters. We consider inverse problems with partial-differential-equation (PDE) constraints, which are applicable to many seismic problems. Bayesian inference is one of the most widely used approaches to precisely quantify statistics through a posterior distribution, incorporating uncertainties in observed data, modeling kernel, and prior knowledge of parameters. Typically when formulating the posterior distribution, the PDE constraints are required to be exactly satisfied, resulting in a highly nonlinear forward map and a posterior distribution with many local maxima. These drawbacks make it difficult to find an appropriate approximation for the posterior distribution. Another complicating factor is that traditional Markov chain Monte Carlo (MCMC) methods are known to converge slowly for realistically sized problems. To overcome these drawbacks, we relax the PDE constraints by introducing an auxiliary variable, which allows for Gaussian errors in the PDE and yields a bilinear posterior distribution with weak PDE constraints that is more amenable to uncertainty quantification because of its special structure. We determine that for a particular range of variance choices for the PDE misfit term, the new posterior distribution has fewer modes and can be well-approximated by a Gaussian distribution, which can then be sampled in a straightforward manner. Because it is prohibitively expensive to explicitly construct the dense covariance matrix of the Gaussian approximation for problems with more than [Formula: see text] unknowns, we have developed a method to implicitly construct it, which enables efficient sampling. We apply this framework to 2D seismic inverse problems with 1800 and 92,455 unknown parameters. The results illustrate that our framework can produce comparable statistical quantities with those produced by conventional MCMC-type methods while requiring far fewer PDE solves, which are the main computational bottlenecks in these problems.

Utilização de Modelos Autoregressivos na Quantificação de Incertezas em Problemas de Transporte Linear

The goal of this work is to study the uncertainty quantification in inverseproblems of miscible flows in heterogeneous porous media in Bayesian framework and propose a permeability update based on observed measurements of spatially sparse tracer concentration at certain times. The Successive Sum of Independent Gaussian Fields method is used to parametrization of the uncertainty in heterogeneous porous media. A two-stageMarkov chain Monte Carlo (MCMC) method is used to solving the inverse problem. For the construction of Markov chains is used the Metropolis-Hastings algorithm based on a random walk sampling process employs an auto-regressive instrumental distribution. The main idea of MCMC is to generate a Markov chain with an equilibrium distribution equal to the posterior distribution of interest. Numerical results are presented for a set sampled realizations of the permeability fields.

Standard error computations for uncertainty quantification in inverse problems: Asymptotic theory vs. bootstrapping

We computationally investigate two approaches for uncertainty quantification in inverse problems for nonlinear parameter dependent dynamical systems. We compare the bootstrapping and asymptotic theory approaches for problems involving data with several noise forms and levels. We consider both constant variance absolute error data and relative error, which produce non-constant variance data in our parameter estimation formulations. We compare and contrast parameter estimates, standard errors, confidence intervals, and computational times for both bootstrapping and asymptotic theory methods.

Bayesian uncertainty quantification for flows in heterogeneous porous media using reversible jump Markov chain Monte Carlo methods

In this paper, we study the uncertainty quantification in inverse problems for flows in heterogeneous porous media. Reversible jump Markov chain Monte Carlo algorithms (MCMC) are used for hierarchical modeling of channelized permeability fields. Within each channel, the permeability is assumed to have a log-normal distribution. Uncertainty quantification in history matching is carried out hierarchically by constructing geologic facies boundaries as well as permeability fields within each facies using dynamic data such as production data. The search with Metropolis–Hastings algorithm results in very low acceptance rate, and consequently, the computations are CPU demanding. To speed-up the computations, we use a two-stage MCMC that utilizes upscaled models to screen the proposals. In our numerical results, we assume that the channels intersect the wells and the intersection locations are known. Our results show that the proposed algorithms are capable of capturing the channel boundaries and describe the permeability variations within the channels using dynamic production history at the wells.

Expanded uncertainty quantification in inverse problems: Hierarchical Bayes and empirical Bayes

A common way to account for uncertainty in inverse problems is to apply Bayes' rule and obtain a posterior distribution of the quantities of interest given a set of measurements. A conventional Bayesian treatment, however, requires assuming specific values for parameters of the prior distribution and of the distribution of the measurement errors (e.g., the standard deviation of the errors). In practice, these parameters are often poorly known a priori, and choosing a particular value is often problematic. Moreover, the posterior uncertainty is computed assuming that these parameters are fixed; if they are not well known a priori, the posterior uncertainties have dubious value.This paper describes extensions to the conventional Bayesian treatment that assign uncertainty to the parameters defining the prior distribution and the distribution of the measurement errors. These extensions are known in the statistical literature as “empirical Bayes” and “hierarchical Bayes.” We demonstrate the practical application of these approaches to a simple linear inverse problem: using seismic traveltimes measured by a receiver in a well to infer compressional wave slowness in a 1D earth model. These procedures do not require choosing fixed values for poorly known parameters and, at most, need a realistic range (e.g., a minimum and maximum value for the standard deviation of the measurement errors). Inversion is thus made easier for general users, who are not required to set parameters they know little about.