A Bayesian Conjugate Gradient Method (with Discussion)

Ilse C.F Ipsen,Mark Girolami,Chris J Oates,Jon Cockayne

doi:10.1214/19-ba1145

Abstract

A fundamental task in numerical computation is the solution of large linear systems. The conjugate gradient method is an iterative method which offers rapid convergence to the solution, particularly when an effective preconditioner is employed. However, for more challenging systems a substantial error can be present even after many iterations have been performed. The estimates obtained in this case are of little value unless further information can be provided about, for example, the magnitude of the error. In this paper we propose a novel statistical model for this error, set in a Bayesian framework. Our approach is a strict generalisation of the conjugate gradient method, which is recovered as the posterior mean for a particular choice of prior. The estimates obtained are analysed with Krylov subspace methods and a contraction result for the posterior is presented. The method is then analysed in a simulation study as well as being applied to a challenging problem in medical imaging.

Highlights

This paper presents an iterative method for solution of systems of linear equations of the form Ax∗ = b, (1)where A ∈ Rd×d is an invertible matrix and b ∈ Rd is a vector, each given, while x∗ ∈ Rd is to be determined
We argue that placing a prior on the solution space is more intuitive than existing probabilistic numerical methods and corresponds more directly with classical iterative methods
Second we present an application to electrical impedance tomography, a challenging medical imaging technique in which linear systems must be repeatedly solved

Summary

Introduction

Reinarz et al, 2018) Another example arises in computation with Gaussian measures (Bogachev, 1998; Rasmussen, 2004), in which analytic covariance functions, such as the exponentiated quadratic, give rise to challenging linear systems. This has an impact in a number of related fields, such as symmetric collocation solution of PDEs (Fasshauer, 1999; Cockayne et al, 2016), numerical integration (Larkin, 1972; Briol et al, 2018) and generation of spatial random fields (Besag and Green, 1993; Parker and Fox, 2012; Schafer et al, 2017). It is clear that there exist many important situations in which error in the solution of a linear system cannot practically be avoided

Objectives

Methods

Results

Discussion

Conclusion