Regularized Gauss-Newton Method Research Articles

A stochastic iteratively regularized Gauss-Newton method

Abstract This work focuses on developing and motivating a stochastic version of a well-known inverse problem methodology. Specifically, we consider the iteratively regularized Gauss-Newton method, originally proposed by Bakushinskii for infinite-dimensional problems. Recent work have extended this method to handle sequential observations, rather than a single instance of the data, demonstrating notable improvements in reconstruction accuracy. In this paper, we further extend these methods to a stochastic framework through mini-batching, introducing a new algorithm, the stochastic iteratively regularized Gauss-Newton method (SIRGNM). Our algorithm is designed through the use randomized sketching. We provide an analysis for the SIRGNM, which includes a preliminary error decomposition and a convergence analysis, related to the residuals. We provide numerical experiments on a 2D elliptic PDE example. This illustrates the effectiveness of the SIRGNM, through maintaining a similar level of accuracy while reducing on the computational time.

Application of the iteratively regularized Gauss–Newton method to parameter identification problems in Computational Fluid Dynamics

Field Inversion and Machine Learning is an active field of research in Computational Fluid Dynamics (CFD). This approach can be leveraged to obtain a closed-form correction for a given turbulence model to improve the predictions. The fundamental approach is to insert a parameter into the system of RANS equations and determine it in a way such that, for example, a given pressure distribution is better approximated compared to the one obtained with the original set of equations. The goal of this article is twofold. Numerical arguments are presented that these kinds of problems can be severely ill-posed. In the second part, an approach is presented to directly reconstruct the turbulent viscosity field along with an example. The Iteratively Regularized Gauss-Newton Method (IRGNM) is used for a realization. The construction of a problem-adapted norm for a finite volume method is presented. Finally, an outlook is presented on how this approach can be used to possibly modify or improve turbulence models such that not only one, but a larger number of test cases are considered.

Reconstruction of inhomogeneous media by an iteration algorithm with a learned projector

This paper is concerned with the inverse problem of reconstructing an inhomogeneous medium from the acoustic far-field data at a fixed frequency in two dimensions. This inverse problem is severely ill-posed (and also strongly nonlinear), and certain regularization strategy is thus needed. However, it is difficult to select an appropriate regularization strategy which should enforce some a priori information of the unknown scatterer. To address this issue, we plan to use a deep learning approach to learn some a priori information of the unknown scatterer from certain ground truth data, which is then combined with a traditional iteration method to solve the inverse problem. Specifically, we propose a deep learning-based iterative reconstruction algorithm for the inverse problem, based on a repeated application of a deep neural network and the iteratively regularized Gauss–Newton method (IRGNM). Our deep neural network (called the learned projector in this paper) mainly focuses on learning the a priori information of the shape of the unknown contrast with a normalization technique in the training processes and is trained to act like a projector which is helpful for projecting the solution into some feasible region. Extensive numerical experiments show that our reconstruction algorithm provides good reconstruction results even for the high contrast case and has a satisfactory generalization ability.

Improved quantification in CEST-MRI by joint spatial total generalized variation.

In this work, the use of joint Total Generalized Variation (TGV) regularization to improve Multipool-Lorentzian fitting of chemical exchange saturation transfer (CEST) Spectra in terms of stability and parameter signal-to-noise ratio (SNR) was investigated. The joint TGV term was integrated into the nonlinear parameter fitting problem. To increase convergence and weight the gradients, preconditioning using a voxel-wise singular value decomposition was applied to the problem, which was then solved using the iteratively regularized Gauss-Newton method combined with a Primal-Dual splitting algorithm. The TGV method was evaluated on simulated numerical phantoms, 3T phantom data and 7T in vivo data with respect to systematic errors and robustness. Three reference methods were also implemented: The standard nonlinear fitting, a method using a nonlocal-means filter for denoising and the pyramid scheme, which uses downsampled images to acquire accurate start values. The proposed regularized fitting method showed significantly improved robustness (compared to the reference methods). In testing, over a range of SNR values the TGV fit outperformed the other methods and showed accurate results even for large amounts of added noise. Parameter values found were closer or comparable to the ground truth. For in vivo datasets, the added regularization increased the parameter map SNR and prevented instabilities. The proposed fitting method using TGV regularization leads to improved results over a range of different data-sets and noise levels. Furthermore, it can be applied to all Z-spectrum data, with different amounts of pools, where the improved SNR and stability can increase diagnostic confidence.

On a Dynamic Variant of the Iteratively Regularized Gauss–Newton Method with Sequential Data

On a Dynamic Variant of the Iteratively Regularized Gauss–Newton Method with Sequential Data

Implicit Solutions of the Electrical Impedance Tomography Inverse Problem in the Continuous Domain with Deep Neural Networks.

Electrical impedance tomography (EIT) is a non-invasive imaging modality used for estimating the conductivity of an object Ω from boundary electrode measurements. In recent years, researchers achieved substantial progress in analytical and numerical methods for the EIT inverse problem. Despite the success, numerical instability is still a major hurdle due to many factors, including the discretization error of the problem. Furthermore, most algorithms with good performance are relatively time consuming and do not allow real-time applications. In our approach, the goal is to separate the unknown conductivity into two regions, namely the region of homogeneous background conductivity and the region of non-homogeneous conductivity. Therefore, we pose and solve the problem of shape reconstruction using machine learning. We propose a novel and simple jet intriguing neural network architecture capable of solving the EIT inverse problem. It addresses previous difficulties, including instability, and is easily adaptable to other ill-posed coefficient inverse problems. That is, the proposed model estimates the probability for a point of whether the conductivity belongs to the background region or to the non-homogeneous region on the continuous space Rd∩Ω with d∈{2,3}. The proposed model does not make assumptions about the forward model and allows for solving the inverse problem in real time. The proposed machine learning approach for shape reconstruction is also used to improve gradient-based methods for estimating the unknown conductivity. In this paper, we propose a piece-wise constant reconstruction method that is novel in the inverse problem setting but inspired by recent approaches from the 3D vision community. We also extend this method into a novel constrained reconstruction method. We present extensive numerical experiments to show the performance of the architecture and compare the proposed method with previous analytic algorithms, mainly the monotonicity-based shape reconstruction algorithm and iteratively regularized Gauss-Newton method.

Comparison of iteratively regularized Gauss-Newton method with Adam optimization for image reconstruction in electrical impedance tomography

Comparison of iteratively regularized Gauss-Newton method with Adam optimization for image reconstruction in electrical impedance tomography

Large-Scale Inversion of Magnetotelluric Data Using Regularized Gauss–Newton Method in the Data Space

Large-Scale Inversion of Magnetotelluric Data Using Regularized Gauss–Newton Method in the Data Space

Defect reconstruction in a two-dimensional semi-analytical waveguide model via derivative-based optimization.

This paper considers an indirect measurement approach to reconstruct a defect in a two-dimensional waveguide model for a non-destructive ultrasonic inspection via derivative-based optimization. The propagation of the mechanical waves is simulated by the scaled boundary finite element method that builds on a semi-analytical approach. The simulated data are then fitted to given data associated with the reflected waves from a defect which is to be reconstructed. For this purpose, we apply an iteratively regularized Gauss-Newton method in combination with algorithmic differentiation to provide the required derivative information accurately and efficiently. We present numerical results for three kinds of defects, namely, a crack, delamination, and corrosion. The objective function and the properties of the reconstruction method are investigated. The examples show that the parameterization of the defect can be reconstructed efficiently as well as robustly in the presence of noise.

Convergence analysis of iteratively regularized Gauss–Newton method with frozen derivative in Banach spaces

Abstract In this paper, we consider the iteratively regularized Gauss–Newton method with frozen derivative and formulate its convergence rates in the settings of Banach spaces. The convergence rates of iteratively regularized Gauss–Newton method with frozen derivative are well studied via generalized source conditions. We utilize the recently developed concept of conditional stability of the inverse mapping to derive the convergence rates. Also, in order to show the practicality of this paper, we show that our results are applicable on an ill-posed inverse problem. Finally, we compare the convergence rates derived in this paper with the existing convergence rates in the literature.

Error estimates for the simplified iteratively regularized Gauss–Newton method under a general source condition

Error estimates for the simplified iteratively regularized Gauss–Newton method under a general source condition

A simplified iteratively regularized projection method for nonlinear ill-posed problems

In this paper, we propose a projection method for the numerical solution of the simplified iteratively regularized Gauss-Newton method of nonlinear integral equations for which an a posteriori stopping rule is proposed to terminate the iteration. Such methods require only the computation of the Fréchet derivative at the initial approximation. Thus the computational work is considerably reduced. Under certain mild conditions, we give the convergence analysis and derive the order optimality. Numerical results are presented to demonstrate the efficiency and accuracy of the proposed method.

Shadowing-Based Data Assimilation Method for Partially Observed Models

In this article we develop further an algorithm for data assimilation based upon a shadowing refinement technique [de Leeuw et al., SIAM J. Appl. Dyn. Sys., 17 (2018)] to take partial observations into account. Our method is based on regularized Gauss-Newton method. We prove local convergence to the solution manifold and provide a lower bound on the algorithmic time step. We use numerical experiments with the Lorenz 63 and Lorenz 96 models to illustrate convergence of the algorithm and show that the results compare favourably with a variational technique (weak-constraint four-dimensional variational method) and a shadowing technique (pseudo-orbit data assimilation). Numerical experiments show that a preconditioner chosen based on a cost function allows the algorithm to find an orbit of the dynamical system in the vicinity of the true solution.

Porosity reconstruction based on Biot elastic model of porous media by homotopy perturbation method

Fluid-saturated porous media are two-phase media, which are composed of solid and liquid phases. Biot theory for fluid-saturated porous media holds that underground media are composed of porous elastic solid and compressible viscous fluid filled with pore space. Compared with the single-phase media theory, the fluid-saturated porous media theory can describe the subsurface media more precisely, and the elastic wave equations in the fluid-saturated porous media contain more parameters used to describe the formation properties. Therefore, fluid-saturated porous media theory is widely used in geophysical exploration, seismic engineering, and other fields.This paper considers the porosity reconstruction problem of the elastic wave equations in the fluid-saturated porous media based on Biot theory, which is a typically nonlinear ill-posed inverse problem from mathematical viewpoint. The proposed method is a novel iteration regularization scheme based on the homotopy perturbation technique. To verify the validity and applicability, numerical experiments of two-dimensional and three-dimensional porosity models have been carried out. Numerical results illustrate that this method can overcome the numerical instability and are robust to data noise in the reconstruction procedure. Furthermore, compared with the classical regularized Gauss-Newton method, the homotopy perturbation method greatly widens the convergence region while keeping the fast convergence rate.

Accuracy Estimation for a Class of Iteratively Regularized Gauss–Newton Methods with a posteriori Stopping Rule

Accuracy Estimation for a Class of Iteratively Regularized Gauss–Newton Methods with a posteriori Stopping Rule

Analysis of a generalized regularized Gauss–Newton method under heuristic rule in Banach spaces

In this work, we propose a generalized regularized Gauss–Newton method to solve nonlinear inverse problems in Banach spaces and consider its heuristic stopping rule. The proposed method only requires a generalized operator which could be independent of the Fréchet derivative of the forward operator, thus it can solve smooth as well as non-smooth inverse problems. In addition, the proposed heuristic rule is fully data driven and does not require any information on the noise level. Therefore, it can cope with inverse problems whose noise levels are unknown. An a posteriori error for this method under heuristic rule is obtained and general convergence result is given under some standard assumptions. Numerical experiments are presented to illustrate the efficiency of the proposed method.

Iteratively regularized Gauss-Newton method in the inverse problem of ionospheric radiosonding

Iteratively regularized Gauss-Newton method in the inverse problem of ionospheric radiosonding

Convergence rates for iteratively regularized Gauss–Newton method subject to stability constraints

In this paper we formulate the convergence rates of the iteratively regularized Gauss–Newton method by defining the iterates via convex optimization problems in a Banach space setting. We employ the concept of conditional stability to deduce the convergence rates in place of the well known concept of variational inequalities. To validate our abstract theory, we also discuss an ill-posed inverse problem that satisfies our assumptions. We also compare our results with the existing results in the literature.

Projected finite dimensional iteratively regularized Gauss–Newton method with a posteriori stopping for the ionospheric radiotomography problem

We investigate a class of finite dimensional iteratively regularized Gauss–Newton methods for solving nonlinear irregular operator equations in a Hilbert space. The developed technique allows to investigate in a uniform style various discretization methods such as projection, quadrature and collocation schemes and to take into account available restrictions on the solution. We propose an a posteriori stopping rule for the iterative process and establish an accuracy estimate for obtained approximation. The regularized Gauss–Newton method combined with the quadrature discretization and the a posteriori iteration stopping is applied to a model ionospheric radiotomography problem. The problem is reduced to a nonlinear integral equation describing the phase shift of a sounding radio signal in dependence of the free electron concentration in the ionospheric plasma. We establish the unique solvability of the inverse problem in the class of analytic functions.

A Proof-of-Concept Algorithm for the Retrieval of Total Column Amount of Trace Gases in a Multi-Dimensional Atmosphere

An algorithm for the retrieval of total column amount of trace gases in a multi-dimensional atmosphere is designed. The algorithm uses (i) certain differential radiance models with internal and external closures as inversion models, (ii) the iteratively regularized Gauss–Newton method as a regularization tool, and (iii) the spherical harmonics discrete ordinate method (SHDOM) as linearized radiative transfer model. For efficiency reasons, SHDOM is equipped with a spectral acceleration approach that combines the correlated k-distribution method with the principal component analysis. The algorithm is used to retrieve the total column amount of nitrogen for two- and three-dimensional cloudy scenes. Although for three-dimensional geometries, the computational time is high, the main concepts of the algorithm are correct and the retrieval results are accurate.