Ill-posed Nonlinear Inverse Problem Research Articles

Convergence analysis of Tikhonov regularization for non-linear statistical inverse problems

We study a non-linear statistical inverse problem, where we observe the noisy image of a quantity through a non-linear operator at some random design points. We consider the widely used Tikhonov regularization (or method of regularization) approach to estimate the quantity for the non-linear ill-posed inverse problem. The estimator is defined as the minimizer of a Tikhonov functional, which is the sum of a data misfit term and a quadratic penalty term. We develop a theoretical analysis for the minimizer of the Tikhonov regularization scheme using the concept of reproducing kernel Hilbert spaces. We discuss optimal rates of convergence for the proposed scheme, uniformly over classes of admissible solutions, defined through appropriate source conditions.

On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

In this work, we analyze the regularizing property of the stochastic gradient descent for the efficient numerical solution of a class of nonlinear ill-posed inverse problems in Hilbert spaces. At each step of the iteration, the method randomly chooses one equation from the nonlinear system to obtain an unbiased stochastic estimate of the gradient, and then performs a descent step with the estimated gradient. It is a randomized version of the classical Landweber method for nonlinear inverse problems, and it is highly scalable to the problem size and holds significant potentials for solving large-scale inverse problems. Under the canonical tangential cone condition, we prove the regularizing property for a priori stopping rules, and then establish the convergence rates under suitable sourcewise condition and range invariance condition.

Convergence analysis of (statistical) inverse problems under conditional stability estimates

Conditional stability estimates require additional regularization for obtaining stable approximate solutions if the validity area of such estimates is not completely known. In this context, we consider ill-posed nonlinear inverse problems in Hilbert scales satisfying conditional stability estimates characterized by general concave index functions. For that case, we exploit Tikhonov regularization and provide convergence and convergence rates of regularized solutions for both deterministic and stochastic noise. We further discuss a priori and a posteriori parameter choice rules and illustrate the validity of our assumptions in different model and real world situations.

Body coil reference for inverse reconstructions of multi-coil data-the case for real-time MRI.

Real-time magnetic resonance imaging (MRI) or model-based MRI reconstructions of parametric maps require the solution of an ill-posed nonlinear inverse problem. Respective algorithms, e.g., the iteratively regularized Gauss-Newton method, implicitly combine datasets from multiple receive coils. Because these local coils may exhibit complex sensitivity profiles with rather different phase offsets, the numerical optimization may lead to phase singularities which in turn cause "black holes" in magnitude images. The purpose of this work is to develop a method for inverse reconstructions of multi-coil MRI data which avoids the generation of such spatially selective phase singularities. It is proposed to use volumetric body coil data and start the iterative reconstruction of multi-coil data with a reference image which offers proper phase information. In more detail, inverse reconstructions of multi-coil data are initialized with a complex "seed" image which is obtained by a Fast Fourier Transform (FFT) reconstruction of data from a single body coil element. This is accomplished at no additional cost as only very few body coil scans with identical conditions as the multi-coil acquisitions are needed as part of the regular prep scan period. The method is evaluated for anatomical real-time MRI and model-based phase-contrast flow MRI in real-time at 3 T. The proposed method overcomes phase singularities in all cases for arbitrary sets of receive coils. In conclusion, the automatic use of a single body coil reference image is simple, robust, and further improves the reliability of advanced MRI reconstructions from multi-coil data.

Quadratic convergence of Levenberg-Marquardt method for general nonlinear inverse problems with two parameters

In this paper, we shall present the quadratic convergence of Levenberg-Marquardt (L-M) method for solving general ill-posed nonlinear inverse problems including two parameters to be identified. As the Tikhonov regularized minimizations are non-convex due to the nonlinearity of the nonlinear inverse problems, the L-M method transforms them into convex minimizations. The quadratic convergence of the L-M method is rigorously demonstrated under some basic assumptions. An application of the L-M method for the simultaneous identification of the Robin coefficient and heat flux in an elliptic system is presented, and the surrogate functional technique is introduced to solve the convex but still strongly ill-conditioned minimizations, resulting in an explicit minimizer of the minimization at each L-M iteration. Numerical experiments are presented to illustrate the efficiency and accuracy of the proposed algorithm.

Penalized sieve GEL for weighted average derivatives of nonparametric quantile IV regressions

This paper considers estimation and inference for a weighted average derivative (WAD) of a nonparametric quantile instrumental variables regression (NPQIV). NPQIV is a non-separable and nonlinear ill-posed inverse problem, which might be why there is no published work on the asymptotic properties of any estimator of its WAD. We first characterize the semiparametric efficiency bound for a WAD of a NPQIV, which, unfortunately, depends on an unknown conditional derivative operator and hence an unknown degree of ill-posedness, making it difficult to know if the information bound is singular or not. In either case, we propose a penalized sieve generalized empirical likelihood (GEL) estimation and inference procedure, which is based on the unconditional WAD moment restriction and an increasing number of unconditional moments that are implied by the conditional NPQIV restriction, where the unknown quantile function is approximated by a penalized sieve. Under some regularity conditions, we show that the self-normalized penalized sieve GEL estimator of the WAD of a NPQIV is asymptotically standard normal. We also show that the quasi likelihood ratio statistic based on the penalized sieve GEL criterion is asymptotically chi-square distributed regardless of whether or not the information bound is singular.

Non-negative constrained inverse eigenvalue problems – Application to damage identification

Abstract Damage identification using eigenvalue shifts is ill-posed because the number of identifiable eigenvalues is typically far less than the number of potential damage locations. This paper shows that if damage is defined by sparse and non-negative vectors, such as the case for local stiffness reductions, then the non-negative solution to the linearized inverse eigenvalue problem can be made unique with respect to a subset of eigenvalues significantly smaller than the number of potential damage locations. Theoretical evidence, numerical simulations, and performance comparisons to sparse vector recovery methods based on l 1 -norm optimization are used to validate the findings. These results are then extrapolated to the ill-posed nonlinear inverse eigenvalue problem in cases where damage is large, and linearization induces non-negligible truncation errors. In order to approximate the solution to the non-negative nonlinear least squares, a constrained finite element model updating approach is presented. The proposed method is verified using three simulated structures of increasing complexity: a one-dimensional shear beam, a planar truss, and a three-dimensional space structure. For multiple structures, this paper demonstrates that the proposed method finds sparse solutions in the presence of measurement noise.

Incorporating structural prior information and sparsity into EIT using parallel level sets

EIT is a non-linear ill-posed inverse problem which requires sophisticated regularisation techniques to achieve good results. In this paper we consider the use of structural information in the form of edge directions coming from an auxiliary image of the same object being reconstructed. In order to allow for cases where the auxiliary image does not provide complete information we consider in addition a sparsity regularization for the edges appearing in the EIT image. The combination of these approaches is conveniently described through the parallel level sets approach. We present an overview of previous methods for structural regularisation and then provide a variational setting for our approach and explain the numerical implementation. We present results on simulations and experimental data for different cases with accurate and inaccurate prior information. The results demonstrate that the structural prior information improves the reconstruction accuracy, even in cases when there is reasonable uncertainty in the prior about the location of the edges or only partial edge information is available.

A Learning-Based Method for Solving Ill-Posed Nonlinear Inverse Problems: A Simulation Study of Lung EIT

This paper proposes a new approach for solving ill-posed nonlinear inverse problems. For ease of explanation of the proposed approach, we use the example of lung electrical impedance tomography (EIT), which is known to be a nonlinear and ill-posed inverse problem. Conventionally, penalty-based regularization methods have been used to deal with the ill-posed problem. However, experiences over the last three decades have shown methodological limitations in utilizing prior knowledge about tracking expected imaging features for medial diagnosis. The proposed method's paradigm is completely different from conventional approaches; the proposed reconstruction uses a variety of training data sets to generate a low dimensional manifold of approximate solutions, which allows to convert the ill-posed problem to a well-posed one. Variational autoencoder was used to produce a compact and dense representation for lung EIT images with a low dimensional latent space. Then, we learn a robust connection between the EIT data and the low-dimensional latent data. Numerical simulations validate the effectiveness and feasibility of the proposed approach.

Simultaneous identification of volatility and interest rate functions-a two-parameter regularization approach

This paper investigates a specific ill-posed nonlinear inverse problem that arises in financial markets. Precisely, as a benchmark problem in the context of volatility surface calibration, we consider the simultaneous recovery of implied volatility and interest rate functions over a finite time interval from corresponding call- and put-price functions for idealized continuous families of European vanilla options over the same maturity interval. We prove identifiability of the pair of functions to be identified by showing injectivity of the forward operator in L2-spaces. To overcome the ill-posedness we employ a two-parameter Tikhonov regularization with heuristic parameter choice rules and demonstrate chances and limitations by means of numerical case studies using synthetic data.

Pattern-based calibration of complex subsurface flow models against dynamic response data

Abstract Calibration of heterogeneous subsurface flow models usually leads to ill-posed nonlinear inverse problems, where too many unknown parameters are estimated from limited response measurements. When the underlying parameters form complex (non-Gaussian) structured spatial connectivity patterns, classical variogram-based geostatistical techniques cannot describe the underlying distributions. Modern pattern-based geostatistical methods that incorporate higher-order spatial statistics are more suitable for describing such complex spatial patterns. Moreover, when the unknown parameters are discrete (e.g., geologic facies distribution), conventional model calibration techniques that are designed for continuous parameters cannot be applied directly. In this paper, we introduce a novel pattern-based model calibration method to reconstruct spatially complex facies distributions from dynamic flow response data. To reproduce complex connectivity patterns during model calibration, we impose a geologic feasibility constraint that ensures the solution honors the expected higher-order spatial statistics. For model calibration, we adopt a regularized least-squares formulation, involving (i) data mismatch, (ii) pattern connectivity, and (iii) feasibility constraint terms. Using an alternating directions optimization algorithm, the regularized objective function is divided into a parameterized model calibration sub-problem, which is solved via gradient-based optimization. The resulting parameterized solution is then mapped onto the feasible set, using the k-nearest neighbors (k-NN) as a supervised machine learning approach, to honor the expected spatial statistics. The two steps of the model calibration formulation are repeated until the convergence criterion is met. Several numerical examples are used to evaluate the performance of the developed method.

Deep D-Bar: Real-Time Electrical Impedance Tomography Imaging With Deep Neural Networks.

The mathematical problem for electrical impedance tomography (EIT) is a highly nonlinear ill-posed inverse problem requiring carefully designed reconstruction procedures to ensure reliable image generation. D-bar methods are based on a rigorous mathematical analysis and provide robust direct reconstructions by using a low-pass filtering of the associated nonlinear Fourier data. Similarly to low-pass filtering of linear Fourier data, only using low frequencies in the image recovery process results in blurred images lacking sharp features, such as clear organ boundaries. Convolutional neural networks provide a powerful framework for post-processing such convolved direct reconstructions. In this paper, we demonstrate that these CNN techniques lead to sharp and reliable reconstructions even for the highly nonlinear inverse problem of EIT. The network is trained on data sets of simulated examples and then applied to experimental data without the need to perform an additional transfer training. Results for absolute EIT images are presented using experimental EIT data from the ACT4 and KIT4 EIT systems.

Analysis of the iteratively regularized Gauss–Newton method under a heuristic rule

The iteratively regularized Gauss–Newton method is one of the most prominent regularization methods for solving nonlinear ill-posed inverse problems when the data is corrupted by noise. In order to produce a useful approximate solution, this iterative method should be terminated properly. The existing a priori and a posteriori stopping rules require accurate information on the noise level, which may not be available or reliable in practical applications. In this paper we propose a heuristic selection rule for this regularization method, which requires no information on the noise level. By imposing certain conditions on the noise, we derive a posteriori error estimates on the approximate solutions under various source conditions. Furthermore, we establish a convergence result without using any source condition. Numerical results are presented to illustrate the performance of our heuristic selection rule.

Accelerating nonlinear reconstruction in laminar optical tomography by use of recursive SVD inversion.

Image reconstruction in the most model-based biophotonic imaging modalities essentially poses an ill-posed nonlinear inverse problem, which has been effectively tackled in the diffusion-approximation-satisfied scenarios such as diffuse optical tomography. Nevertheless, a nonlinear implementation in high-resolution laminar optical tomography (LOT) is normally computationally-costly due to its strong dependency on a dense source-detector configuration and a physically-rigorous photon-transport model. To circumvent the adversity, we herein propose a practical nonlinear LOT approach to the absorption reconstruction. The scheme takes advantage of the numerical stability of the singular value decomposition (SVD) for the ill-posed linear inversion, and is accelerated by adopting an explicitly recursive strategy for the time-consuming repeated SVD inversion, which is based on a scaled expression of the sensitivity matrix. Experiments demonstrate that the proposed methodology can perform as well as the traditional nonlinear one, while the computation time of the former is merely 26.27% of the later on average.

Spectral CT Material Decomposition in the Presence of Poisson Noise: A Kullback–Leibler Approach

Context: Spectral Computed Tomography (SPCT) acquires energy-resolved projections that can be used to evaluate the concentrations of the different materials of an object. This can be achieved by decomposing the projection images in a material basis, in a first step, followed by a standard tomographic reconstruction step. However, material decomposition is a challenging non-linear ill-posed inverse problem, which is very sensitive to the photonic noise corrupting the data.Methods: In this paper, material decomposition is solved in a variational framework. We investigate two fidelity terms: a weighted least squares (WLS) term adapted to Gaussian noise and the Kullback–Leibler (KL) distance adapted to Poisson Noise. The decomposition problem is then solved iteratively by a regularized Gauss–Newton algorithm.Results: Simulations are performed in a numerical mouse phantom and the decompositions show that KL outperforms WLS for low numbers of photons, i.e., for high photonic noise.Conclusion: In this work, a new method for material decomposition in SPCT is presented. It is based on the KL distance to take into account the noise statistics. This new method could be of particular interest when low-dose acquisitions are performed.

An integrated solution for reducing ill-conditioning and testing the results in non-linear 3D similarity transformations

To find 3D similarity transformation parameters (a scale, three rotational angles, and three translation elements) between two orthogonal coordinate systems in 3D is an ill-posed non-linear inverse problem by means of common points (their Cartesian components are known in the both systems). The problem can be solved via Linearized Least Squares (LLS) or Direct (non-iterative) Least Squares (DLS). Since the parameters in LLS take different quantities (and units) from each other, the condition problems can arise during the solution of normal equations. In this paper, we propose a combined solution to reducing ill-conditioning and to perform precision analysis and global outlier test in LLS accordingly. The way is based on column norming and uses the normalized unknowns instead of the original ones at the solution stage of the normal equations. While the global outlier test is fulfilled on the normalized unknowns, the original unknowns and their precisions obtained using the normalized matrix with linear transformation to the normalized unknowns and by the error propagation law to their variance-covariance matrix. A direct method (Direct Least Squares, DLS) is used to compute the initial values of LLS for reducing iteration number (to 1–3 cycles) with a 1.E-7 threshold in the paper. And, when the solution of 3D-ST having large rotational angles and uncertainties, it is also shown that DLS goes to a worse solution than LLS by means of a simulated transformation problem.

Incorporating a Spatial Prior into Nonlinear D-Bar EIT Imaging for Complex Admittivities.

Electrical Impedance Tomography (EIT) aims to recover the internal conductivity and permittivity distributions of a body from electrical measurements taken on electrodes on the surface of the body. The reconstruction task is a severely ill-posed nonlinear inverse problem that is highly sensitive to measurement noise and modeling errors. Regularized D-bar methods have shown great promise in producing noise-robust algorithms by employing a low-pass filtering of nonlinear (nonphysical) Fourier transform data specific to the EIT problem. Including prior data with the approximate locations of major organ boundaries in the scattering transform provides a means of extending the radius of the low-pass filter to include higher frequency components in the reconstruction, in particular, features that are known with high confidence. This information is additionally included in the system of D-bar equations with an independent regularization parameter from that of the extended scattering transform. In this paper, this approach is used in the 2-D D-bar method for admittivity (conductivity as well as permittivity) EIT imaging. Noise-robust reconstructions are presented for simulated EIT data on chest-shaped phantoms with a simulated pneumothorax and pleural effusion. No assumption of the pathology is used in the construction of the prior, yet the method still produces significant enhancements of the underlying pathology (pneumothorax or pleural effusion) even in the presence of strong noise.

A convergent adaptive finite element method for electrical impedance tomography

In this work we develop and analyze an adaptive finite element method for efficiently solving electrical impedance tomography -- a severely ill-posed nonlinear inverse problem for recovering the conductivity from boundary voltage measurements. The reconstruction technique is based on Tikhonov regularization with a Sobolev smoothness penalty and discretizing the forward model using continuous piecewise linear finite elements. We derive an adaptive finite element algorithm with an a posteriori error estimator involving the concerned state and adjoint variables and the recovered conductivity. The convergence of the algorithm is established, in the sense that the sequence of discrete solutions contains a convergent subsequence to a solution of the optimality system for the continuous formulation. Numerical results are presented to verify the convergence and efficiency of the algorithm.

Stochastic projection methods and applications to some nonlinear inverse problems of phase retrieving

In this short paper we present a stochastic projection based Monte Carlo algorithm for solving a nonlinear ill-posed inverse problem of recovering the phase of a complex-valued function provided its absolute value is known, under some additional information. The method is developed here for retrieving the step structure of the epitaxial films from the X-ray diffraction analysis. We suggest to extract some additional information from the measurements which makes the problem well-posed, and with this information, the method suggested works well even for noisy measurements. Results of simulations for a layer structure recovering problem with 26 sublayers are presented.

Landweber-Kaczmarz method in Banach spaces with inexact inner solvers

In recent years the Landweber-Kaczmarz method has been proposed for solving nonlinear ill-posed inverse problems in Banach spaces using general convex penalty functions. The implementation of this method involves solving a (nonsmooth) convex minimization problem at each iteration step and the existing theory requires its exact resolution which in general is impossible in practical applications. In this paper we propose a version of the Landweber-Kaczmarz method in Banach spaces in which the minimization problem involved in each iteration step is solved inexactly. Based on the -subdifferential calculus we give a convergence analysis of our method. Furthermore, using Nesterov's strategy, we propose a possible accelerated version of the Landweber-Kaczmarz method. Numerical results on computed tomography and parameter identification in partial differential equations are provided to support our theoretical results and to demonstrate our accelerated method.