Discrete Ill-posed Problems Research Articles

Priorconditioned CGLS-Based Quasi-MAP Estimate, Statistical Stopping Rule, and Ranking of Priors

We consider linear discrete ill-posed problems within the Bayesian framework, assuming a Gaussian additive noise model and a Gaussian prior whose covariance matrices may be known modulo multiplicative scaling factors. In that context, we propose a new pointwise estimator for the posterior density, the priorconditioned CGLS-based quasi-MAP (qMAP) as a computationally attractive approximation of the classical maximum a posteriori (MAP) estimate, in particular when the effective rank of the matrix ${\mathsf A}$ is much smaller than the dimension of the unknown. Exploiting the Bayesian paradigm and the connection between standard normal random variables and the $\chi^2$ distribution of their squared Euclidean norms, we propose a new stopping rule, the Max$\chi^2$ rule, to terminate the CGLS iterations in the computation of the qMAP estimate. Moreover, we show that the proposed stopping rule can be used to estimate the possibly unknown scaling factor of the noise covariance, which is tantamount to estimating the noise level. It is shown that the Max$\chi^2$ stopping rule and the proposed noise level estimation are affected by the correlation structure of priorconditioner, i.e., prior-related right preconditioner, but not by its scaling, which in the classical regularization framework would correspond to the Tikhonov regularization parameter. We will also show how the relation between priorconditioners and whitening transformations of the unknown can be used to rank different priorconditioners according to their agreement with the data.

Modulus-based iterative methods for constrained Tikhonov regularization

Tikhonov regularization is one of the most popular methods for the solution of linear discrete ill-posed problems. In many applications the desired solution is known to lie in the nonnegative cone. It is then natural to require that the approximate solution determined by Tikhonov regularization also lies in this cone. The present paper describes two iterative methods, that employ modulus-based iterative methods, to compute approximate solutions in the nonnegative cone of large-scale Tikhonov regularization problems. The first method considered consists of two steps: first the given linear discrete ill-posed problem is reduced to a small problem by a Krylov subspace method, and then the reduced Tikhonov regularization problems so obtained is solved. The second method described explores the structure of certain image restoration problems. Computed examples illustrate the performances of these methods.

A special generalized HSS method for discrete ill-posed problems

In this paper, we use a new special generalized Hermitian and skew-Hermitian splitting (SGHSS) method for solving ill-posed inverse problems. Based on an augmented system formulation, we apply a new splitting for the Hermitian part of the coefficient matrix and discuss on the convergence of this iterative method. Moreover, the optimal parameter minimizing the spectral radius of iteration matrix is derived. Finally, we present some numerical examples to demonstrate the performance of the presented method.

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent CGLS applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. In this paper, we establish bounds for the distance between the $k$-dimensional Krylov subspace and the $k$-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank $k$ approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.

Fourier Collocation Algorithm for identification of a spacewise dependent source in wave equation from Neumann-type measured data

Inverse problem of identifying the unknown spacewise dependent source F(x) in 1D wave equation utt=c2uxx+F(x)G(t)+h(x,t), (x,t)∈(0,1)×(0,T), from the Neumann-type measured output g(t):=ux(0,t) is investigated. Most studies have attempted to reconstruct an unknown spacewise dependent source F(x) from the final observation uT(x):=u(x,T). Since a boundary measured data is most feasible from an engineering viewpoint, the identification problem has wide applications, in particular, in electrical networks governed by harmonically varying source for the linear wave equation utt−uxx=F(x)cos(ωt), where ω>0 is the frequency and F(x) is an unknown source term. In this paper Fourier Collocation Algorithm for reconstructing the spacewise dependent source F(x) is developed. This algorithm is based on Fourier expansion of the direct problem solution applied to the minimization problem for Tikhonov functional, by taking then a partial N-sum of the Fourier expansion. Tikhonov regularization is then applied to the obtained discrete ill-posed problem. To obtain high quality reconstruction in large values of the noise level, a numerical filtering algorithm is used for smoothing the noisy data. As an application, we demonstrate the ability of the algorithm on benchmark problems, in particular, on source identification problem in electrical networks governed by mono-frequency source. Numerical results show that the proposed algorithm allows to reconstruct the spacewise dependent source F(x) with enough high accuracy, in the presence of high noise levels.

Simultaneous identification of bearing dynamic coefficients in a water-gas lubricated hydrostatic spindle with a big thrust disc

A water-gas lubricated hydrostatic spindle system with big thrust disc, supported by a water-lubricated journal bearing and a gaslubricated thrust bearing, is developed to improve spindle stiffness and to reduce friction power of spindle. A new identification algorithm is derivated to identify simultaneously the eighteen dynamic coefficients of the coupled journal and thrust bearings. A comparison with the available experimental results reveals that the bearing coefficients is well estimated by the present theoretical model. The results also indicate that, the estimated bearing parameters by regularization solution are in good agreement with the assumed values; while it has opposite effect by directed solution for discrete ill-posed problems. It is concluded that, the identification accuracy of bearing parameters is govern by the measured errors; contrast to unbalance mass error and unbalance phase error, the displacement error leads the maximal percentage deviations of estimated coefficients.

Multidirectional Subspace Expansion for One-Parameter and Multiparameter Tikhonov Regularization

Tikhonov regularization is a popular method to approximate solutions of linear discrete ill-posed problems when the observed or measured data is contaminated by noise. Multiparameter Tikhonov regularization may improve the quality of the computed approximate solutions. We propose a new iterative method for large-scale multiparameter Tikhonov regularization with general regularization operators based on a multidirectional subspace expansion. The multidirectional subspace expansion may be combined with subspace truncation to avoid excessive growth of the search space. Furthermore, we introduce a simple and effective parameter selection strategy based on the discrepancy principle and related to perturbation results.

Model Selection Criteria for a Linear Model to Solve Discrete Ill-Posed Problems on the Basis of Singular Decomposition and Random Projection

Criteria are developed to determine the optimal number of components of a linear model in solving a discrete ill-posed problem by the methods of truncated singular value decomposition and random projection. To this end, the behavior of dependencies of the error vector of the solution and the restoration error of the vector of the right side on the model dimensionality and their minima is investigated. An experimental investigation of the developed criteria was also pursued and its results are provided.

A stopping criterion for iterative regularization methods

We present a discrepancy-like stopping criterium for iterative regularization methods for the solution of linear discrete ill-posed problems. The presented criterium terminates the iterations of the iterative method when the residual norm of the computed solution becomes less or equal to the residual norm of a regularized Truncated Singular Value Decomposition (TSVD) solution. We present two algorithms for the automatic computation of the TSVD residual norm using the Discrete Picard Condition. The first algorithm uses the SVD coefficients while the second one uses the Fourier coefficients. In this work, we mainly focus on the Conjugate Gradient Least Squares method, but the proposed criterium can be used for terminating the iterations of any iterative regularization method. Many numerical tests on some selected one dimensional and image deblurring problems are presented and the results are compared with those obtained by state-of-the-art parameter selection rules. The numerical results show the efficiency and robustness of the proposed criterium.

A hybrid splitting method for smoothing Tikhonov regularization problem

In this paper, a hybrid splitting method is proposed for solving a smoothing Tikhonov regularization problem. At each iteration, the proposed method solves three subproblems. First of all, two subproblems are solved in a parallel fashion, and the multiplier associated to these two block variables is updated in a rapid sequence. Then the third subproblem is solved in the sense of an alternative fashion with the former two subproblems. Finally, the multiplier associated to the last two block variables is updated. Global convergence of the proposed method is proven under some suitable conditions. Some numerical experiments on the discrete ill-posed problems (DIPPs) show the validity and efficiency of the proposed hybrid splitting method.

Conjugate Gradient Method for the Solution of Inverse Problems: Application in Linear Seismic Tomography

We consider the conjugate gradient method for the normal equations in the solution of discrete ill-posed problems arising from seismic tomography. We use a linear approach of traveltime tomography that is characterized by an ill-conditioned linear system whose unknowns are the slownesses in each block of the computational domain. The algorithms considered in this work regularize the linear system by stopping the conjugate gradient method in an early iteration. They do not depend on the singular-value decomposition and represent an attractive and economic alternative for large-scale problems. We review two recently proposed stopping criteria and propose a modified stopping criterion that takes into account the oscillations in the approximate solution.

Regularization matrices determined by matrix nearness problems

This paper is concerned with the solution of large-scale linear discrete ill-posed problems with error-contaminated data. Tikhonov regularization is a popular approach to determine meaningful approximate solutions of such problems. The choice of regularization matrix in Tikhonov regularization may significantly affect the quality of the computed approximate solution. This matrix should be chosen to promote the recovery of known important features of the desired solution, such as smoothness and monotonicity. We describe a novel approach to determine regularization matrices with desired properties by solving a matrix nearness problem. The constructed regularization matrix is the closest matrix in the Frobenius norm with a prescribed null space to a given matrix. Numerical examples illustrate the performance of the regularization matrices so obtained.

A new framework for multi-parameter regularization

This paper proposes a new approach for choosing the regularization parameters in multi-parameter regularization methods when applied to approximate the solution of linear discrete ill-posed problems. We consider both direct methods, such as Tikhonov regularization with two or more regularization terms, and iterative methods based on the projection of a Tikhonov-regularized problem onto Krylov subspaces of increasing dimension. The latter methods regularize by choosing appropriate regularization terms and the dimension of the Krylov subspace. Our investigation focuses on selecting a proper set of regularization parameters that satisfies the discrepancy principle and maximizes a suitable quantity, whose size reflects the quality of the computed approximate solution. Theoretical results are shown and illustrated by numerical experiments.

Generalized averaged Gauss quadrature rules for the approximation of matrix functionals

The need to compute expressions of the form \(u^*f(A)v\), where A is a large square matrix, u and v are vectors, and f is a function, arises in many applications, including network analysis, quantum chromodynamics, and the solution of linear discrete ill-posed problems. Commonly used approaches first reduce A to a small matrix by a few steps of the Hermitian or non-Hermitian Lanczos processes and then evaluate the reduced problem. This paper describes a new method to determine error estimates for computed quantities and shows how to achieve higher accuracy than available methods for essentially the same computational effort. Our methods are based on recently proposed generalized averaged Gauss quadrature formulas.

Regularization of geophysical ill-posed problems by iteratively re-weighted and refined least squares

The iteratively re-weighted least squares (IRLS) is a commonly used algorithm which has received significant attention in geophysics and other fields of scientific computing for regularization of discrete ill-posed problems. The IRLS replaces a difficult optimization problem by a sequence of weighted linear systems. The optimum solution of the original problem is usually determined by computing the solution for various regularization parameters λ, each needing several re-weighted iterations (usually 10–15). In this paper, in order to decrease the required computation time (iterations) while maintaining good properties of the algorithm such as edge-preserving, the IRLS is augmented with a refinement strategy and the value of λ is progressively updated in a geometrical form during the iterations. The new algorithm, called iteratively re-weighted and refined least squares (IRRLS), can be interpreted as a Landweber iteration with a non-stationary shaping matrix which is updated based on the solution obtained from previous iteration. Two main properties of IRRLS are (1) the regularization parameter is the stopping iteration and (2) it is equipped with a tuning parameter which makes it flexible for recovering models with different smoothness. We show numerically that both the residual and regularization norms are monotone functions of iteration and hence well behaved for automatic determination of stopping parameter. The Stain’s unbiased risk estimate (SURE), generalized cross validation (GCV), L-curve analysis, and discrepancy principle (DCP) techniques are employed for automatic determination of optimum iteration. Experimental results from seismic deconvolution and seismic tomography are included showing that the proposed methodology outperforms the conventional IRLS with significantly lower computational burden.

On the choice of solution subspace for nonstationary iterated Tikhonov regularization

Tikhonov regularization is a popular method for the solution of linear discrete ill-posed problems with error-contaminated data. Nonstationary iterated Tikhonov regularization is known to be able to determine approximate solutions of higher quality than standard Tikhonov regularization. We investigate the choice of solution subspace in iterative methods for nonstationary iterated Tikhonov regularization of large-scale problems. Generalized Krylov subspaces are compared with Krylov subspaces that are generated by Golub---Kahan bidiagonalization and the Arnoldi process. Numerical examples illustrate the effectiveness of the methods.

A restoration–segmentation algorithm based on flexible Arnoldi–Tikhonov method and Curvelet denoising

In this paper, we illustrate a two-stage algorithm consisting of restoration and segmentation to reach binary segmentation from the noisy and blurry image. The results of our method can be applied in main fields of the image processing such as object extraction. In the first stage, we have a linear discrete ill-posed problem with a noise-contaminated right-hand side, arising from the image restoration. We consider problems in which the coefficient matrix is the sum of Kronecker products of matrices and present a global flexible Arnoldi–Tikhonov method coupled with the generalized cross-validation for the computation of the regularization parameter at each iteration. The proposed algorithm is based on the global Arnoldi method that allows using a more flexible solution subspace. In the second stage, we segment the restored image in order to reach a binary image in which the target object is emphasized. In our segmentation method, we use Gaussian scale-space technique to compute discrete gradients of the restored image for pre-segmenting. Also, in order to denoise, we use a tight frame of Curvelet transforms and thresholding which is based on the principle obtained by minimizing Stein’s unbiased risk estimate. This algorithm has an iterative part based on the iterative part of TFA (Cai et al. in SIAM J Imaging 6(1):464–486, 2013), but we use eigenvectors of Hessian matrix of image for improving this part. Theoretical properties of the method of both stages and numerical experiments are presented.

Analytical Study of Error Components for Solving Discrete Ill-Posed Problems Using Random Projections

This article analytically studies the dependence of components of the error of reconstructing the true signal on the number of rows of a random projection matrix. It is shown that, with increasing the dimension of the random projector, the deterministic error component decreases and the stochastic one increases. Expressions are obtained for calculating the interval of noise levels that ensure the availability of the global minimum of the error. The analytical results are confirmed by numerical experiments.

Chemical species tomography of turbulent flows: Discrete ill-posed and rank deficient problems and the use of prior information

Due to the inherent ill-posed nature of chemical species tomography (CST) problems, additional information based on the presumed species distribution must be introduced into the reconstruction procedure. The role that this prior information plays in tomographic reconstruction differs depending on whether the CST problem is discrete ill-posed or rank-deficient. The former case arises mainly in laboratory studies involving small scale problems with high degrees of optical access and often a stationary flow field, while the later occurs when the number and arrangement of measurements are limited by the size and/or the optical access afforded by the containing geometry. This paper elucidates the difference between these two types of CST problems, and reviews various ways that prior information can be used to enhance reconstruction accuracy of CST experiments on turbulent flows.

Inheritance of the discrete Picard condition in Krylov subspace methods

When projection methods are employed to regularize linear discrete ill-posed problems, one implicitly assumes that the discrete Picard condition (DPC) is somehow inherited by the projected problems. In this paper we show that, under some assumptions, the DPC holds for the projected uncorrupted systems computed by various Krylov subspace methods. By exploiting the inheritance of the DPC, some estimates on the behavior of the projected problems are also derived. Numerical examples are provided in order to illustrate the accuracy of the derived estimates.