Discrete Ill-posed Problems Research Articles

Weighted tensor Golub–Kahan–Tikhonov-type methods applied to image processing using a t-product

This paper discusses weighted tensor Golub–Kahan-type bidiagonalization processes using the t-product. This product was introduced in M.E. Kilmer and C.D. Martin (2011). A few steps of a bidiagonalization process with a weighted least squares norm are carried out to reduce a large-scale linear discrete ill-posed problem to a problem of small size. The weights are determined by symmetric positive definite (SPD) tensors. Tikhonov regularization is applied to the reduced problem. An algorithm for tensor Cholesky factorization of SPD tensors is presented. The data is a laterally oriented matrix or a general third order tensor. The use of a weighted Frobenius norm in the fidelity term of Tikhonov minimization problems is appropriate when the noise in the data has a known covariance matrix that is not the identity. We use the discrepancy principle to determine both the regularization parameter in Tikhonov regularization and the number of bidiagonalization steps. Applications to image and video restoration are considered.

RBF approximation of three dimensional PDEs using tensor Krylov subspace methods

In this paper, we propose different algorithms for the solution of a tensor linear discrete ill-posed problem arising in the application of the meshless method for solving PDEs in three-dimensional space using multiquadric radial basis functions. It is well known that the truncated singular value decomposition (TSVD) is the most common effective solver for ill-conditioned systems, but unfortunately the operation count for solving a linear system with the TSVD is computationally expensive for large-scale matrices. In the present work, we propose algorithms based on the use of the well known Einstein product for two tensors to define the tensor global Arnoldi and the tensor Gloub Kahan bidiagonalization algorithms. Using the so-called Tikhonov regularization technique, we will be able to provide computable approximate regularized solutions in a few iterations. The tensor formulation will allow us to develop an RBF approximation for high dimensional PDEs based on Hierarchical tensors and Adaptive Cross Approximation, which in turn reduces significantly the storage and computational costs, while the accuracy of the method is preserved. The performance of the proposed methods is illustrated with a variety of benchmark examples and large-scale industrial applications with high degrees of freedom.

A new two-step iteration method for discrete ill-posed problems and image restoration

In this study, for the augmented linear system of discrete ill-posed problems we establish a new two-step (NTS) iteration method containing a parameter and a parameter matrix, which is based on the Hermitian and skew-Hermitian splitting (HSS) and the upper and lower triangular splitting (ULT) of the coefficient matrix. Then, we theoretically study its convergence properties and determine its optimal iteration parameters. It is seen that the NTS method converges faster when the parameters are chosen properly. Experimental examples are carried out to further validate the effectiveness and accuracy of the new method compared to the newly developed methods in terms of the numerical performance and image recovering quality.

Tensor Arnoldi–Tikhonov and GMRES-Type Methods for Ill-Posed Problems with a t-Product Structure

This paper describes solution methods for linear discrete ill-posed problems defined by third order tensors and the t-product formalism introduced in (Linear Algebra Appl 435:641–658, 2011). A t-product Arnoldi (t-Arnoldi) process is defined and applied to reduce a large-scale Tikhonov regularization problem for third order tensors to a problem of small size. The data may be represented by a laterally oriented matrix or a third order tensor, and the regularization operator is a third order tensor. The discrepancy principle is used to determine the regularization parameter and the number of steps of the t-Arnoldi process. Numerical examples compare results for several solution methods, and illustrate the potential superiority of solution methods that tensorize over solution methods that matricize linear discrete ill-posed problems for third order tensors.

Tikhonov regularization with MTRSVD method for solving large-scale discrete ill-posed problems

Regularization is possibly the most popular method for solving discrete ill-posed problems, whose solution is less sensitive to the error in the observed vector in the right hand than the original solution. This paper presents a new modified truncated randomized singular value decomposition (TR-MTRSVD) method for large Tikhonov regularization in standard form. The proposed TR-MTRSVD algorithm introduces the idea of randomized algorithm into the improved truncated singular value decomposition (MTSVD) method to solve large Tikhonov regularization problems. The approximation matrix Ãℓ produced by randomized SVD is replaced by the closest matrix Ãk̃ in a unitarily invariant matrix norm with the same spectral condition number. The regularization parameters are determined by the discrepancy principle. Numerical examples show the effectiveness and efficiency of the proposed TR-MTRSVD algorithm for large Tikhonov regularization problems.

On the Generalization of the Random Projection Method for Problems of the Recovery of Object Signal Described by Models of Convolution Type

Introduction. In technical systems, there is a common situation when transformation input-output is described by the integral equation of convolution type. This situation accurses if the object signal is recovered by the results of remote measurements. For example, in spectrometric tasks, for an image deblurring, etc. Matrices of the discrete representation for the output signal and the kernel of convolution are known. We need to find a matrix of the discrete representation of a signal of the object. The well known approach for solving this problem includes the next steps. First, the kernel matrix has to be represented as the Kroneker product. Second, the input-output transformation has to be presented with the usage of Kroneker product matrices. Third, the matrix of the discrete representation of the object has to be found. The object signal matrix estimation obtained with the help of pseudo inverting of Kroneker decomposition matrices is unstable. The instability of the object signal estimation in the case of usage of Kroneker decomposition matrices is caused by their discrete ill posed matrix properties (condition number is big and the series of the singular numbers smoothly decrease to zero). To find solutions of discrete ill-posed problems we developed methods based on the random projection and the random projection with an averaging by the random matrices. These methods provide a stable solutions with a small computational complexity. We consider the problem of object signals recovering in the systems where an input-output transformation is described by the integral equation of a convolution. To find a solution for these problems we need to build a generalization for two-dimensional signals case of the random projection method. Purpose. To develop a stable method of the recovery of object signal for the case in which an input-output transformation is described by the integral equation of a convolution. Results and conclusions. We developed the method of a stable recovery of object signal for the case in which an input-output transformation is described by the integral equation of a convolution. The stable estimation of the object signal is provided by Kroneker decomposition of the kernel matrix of convolution, computation of random projections for Kroneker factorization matrices, and a selection of the optimal dimension of a projector matrix. The method is illustrated by its application in technical problems.

Tensor Krylov subspace methods with an invertible linear transform product applied to image processing

This paper discusses several transform-based methods for solving linear discrete ill-posed problems for third order tensor equations based on a tensor-tensor product defined by an invertible linear transform. Linear transform-based tensor-tensor products were first introduced in Kernfeld et al. (2015) [16]. These tensor-tensor products are applied to derive Tikhonov regularization methods based on Golub-Kahan-type bidiagonalization and Arnoldi-type processes. GMRES-type solution methods based on the latter process also are described. By applying only a fairly small number of steps of these processes, large-scale problems are reduced to problems of small size. The number of steps required by these processes and the regularization parameter are determined by the discrepancy principle. The data tensor is a general third order tensor or a tensor defined by a laterally oriented matrix. A quite general regularization tensor can be applied in Tikhonov regularization. Applications to color image and video restorations illustrate the effectiveness of the proposed methods.

Regularization of inverse problems by an approximate matrix-function technique

In this work, we introduce and investigate a class of matrix-free regularization techniques for discrete linear ill-posed problems based on the approximate computation of a special matrix-function. In order to produce a regularized solution, the proposed strategy employs a regular approximation of the Heavyside step function computed into a small Krylov subspace. This particular feature allows our proposal to be independent from the structure of the underlying matrix. If on the one hand, the use of the Heavyside step function prevents the amplification of the noise by suitably filtering the responsible components of the spectrum of the discretization matrix, on the other hand, it permits the correct reconstruction of the signal inverting the remaining part of the spectrum. Numerical tests on a gallery of standard benchmark problems are included to prove the efficacy of our approach even for problems affected by a high level of noise.

Golub\u2013Kahan vs. Monte Carlo: a comparison of bidiagonlization and a randomized SVD method for the solution of linear discrete ill-posed problems

Randomized methods can be competitive for the solution of problems with a large matrix of low rank. They also have been applied successfully to the solution of large-scale linear discrete ill-posed problems by Tikhonov regularization (Xiang and Zou in Inverse Probl 29:085008, 2013). This entails the computation of an approximation of a partial singular value decomposition of a large matrix A that is of numerical low rank. The present paper compares a randomized method to a Krylov subspace method based on Golub–Kahan bidiagonalization with respect to accuracy and computing time and discusses characteristics of linear discrete ill-posed problems that make them well suited for solution by a randomized method.

Generalized cross validation for \u2113p-\u2113q minimization

Discrete ill-posed inverse problems arise in various areas of science and engineering. The presence of noise in the data often makes it difficult to compute an accurate approximate solution. To reduce the sensitivity of the computed solution to the noise, one replaces the original problem by a nearby well-posed minimization problem, whose solution is less sensitive to the noise in the data than the solution of the original problem. This replacement is known as regularization. We consider the situation when the minimization problem consists of a fidelity term, that is defined in terms of a p-norm, and a regularization term, that is defined in terms of a q-norm. We allow 0 < p,q ≤ 2. The relative importance of the fidelity and regularization terms is determined by a regularization parameter. This paper develops an automatic strategy for determining the regularization parameter for these minimization problems. The proposed approach is based on a new application of generalized cross validation. Computed examples illustrate the performance of the method proposed.

Iteratively Reweighted FGMRES and FLSQR for Sparse Reconstruction

This paper presents two new algorithms to compute sparse solutions of large-scale linear discrete ill-posed problems. The proposed approach consists in constructing a sequence of quadratic problems...

A novel modified TRSVD method for large-scale linear discrete ill-posed problems

The truncated singular value decomposition (TSVD) is a popular method for solving linear discrete ill-posed problems with a small to moderately sized matrix A. This method replaces the matrix A by the closest matrix Ak of low rank k, and then computes the minimal norm solution of the linear system of equations with a rank-deficient matrix so obtained. The modified TSVD (MTSVD) method improves the TSVD method, by replacing A by a matrix that is closer to A than Ak in a unitarily invariant matrix norm and has the same spectral condition number as Ak. Approximations of the SVD of a large matrix A can be computed quite efficiently by using a randomized SVD (RSVD) method. This paper presents a novel modified truncated randomized singular value decomposition (MTRSVD) method for computing approximate solutions to large-scale linear discrete ill-posed problems. The rank, k, is determined with the aid of the discrepancy principle, but other techniques for selecting a suitable rank also can be used. Numerical examples illustrate the effectiveness of the proposed method and compare it to the truncated RSVD method.

The regularizing properties of global GMRES for solving large-scale linear discrete ill-posed problems with several right-hand sides

The global Krylov subspace iterative methods are an attractive class of iterative solvers for solving linear systems with several right-hand sides. In this paper, the global version of the GMRES method is applied to solve linear discrete ill-posed problems that arise from the discretization of linear ill-posed problems, pattern classification and dimensionality reduction. The regularizing properties of the global GMRES method are analyzed, and a regularized global GMRES method is developed for solving linear discrete ill-posed problems with multiple right-hand sides contaminated by errors. Some numerical experiments on test matrices are presented to show the efficiency of the proposed method.

Modulus-based iterative methods for constrained ℓp–ℓq minimization

The need to solve discrete ill-posed problems arises in many areas of science and engineering. Solutions of these problems, if they exist, are very sensitive to perturbations in the available data. Regularization replaces the original problem by a nearby regularized problem, whose solution is less sensitive to the error in the data. The regularized problem contains a fidelity term and a regularization term. Recently, the use of a p-norm to measure the fidelity term and a q-norm to measure the regularization term has received considerable attention. The balance between these terms is determined by a regularization parameter. In many applications, such as in image restoration, the desired solution is known to live in a convex set, such as the nonnegative orthant. It is natural to require the computed solution of the regularized problem to satisfy the same constraint(s). This paper shows that this procedure induces a regularization method and describes a modulus-based iterative method for computing a constrained approximate solution of a smoothed version of the regularized problem. Convergence of the iterative method is shown, and numerical examples that illustrate the performance of the proposed method are presented.

Regularization properties of LSQR for linear discrete ill-posed problems in the multiple singular value case and best, near best and general low rank approximations**The work was supported by the National Science Foundation of China (No. 11771249).

For the large-scale linear discrete ill-posed problem min‖Ax − b‖ or Ax = b with b contaminated by white noise, the Golub–Kahan bidiagonalization based LSQR method and its mathematically equivalent CGLS, the conjugate gradient (CG) method applied to ATAx = ATb, are most commonly used. They have intrinsic regularizing effects, where the iteration number k plays the role of regularization parameter. The long-standing fundamental question is: Can LSQR and CGLS find two-norm filtering best possible regularized solutions? The author has given definitive answers to this question for severely and moderately ill-posed problems when the singular values of A are simple. This paper extends the results to the multiple singular value case, and studies the approximation accuracy of Krylov subspaces, the quality of low rank approximations generated by Golub–Kahan bidiagonalization and the convergence properties of Ritz values. For the two kinds of problems, we prove that LSQR finds two-norm filtering best possible regularized solutions at semi-convergence. Particularly, we consider some important and untouched issues on best, near best and general rank k approximations to A for the ill-posed problems with the singular values with α > 0, and the relationships between them and their nonzero singular values. Numerical experiments confirm our theory. The results on general rank k approximations and the properties of their nonzero singular values apply to several Krylov solvers, including LSQR, CGME, MINRES, MR-II, GMRES and RRGMRES.

A joint bidiagonalization based iterative algorithm for large scale general-form Tikhonov regularization

Based on the joint bidiagonalization (JBD) process of the matrix pair {A,L}, an iterative regularization algorithm, called JBDQR, is proposed and developed for large scale linear discrete ill-posed problems in general-form Tikhonov regularization. It is proved that the JBDQR iterates take the form of attractive filtered generalized singular value decomposition (GSVD) expansions, where the filters are given explicitly and insightful. This result and a detailed analysis on it show that JBDQR must have the desired semi-convergence property, where the iteration number k plays the role of the regularization parameter. Embedded with the L-curve criterion or the discrepancy principle that is used to estimate the optimal k⁎ at which the semi-convergence occurs, JBDQR can compute a satisfying good regularized solution. JBDQR is theoretically solid and effective, and it is simple to implement. Numerical experiments confirm our results and the robustness of JBDQR.

Determination of regularization parameter via solving a multi-objective optimization problem

This paper presents a multi-objective optimization approach for choosing an appropriate regularization parameter in Tikhonov-type regularization of discrete ill-posed problems. Finding the regularization parameter is formulated as a multi-objective optimization problem and some transformations are introduced to construct a new problem equivalent to it. Then, by using the standard multi-objective optimization methods, a single-objective problem is derived that its minimizer gives a proper estimation of the regularization parameter. Via considering some test problems, the numerical efficiency of the presented method is compared with the L-curve and the GCV parameter choice methods.

The low rank approximations and Ritz values in LSQR for linear discrete ill-posed problem**The work was supported by the National Science Foundation of China (No. 11771249).

LSQR and its mathematically equivalent CGLS have been popularly used over the decades for large-scale linear discrete ill-posed problems, where the iteration number k plays the role of the regularization parameter. It has been long known that if the Ritz values in LSQR converge to the large singular values of A in natural order, that is, they interlace the first k + 1 large singular values of A, until the semi-convergence of LSQR occurs then LSQR must have the same the regularization ability as the truncated singular value decomposition (TSVD) method and can compute a 2-norm filtering best possible regularized solution. However, hitherto there has been no definitive rigorous result on the approximation behavior of the Ritz values in the context of ill-posed problems. In this paper, for severely, moderately and mildly ill-posed problems, we give accurate solutions of the two closely related fundamental and highly challenging problems on the regularization of LSQR: (i) how accurate are the low rank approximations generated by Golub–Kahan bidiagonalization? (ii) Whether or not the Ritz values involved in LSQR approximate the large singular values of A in natural order? We also show how to reliably judge the accuracy of low rank approximations cheaply. Numerical experiments confirm our results.

A special modified Tikhonov regularization matrix for discrete ill-posed problems

In this paper, we investigate the solution of large-scale linear discrete ill-posed problems with error-contaminated data. If the solution exists, it is very sensitive to perturbations in the data. Tikhonov regularization, i.e., replacement of the available ill-conditioned problem by a nearby better conditioned one with a fidelity term and a penalty term, can reduce this sensitivity and is a popular approach to determine meaningful approximate solutions of such problems. The penalty term is determined by a regularization matrix. The choice of this matrix may significantly affect the quality of the computed approximate solution. In order to get an appropriate solution with improved accuracy, the paper constructs a special modified Tikhonov regularization (SMTR) matrix so as to include more useful information. The parameters involved in the presented regularization matrix are discussed. Besides, the corresponding preconditioner PSMTR is designed to accelerate the convergence rate of the CGLS method for solving Tikhonov regularization least-squares system. Furthermore, numerical experiments illustrate that the SMTR method and PSMTR preconditioner significantly outperform the related methods in terms of solution accuracy.

An [formula omitted]-[formula omitted] minimization method with cross-validation for the restoration of impulse noise contaminated images

Discrete ill-posed problems arise in many areas of science and engineering. Their solutions, if they exist, are very sensitive to perturbations in the data. Regularization aims to reduce this sensitivity. Many regularization methods replace the original problem by a minimization problem with a fidelity term and a regularization term. Recently, the use of a p-norm to measure the fidelity term and a q-norm to measure the regularization term has received considerable attention. The relative importance of these terms is determined by a regularization parameter. When the perturbation in the available data is made up of impulse noise and a sparse solution is desired, it often is beneficial to let 0<p,q<1. Then the p- and q-norms are not norms. The choice of a suitable regularization parameter is crucial for the quality of the computed solution. It therefore is important to develop methods for determining this parameter automatically, without user-interaction. However, the latter has so far not received much attention when the data is contaminated by impulse noise. This paper discusses two approaches based on cross validation for determining the regularization parameter in this situation. Computed examples that illustrate the performance of these approaches when applied to the restoration of impulse noise contaminated images are presented.