Ill-posed Linear System Research Articles

Regularization parameter determination for discrete ill-posed problems

The straightforward solution of discrete ill-posed linear systems of equations or least-squares problems with error contaminated data does not, in general, give meaningful results, because the propagated error destroys the computed solution. The problems have to be modified to reduce their sensitivity to the error in the data. The amount of modification is determined by a regularization parameter. It can be difficult to determine a suitable value of the regularization parameter when no knowledge of the norm of error in the data is available. This paper proposes a new simple technique for determining a value of the regularization parameter that can be applied in this situation. It is based on comparing computed solutions determined by Tikhonov regularization and truncated singular value decomposition. Analogous comparisons are proposed for large-scale problems. The technique for determining the regularization parameter implicity provides an estimate for the norm of the error in the data.

A maximal projection solution of ill-posed linear system in a column subspace, better than the least squares solution

A better method than the least squares solution is proposed in this paper to solve ann-dimensional ill-posed linear equations system Ax=b in an m-dimensional column subspace Cm, which is selected in such a way that each column in Cm is in a closer proximity to b. We maximize the orthogonal projection of b onto y≔Ax to find an approximate solution x∈span{a,Cm}, where a is a nonzero free vector. Then, we can prove that the maximal projection solution (MP) is better than the least squares solution (LS) with ‖b−AxMP‖<‖b−AxLS‖. Numerical examples of inverse problems under a large noise maybe up to 30% are discussed which confirm the efficiency of presently developed MP algorithms: MPA and MPA(m).

A double optimal descent algorithm for iteratively solving ill-posed linear inverse problems

In the iterative solution of an ill-posed linear system , how to select a fast and easily established descent direction to reduce the residual is an important issue. A mathematical procedure to find a double optimal descent direction in , without inverting , is developed in an -dimensional Krylov subspace. The novelty is that we expand in an affine Krylov subspace with undetermined coefficients, and then two optimization techniques are used to determine these coefficients in closed form, which can greatly accelerate the convergence speed in solving the ill-posed linear problems. The double optimal descent algorithm is proven to be absolutely convergent very fast, accurate and robust against noise, which is confirmed by numerical tests of several linear inverse problems, including the heat source identification problem, the backward heat conduction problem, the inverse Cauchy problem and the external force identification problem.

Multicore Performance of Block Algebraic Iterative Reconstruction Methods

Algebraic iterative methods are routinely used for solving the ill-posed sparse linear systems arising in tomographic image reconstruction. Here we consider the algebraic reconstruction technique (ART) and the simultaneous iterative reconstruction techniques (SIRT), both of which rely on semiconvergence. Block versions of these methods, based on a partitioning of the linear system, are able to combine the fast semiconvergence of ART with the better multicore properties of SIRT. These block methods separate into two classes: those that, in each iteration, access the blocks in a sequential manner, and those that compute a result for each block in parallel and then combine these results before the next iteration. The goal of this work is to demonstrate which block methods are best suited for implementation on modern multicore computers. To compare the performance of the different block methods, we use a fixed relaxation parameter in each method, namely, the one that leads to the fastest semiconvergence. Comp...

The pre/post equilibrated conditioning methods to solve Cauchy problems

In the present paper, the inverse Cauchy problems of Laplace equation and biharmonic equation are transformed, by using the method of fundamental solutions (MFS) and the Trefftz method (TM), to the systems of linear equations for determining the expansion coefficients. Then, we propose three different conditioners together with the conjugate gradient method (CGM) to solve the resultant ill-posed linear systems. They are the post-conditioning CGM and the pre-conditioning CGM based on the idea of equilibrated norm for the conditioned matrices, as well as a minimum-distance conditioner. These algorithms are convergent fast and accurate by solving the inverse Cauchy problems under random noise.

An optimized parallel LSQR algorithm for seismic tomography

The LSQR algorithm developed by Paige and Saunders (1982) is considered one of the most efficient and stable methods for solving large, sparse, and ill-posed linear (or linearized) systems. In seis...

A globally optimal tri-vector method to solve an ill-posed linear system

In the present paper, a future cone in the Minkowski space defined in terms of the square-norm of the residual vector for an ill-posed linear system to be solved, is used to derive a nonlinear system of ordinary differential equations. Then the forward Euler scheme is used to generate an iterative algorithm. Two critical values in the critical descent tri-vector are derived, which lead to the largest convergence rate of the resultant iterative algorithm, namely the globally optimal tri-vector method (GOTVM). Some numerical examples are used to reveal the superior performance of the GOTVM than the famous methods of conjugate gradient (CGM) and generalized minimal residual (GMRES). Through the numerical tests we also set forth the rationale by assuming the tri-vector as being a better descent direction.

An optimized parallel LSQR algorithm for seismic tomography

The LSQR algorithm developed by Paige and Saunders (1982) is considered one of the most efficient and stable methods for solving large, sparse, and ill-posed linear (or linearized) systems. In seismic tomography, the LSQR method has been widely used in solving linearized inversion problems. As the amount of seismic observations increase and tomographic techniques advance, the size of inversion problems can grow accordingly. Currently, a few parallel LSQR solvers are presented or available for solving large problems on supercomputers, but the scalabilities are generally weak because of the significant communication cost among processors. In this paper, we present the details of our optimizations on the LSQR code for, but not limited to, seismic tomographic inversions. The optimizations we have implemented to our LSQR code include: reordering the damping matrix to reduce its band-width for simplifying the communication pattern and reducing the amount of communication during calculations; adopting sparse matrix storage formats for efficiently storing and partitioning matrices; using the MPI I/O functions to parallelize the date reading and result writing processes; providing different data partition strategies for efficiently using computational resources. A large seismic tomographic inversion problem, the full-3D waveform tomography for Southern California, is used to explain the details of our optimizations and examine the performance on Yellowstone supercomputer at the NCAR-Wyoming Supercomputing Center (NWSC). The results showed that the required wall time of our code for the same inversion problem is much less than that of the LSQR solver from the PETSc library (Balay et al., 1997).

A vector regularization method to solve linear inverse problems

The linear inverse problem is discretized to be an n-dimensional ill-posed linear equations system . In the present paper, an invariant manifold defined in terms of the square norm of a residual vector is used to derive an iterative algorithm with a fast descent direction , which is close to, but not exactly equal to, the best descent direction . The matrix is obtained by using a vector regularization method together with a matrix conjugate gradient method to find the right inversion of : . The vector regularization iterative algorithm is proven to be Lyapunov stable, and the direct inversion method with solution expressed by converges fast. The accuracy and efficiency of them are verified through the numerical tests of linear inverse problems under a large random noise.

A Two-Side Equilibration Method to Reduce theCondition Number of an Ill-Posed Linear System

A Two-Side Equilibration Method to Reduce theCondition Number of an Ill-Posed Linear System

A State Feedback Controller Used to Solve an Ill-posed Linear System by a GL(n, R) Iterative Algorithm

Starting from a quadratic invariant manifold in terms of the residual vector ${\textbf r}={\textbf B}{\textbf x}-{\textbf b}$ for an $n$-dimensional ill-posed linear algebraic equations system ${\textbf B}{\textbf x}={\textbf b}$, we derive an ODEs system for ${\textbf x}$ which is equipped with a state feedback controller to enforce the orbit of the state vector ${\textbf x}$ on a specified manifold, whose residual-norm is exponentially decayed. To realize the above idea we develop a very powerful implicit scheme based on the novel $GL(n,{\mathbb R})$ Lie-group method to integrate the resultant differential algebraic equation (DAE). Through numerical tests of inverse problems we find that the present Lie-group DAE algorithm can significantly accelerate the convergence speed, and is robust enough against the random noise.

Numerical simulations of 1D inverse heat conduction problems using overdetermined RBF-MLPG method

This paper proposes a numerical method to deal with the one-dimensional inverse heat conduction problem (IHCP). The initial temperature, a condition on an accessible part of the boundary and an additional temperature measurements in time at an arbitrary location in the domain are known, and it is required to determine the temperature and the heat flux on the remaining part of the boundary. Due to the missing boundary condition, the solution of this problem does not depend continuously on the data and therefore its numerical solution requires special care especially when noise is present in the measured data. In the proposed method, the time variable is eliminated by using finite differences approximation. The method uses a weak formulation of the problem to enjoy the stability condition. To avoid the numerical integration on the whole domain, the weak form equations are constructed on local subdomains. The approximate solution is assumed to be a linear combination of Multi Quadric (MQ) radial basis function (RBF) constructed on nodal points in the domain and on the boundary. Since the problem is known to be ill-posed, Thikhonov regularization strategy is employed to solve effectively the discrete ill-posed resultant linear system.

An Optimally Generalized Steepest-Descent Algorithm for Solving Ill-Posed Linear Systems

It is known that the steepest-descent method converges normally at the first few iterations, and then it slows down. We modify the original steplength and descent direction by an optimization argument with the new steplength as being a merit function to be maximized. An optimal iterative algorithm with <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M1" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path id="x1D45A" d="M766 88q-40 -45 -83.5 -72.5t-63.5 -27.5q-39 0 -16 103l49 224q15 68 -10 68q-36 0 -112.5 -78.5t-105.5 -133.5q-27 -101 -38 -165q-35 -4 -77 -18l-6 6q42 153 70 288q12 55 10 78t-19 23q-38 0 -114 -80.5t-101 -131.5q-24 -68 -41 -165q-36 -3 -76 -18l-8 6&#xA;q56 202 87 347q7 33 -3 33q-8 0 -31.5 -17.5t-41.5 -35.5l-12 28q42 45 84 72t63 27q44 0 10 -124l-20 -75h2q92 121 193 178q36 21 64 21q62 0 28 -159l-8 -37h2q50 67 103.5 112t94.5 65q39 19 62 19q60 0 23 -156l-45 -189q-9 -39 2 -39q17 0 71 49z" /></g> </svg>-vector descent direction in a Krylov subspace is constructed, of which the <svg style="vertical-align:-0.10033pt;width:13.5375px;" id="M2" height="7.9499998" version="1.1" viewBox="0 0 13.5375 7.9499998" width="13.5375" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg"> <g transform="matrix(.017,-0,0,-.017,.062,7.675)"><use xlink:href="#x1D45A"/></g> </svg> optimal weighting parameters are solved in closed-form to accelerate the convergence speed in solving ill-posed linear problems. The optimally generalized steepest-descent algorithm (OGSDA) is proven to be convergent with very fast convergence speed, accurate and robust against noisy disturbance, which is confirmed by numerical tests of some well-known ill-posed linear problems and linear inverse problems.

An optimal tri-vector iterative algorithm for solving ill-posed linear inverse problems

A future cone in the Minkowski space, defined in terms of the square-norm of the residual vector for an ill-posed linear system to be solved, is used to derive an optimal tri-vector descent system of nonlinear ordinary differential equations (ODEs). Then, a simple Euler scheme is used to generate an iterative algorithm from these ODEs, of which the two parameters appeared are optimized from a properly defined merit function to accelerate the convergence speed in solving the ill-posed linear systems. The optimal tri-vector iterative algorithm (OTVIA) is fast convergent and accurate, which is proven by numerical tests of inverse problems, including the backward heat conduction problem, the Calderón inverse problem and the inverse Cauchy problems. By defining a suitable convergence rate, we assess the convergence speeds of OTVIA and the conjugate gradient method (CGM), which reveal that the performance of OTVIA is better than the CGM. Also by comparing the OTVIA with the generalized minimal residual method (GMRES), we observe that the OTVIA is better than the GMRES.

A Dynamical Tikhonov Regularization for Solving Ill-posed Linear Algebraic Systems

The Tikhonov method is a famous technique for regularizing ill-posed linear problems, wherein a regularization parameter needs to be determined. This article, based on an invariant-manifold method, presents an adaptive Tikhonov method to solve ill-posed linear algebraic problems. The new method consists in building a numerical minimizing vector sequence that remains on an invariant manifold, and then the Tikhonov parameter can be optimally computed at each iteration by minimizing a proper merit function. In the optimal vector method (OVM) three concepts of optimal vector, slow manifold and Hopf bifurcation are introduced. Numerical illustrations on well known ill-posed linear problems point out the computational efficiency and accuracy of the present OVM as compared with classical ones.

A Globally Optimal Iterative Algorithm to Solve an Ill-Posed Linear System

An iterative algorithm based on the critical descent vector is pro- posed to solve an ill-posed linear system: Bx = b. We define a future cone in the Minkowski space as an invariant manifold, wherein the discrete dynamics evolves. A critical value ac in the critical descent vector u = acr+ B T r is derived, which renders the largest convergence rate as to be the globally optimal iterative al- gorithm (GOIA) among all the numerically iterative algorithms with the descent vector having the form u = ar+ B T r to solve the ill-posed linear problems. Some numerical examples are used to reveal the superior performance of the GOIA.

Solving Ill-Posed Linear Systems With Constraints on Statistical Moments

Abstract-The problem of finding a solution to an ill-posed linear problem Ax = b, with specific statistical properties is addressed, constraining the statistical moments of the N elements in x up to a given order d. It is reformulated as a higher dimension minimization problem with Nd variables, whose objective function is the composition of a convex function and a projection. Although convergence to a local minima is possible in rare cases, simulations show that in the vast majority of the cases, global convergence is attained with a standard descent algorithm. By extension, this opens the possibility to efficiently solve a larger class of problems that are linear in the powers of x. The tomographic reconstruction from a limited number of projection angles, constrained by centered moments, is considered as an example.

Modifications of Steepest Descent Method and Conjugate Gradient Method Against Noise for Ill-posed Linear Systems

It is well known that the numerical algorithms of the steepest descent method (SDM), and the conjugate gradient method (CGM) are effective for solving well-posed linear systems. However, they are vulnerable to noisy disturbance for solving ill-posed linear systems. We propose the modifications of SDM and CGM, namely the modified steepest descent method (MSDM), and the modified conjugate gradient method (MCGM). The starting point is an invariant manifold defined in terms of a minimum functional and a fictitious time-like variable; however, in the final stage we can derive a purely iterative algorithm including an acceleration parameter. Through the Hopf bifurcation, this parameter indeed plays a major role to switch the situation of slow convergence to a new situation that the functional is stepwisely decreased very fast. Several numerical examples are examined and compared with exact solutions, revealing that the new algorithms of MSDM and MCGM have good computational efficiency and accuracy, even for the highly ill-conditioned linear

Solving Ill-Posed Linear Systems with GMRES and a Singular Preconditioner

Almost singular linear systems arise in discrete ill-posed problems. Either because of the intrinsic structure of the problem or because of preconditioning, the spectrum of the coefficient matrix is often characterized by a sizable gap between a large group of numerically zero eigenvalues and the rest of the spectrum. Correspondingly, the right-hand side has leading eigencomponents associated with the eigenvalues away from zero. In this paper the effect of this setting in the convergence of the generalized minimal residual (GMRES) method is considered. It is shown that in the initial phase of the iterative algorithm, the residual components corresponding to the large eigenvalues are reduced in norm, and these can be monitored without extra computation. The analysis is supported by numerical experiments. In particular, ill-posed Cauchy problems for partial differential equations with variable coefficients are considered, where the preconditioner is a fast, low-rank solver for the corresponding problem with...

GMRES implementations and residual smoothing techniques for solving ill-posed linear systems

There are verities of useful Krylov subspace methods to solve nonsymmetric linear system of equations. GMRES is one of the best Krylov solvers with several different variants to solve large sparse linear systems. Any GMRES implementation has some advantages. As the solution of ill-posed problems are important. In this paper, some GMRES variants are discussed and applied to solve these kinds of problems. Residual smoothing techniques are efficient ways to accelerate the convergence speed of some iterative methods like CG variants. At the end of this paper, some residual smoothing techniques are applied for different GMRES methods to test the influence of these techniques on GMRES implementations.