4. Problems with Ill-Determined Rank

Abstract

The purpose of this chapter is to summarize important results about discrete ill-posed problems, i.e., systems of equations (either square or overdetermined) derived from discretization of ill-posed problems. The main feature of these problems is that all the singular values of the coefficient matrix decay gradually to zero, with no gap anywhere in the spectrum. Whatever threshold ϵ is used in Eq. (3.3), the numerical ϵ-rank is highly ill determined, and therefore the concept of “numerical rank” is not useful for these problems.As a consequence, the regularization of discrete ill-posed problems is more complicated than merely filtering out a cluster of small singular values. For this reason, it is convenient to have a variety of mathematical tools at hand for obtaining more insight into the problem as well as the available regularization methods. Among these tools we find the filter factors, the resolution matrix, and the L-curve, all of which are described in detail below. Numerical examples that illustrate all these tools are presented in the last section of this chapter.4.1. Characteristics of Discrete Ill-Posed ProblemsFrom a strictly mathematical point of view, a finite-dimensional problem always satisfies the Picard condition (1.10), the minimum-norm solution is stable, and no regularization is required. Indeed, in a purely mathematical sense the transformation of a continuous problem to a discrete problem (“regularization by discretization”) always has a regularizing effect; see, e.g., [160, Chapter 4], [226, Chapter 3], or [227, Chapter 17]. However, this point of view does not account for the disastrous effects of rounding errors when the system is solved, due to the huge condition number of the coefficient matrix; cf. [160, Eq. (4.10)].In practical treatments of discrete ill-posed problems it is therefore necessary to incorporate some kind of regularization in the solution procedure for the discretized system Ax=b or min∥Ax−b∥2 , in order to compute a useful solution. It is also convenient to introduce the concept of a discrete Picard condition, equivalent to the Picard condition described in §4.5.