A conceptual framework for consistency, conditioning, and stability issues in signal processing

Abstract

The techniques employed for analyzing algorithms in numerical linear algebra have evolved significantly since the 1940s. Significant in this evolution is the partitioning of the terminology into categories in which analyses involving infinite precision effects are distinguished from analyses involving finite precision effects. Although the structure of algorithms in signal processing prevents the direct application of typical analysis techniques employed in numerical linear algebra, much can be gained in signal processing from an assimilation of the terminology found there. This paper addresses the need for a conceptual framework for discussing the computed solution from an algorithm by focusing on the distinction between a perturbation analysis of a problem or a method of solution and the stability analysis of an algorithm. A consistent approach to defining these concepts facilitates the task of assessing the numerical quality of a computed solution. This paper discusses numerical analysis techniques for signal processing algorithms and suggests terminology that is supportive of a centralized framework for distinguishing between errors propagated by the nature of the problem and errors propagated through the use of finite-precision arithmetic. By this, we mean that the numerical stability analysis of a signal processing algorithm can be simplified and the meaning of such an analysis made unequivocal.