A new parameter of assessment of linear iterative algorithms: On their Informative Efficiency and Law of exponential-dissipation

Abstract

In disciplines like digital signal and image processing, acoustics, computer vision, geophysics etc. very often one needs to employ iterative techniques to solve a large linear system which good enough accuracy. Though certainly many such iterative algorithms have been devised and applied in practical purposes, their computational efficiency is of course not same, given the fact that the system may be incompletely probed or the measured parameters may be contaminated and a sound assessment of the algorithms in terms of efficiency parameters is very crucial to employ them in practical-fields. However, none of the conventional parameters can measure how much “effective” information an algorithm may provide regarding the system concerned and clearly a knowledge of this is very important since one of the key concerns of any practical device is to achieve highest possible information at the output (e.g., for an MRI system more information in the reconstructed image will imply acquisition of more detailed structure of some tissue as is highly desired by concerned physicians). Here conditional-entropy is proposed to be a parameter of efficiency (with reasons of justification) of an algorithm as a measure of lost information in the output which can be considered as a novel-measure of how much an algorithm is informatically favourable for various practical purposes. While this quantity shares some familiarity with the community of computer-vision, it was hardly examined as a metric of an iterative process. Numerical simulation is performed for a simple underdetermined linear tomographic system and some well-known iterative algorithms are computationally analysed and assessed based on their conditional entropy at the reconstruction-end. Also, it is showed that, in any linear-system the noise-dominated component of conditional-entropy of reconstruction almost exponentially grows with increasing ratio of noise to signal in the data associated with product-vector of that system (information is exponentially lost) and the weight-factor of exponent being dependent on the algorithm concerned.