Evaluation of receiver operating characteristic curves in terms of information theory

Abstract

Measurement of Receiver Operating Characteristic (ROC) curves provides an objective means for describing the ability of an observer to detect an object structure in any particular imaging situation, independent of the 'attitude' or 'threshold' which the observer assumes. In simple idealized detection situations ROC curves exhibit certain symmetric properties and performance ranking of imaging systems is unambiguous. However ROC curves determined in actual visual detection experiments are characteristically asymmetric and a generally meaningful measure for comparing such curves has not been apparent. The present work has shown that each point on an ROC curve has associated with it some easily calculated 'information content', in the information theoretic sense, which is independent of assumptions about underlying noise probability distributions. The maximum information content available on an ROC curve describes the maximum amount of uncertainty which can be dispelled by the image for the particular situation at hand and provides a useful and theoretically meaningful measure for evaluation of imaging systems in any given situation. This approach has been applied, with encouraging results, to the problem of detecting small bead objects (resembling gall-stones) using several screen-film systems having different noise and optical properties.