Uncertainty and Information: Emergence of Vast New Territories

Abstract

A research program whose objective is to study uncertainty and uncertainty-based information in all their manifestations was introduced in the early 1990’s under the name “generalized information theory” (GIT). This research program, motivated primarily by some fundamental methodological issues emerging from the study of complex systems, is based on a two-dimensional expansion of classical, probability-based information theory. In one dimension, additive probability measures, which are inherent in classical information theory, are expanded to various types of nonadditive measures. In the other dimension, the formalized language of classical set theory, within which probability measures are formalized, is expanded to more expressive formalized languages that are based on fuzzy sets of various types. As in classical information theory, uncertainty is the primary concept in GIT and information is defined in terms of uncertainty reduction. This restricted interpretation of the concept of information is described in GIT by the qualified term “uncertainty-based information”. Each uncertainty theory that is recognizable within the expanded framework is characterized by: (i) a particular formalized language (a theory of fuzzy sets of some particular type); and (ii) a generalized measure of some particular type (additive or nonadditive). The number of possible uncertainty theories is thus equal to the product of the number of recognized types of fuzzy sets and the number of recognized types of generalized measures. This number has been growing quite rapidly with the recent developments in both fuzzy set theory and the theory of generalized measures. In order to fully develop any of these theories of uncertainty requires that issues at each of the following four levels be adequately addressed: (i) the theory must be formalized in terms of appropriate axioms; (ii) a calculus of the theory must be developed by which the formalized uncertainty is manipulated within the theory; (iii) a justifiable way of measuring the amount of relevant uncertainty (predictive, diagnostic, etc.) in any situation formalizable in the theory must be found; and (iv) various methodological aspects of the theory must be developed. Among the many uncertainty theories that are possible within the expanded conceptual framework, only a few theories have been sufficiently developed so far. By and large, these are theories based on various types of generalized measures, which are formalized in the language of classical set theory. Fuzzification of these theories, which can be done in different ways, has been explored only to some degree and only for standard fuzzy sets. One important result of research in the area of GIT is that the tremendous diversity of uncertainty theories made possible by the expanded framework is made tractable due to some key properties of these theories that are invariant across the whole spectrum or, at least, within broad classes of uncertainty theories. One important class of uncertainty theories consists of theories that are viewed as theories of imprecise probabilities. Some of these theories are based on Choquet capacities of various orders, especially capacities of order infinity (the well known theory of evidence), interval-valued probability distributions, and Sugeno λ-measures. While these theories are distinct in many respects, they share several common representations, such as representation by lower and upper probabilities, convex sets of probability distributions, and so-called Mobius representation. These representations are uniquely convertible to one another, and each may be used as needed. Another unifying feature of the various theories of imprecise probabilities is that two types of uncertainty coexist in each of them. These are usually referred to as nonspecificity and conflict. It is significant that well-justified measures of these two types of uncertainty are expressed by functionals of the same form in all the investigated theories of imprecise probabilities, even though these functionals are subject to different calculi in different theories. Moreover, equations that express relationship between marginal, joint, and conditional measures of uncertainty are invariant across the whole spectrum of theories of imprecise probabilities. The tremendous diversity of possible uncertainty theories is thus compensated by their many commonalities.