Convex Sets Of Probability Distributions Research Articles

Credal-C4.5: Decision tree based on imprecise probabilities to classify noisy data

In the area of classification, C4.5 is a known algorithm widely used to design decision trees. In this algorithm, a pruning process is carried out to solve the problem of the over-fitting. A modification of C4.5, called Credal-C4.5, is presented in this paper. This new procedure uses a mathematical theory based on imprecise probabilities, and uncertainty measures. In this way, Credal-C4.5 estimates the probabilities of the features and the class variable by using imprecise probabilities. Besides it uses a new split criterion, called Imprecise Information Gain Ratio, applying uncertainty measures on convex sets of probability distributions (credal sets). In this manner, Credal-C4.5 builds trees for solving classification problems assuming that the training set is not fully reliable. We carried out several experimental studies comparing this new procedure with other ones and we obtain the following principal conclusion: in domains of class noise, Credal-C4.5 obtains smaller trees and better performance than classic C4.5.

Special issue on “uncertainty modeling and analysis with intervals: foundations, tools, applications”

Verification and validation assessment of process modeling and simulation is increasing in importance in various areas of application. They include complex mechatronic and biomechanical tasks with especially strict requirements on numerical accuracy and performance. However, engineers lack precise knowledge regarding the process and its input data. This lack of knowledge and the inherent inexactness in measurement make such general verification and validation cycle tasks as design of a formal model and definition of relevant parameters and their ranges difficult to complete. To assess how reliable a system is, verification and validation analysts have to deal with uncertainty. There are two types of uncertainty: aleatory and epistemic. Aleatory uncertainty refers to variability similar to that arising in games of chance. It cannot be reduced by further empirical study. Epistemic (reducible) uncertainty refers to the incertitude resulting from lack of knowledge. An example is the absence of evidence about the probability distribution of a parameter. Here, interval methods provide a possible solution strategy. Another option, mostly discussed in the context of risk analysis, is to use interval-valued probabilities and imprecisely specified probability distributions. The probability of an event can be specified as an interval; probability bounds analysis propagates constraints on a distribution function through mathematical operations. In a more general setting, the theory of imprecise probabilities is a powerful conceptual framework in which uncertainty is represented by closed, convex sets of probability distributions. Bayesian sensitivity analysis or Dempster-Shafer theory are further options. As the guest editors of this special issue, we are pleased to introduce a collection of articles that were presented and discussed at a Dagstuhl Seminar 11371 (http://www. dagstuhl.de/11371) ‘‘Uncertainty modeling and analysis with intervals—Foundations, tools, applications’’, which took place September 11–16, 2011. The major emphasis of the seminar was on modeling and analyzing uncertainties and propagating them through application systems by using interval arithmetic. This special issue collects twelve papers which present various aspects of the investigations based on interval arithmetic. On one hand, there are theoretical and methodological contributions (Q. Fazal and A. Neumaier, W. Lodwick and O. Jenkins, E. Popova and M. Hladik, F. Zapata et al.). On the other hand, there are presentations of software frameworks for verified scientific computing and modelling of complex uncertain systems (O. Heimlich et al., M. Zimmer et al.), as well as some applications (T. Dotschel et al., S. Kiel et al., P. Shao and N. Stewart). There is also a group of papers which compare or combine interval and probabilistic approaches (M. Beer and V. Kreinovich, G. Rebner et al., Y. Wang). V. Kreinovich (&) Department of Computer Science, University of Texas at El Paso, 500 W. University, El Paso, TX 79968, USA e-mail: vladik@utep.edu

Event‐Tree Analysis with Imprecise Probabilities

Novel methods are proposed for dealing with event-tree analysis under imprecise probabilities, where one could measure chance or uncertainty without sharp numerical probabilities and express available evidence as upper and lower previsions (or expectations) of gambles (or bounded real functions). Sets of upper and lower previsions generate a convex set of probability distributions (or measures). Any probability distribution in this convex set should be considered in the event-tree analysis. This article focuses on the calculation of upper and lower bounds of the prevision (or the probability) of some outcome at the bottom of the event-tree. Three cases of given information/judgments on probabilities of outcomes are considered: (1) probabilities conditional to the occurrence of the event at the upper level; (2) total probabilities of occurrences, that is, not conditional to other events; (3) the combination of the previous two cases. Corresponding algorithms with imprecise probabilities under the three cases are explained and illustrated by simple examples.

Robust Filtering Through Coherent Lower Previsions

The classical filtering problem is re-examined to take into account imprecision in the knowledge about the probabilistic relationships involved. Imprecision is modeled in this paper by closed convex sets of probabilities. We derive a solution of the state estimation problem under such a framework that is very general: it can deal with any closed convex set of probability distributions used to characterize uncertainty in the prior, likelihood, and state transition models. This is made possible by formulating the theory directly in terms of coherent lower previsions, that is, of the lower envelopes of the expectations obtained from the set of distributions. The general solution is specialized to two particular classes of coherent lower previsions. The first consists of a family of Gaussian distributions whose means are only known to belong to an interval. The second is the so-called linear-vacuous mixture model, which is a family made of convex combinations of a known nominal distribution (e.g., a Gaussian) with arbitrary distributions. For the latter case, we empirically compare the proposed estimator with the Kalman filter. This shows that our solution is more robust to the presence of modelling errors in the system and that, hence, appears to be a more realistic approach than the Kalman filter in such a case.

Maximising entropy on the nonparametric predictive inference model for multinomial data

The combination of mathematical models and uncertainty measures can be applied in the area of data mining for diverse objectives with as final aim to support decision making. The maximum entropy function is an excellent measure of uncertainty when the information is represented by a mathematical model based on imprecise probabilities. In this paper, we present algorithms to obtain the maximum entropy value when the information available is represented by a new model based on imprecise probabilities: the nonparametric predictive inference model for multinomial data (NPI-M), which represents a type of entropy-linear program. To reduce the complexity of the model, we prove that the NPI-M lower and upper probabilities for any general event can be expressed as a combination of the lower and upper probabilities for the singleton events, and that this model can not be associated with a closed polyhedral set of probabilities. An algorithm to obtain the maximum entropy probability distribution on the set associated with NPI-M is presented. We also consider a model which uses the closed and convex set of probability distributions generated by the NPI-M singleton probabilities, a closed polyhedral set. We call this model A-NPI-M. A-NPI-M can be seen as an approximation of NPI-M, this approximation being simpler to use because it is not necessary to consider the set of constraints associated with the exact model.

Robust stochastic control based on imprecise probabilities

AbstractThis paper deals with the optimal quadratic control problem for non Gaussian discrete-time linear stochastic systems from the perspective of imprecise probabilities. The adopted philosophy is to use a convex set of probability distributions to characterize the imprecision in the knowledge about the probabilistic relationships present in the system to be controlled. In particular, an uncertain system model, named Linear Gaussian Vacuous Mixture (LGVM), in which disturbances and initial state uncertainty are described as convex combinations (mixtures) of nominal Gaussian distributions and unknown vacuous distributions, is adopted. A novel control approach is then derived, according to a worst-case paradigm, by minimizing the upper expectation of a finite-horizon quadratic cost functional with respect to all admissible probability distributions and exploiting a receding horizon strategy. Simulation experiments demonstrate its robustness in presence of large unexpected impulsive disturbances.

An ensemble method using credal decision trees

Supervised classification learning can be considered as an important tool for decision support. In this paper, we present a method for supervised classification learning, which ensembles decision trees obtained via convex sets of probability distributions (also called credal sets) and uncertainty measures. Our method forces the use of different decision trees and it has mainly the following characteristics: it obtains a good percentage of correct classifications and an improvement in time of processing compared with known classification methods; it not needs to fix the number of decision trees to be used; and it can be parallelized to apply it on very large data sets.

Application of uncertainty measures on credal sets on the naive Bayesian classifier

The naive Bayes classifier is known to obtain good results with a simple procedure. The method is based on the independence of the attribute variables given the variable to be classified. In real databases, where this hypothesis is not verified, this classifier continues to give good results. In order to improve the accuracy of the method, various works have been carried out in an attempt to reconstruct the set of the attributes and to join them so that there is independence between the new sets although the elements within each set are dependent. These methods are included in the ones known as semi-naive Bayes classifiers. In this article, we present an application of uncertainty measures on closed and convex sets of probability distributions, also called credal sets, in classification. We represent the information obtained from a database by a set of probability intervals (a credal set) via the imprecise Dirichlet model and we use uncertainty measures on credal sets in order to reconstruct the set of attributes, such as those mentioned, which shall enable us to improve the result of the naive Bayes classifier in a satisfactory way.

Uncertainty measures on probability intervals from the imprecise Dirichlet model

When we use a mathematical model to represent information, we can obtain a closed and convex set of probability distributions, also called a credal set. This type of representation involves two types of uncertainty called conflict (or randomness) and non-specificity, respectively. The imprecise Dirichlet model (IDM) allows us to carry out inference about the probability distribution of a categorical variable obtaining a set of a special type of credal set (probability intervals). In this paper, we shall present tools for obtaining the uncertainty functions on probability intervals obtained with the IDM, which can enable these functions in any application of this model to be calculated.

Disaggregated total uncertainty measure for credal sets

We present a new approach to measure uncertainty/information applicable to theories based on convex sets of probability distributions, also called credal sets. A definition of a total disaggregated uncertainty measure on credal sets is proposed in this paper motivated by recent outcomes. This definition is based on the upper and lower values of Shannon's entropy for a credal set. We justify the use of the proposed total uncertainty measure and the parts into which it is divided: the maximum difference of entropies, which can be used as a non-specificity measure (imprecision), and the minimum of entropy, which represents a measure of conflict (contradiction).

Additivity of uncertainty measures on credal sets

A general way of representing incomplete information is to use closed and convex sets of probability distributions, which are also called credal sets. Each credal set is associated with uncertainty, whose amount is quantified by an appropriate uncertainty measure. One of the requisite properties of uncertainty measures is the property of additivity, which is associated with the concept of independence. For credal sets, the concept of independence is not unique. This means that different definitions of independence lead to different definitions of additivity for uncertainty measures. In this paper, we compare the various definitions of independence, but our principal aim is to analyze those definitions that are employed in the most significant uncertainty measures established in the literature for credal sets.

Building classification trees using the total uncertainty criterion

We present an application of the measure of total uncertainty on convex sets of probability distributions, also called credal sets, to the construction of classification trees. In these classification trees the probabilities of the classes in each one of its leaves is estimated by using the imprecise Dirichlet model. In this way, smaller samples give rise to wider probability intervals. Branching a classification tree can decrease the entropy associated with the classes but, at the same time, as the sample is divided among the branches the nonspecificity increases. We use a total uncertainty measure (entropy + nonspecificity) as branching criterion. The stopping rule is not to increase the total uncertainty. The good behavior of this procedure for the standard classification problems is shown. It is important to remark that it does not experience of overfitting, with similar results in the training and test samples. © 2003 Wiley Periodicals, Inc.

Nonlinear filtering of convex sets of probability distributions

A solution is provided to the problem of computing a convex set of conditional probability distributions that characterize the state of a nonlinear dynamic system as it evolves in time. The estimator uses the Galerkin approximation to solve Kolmogorov's equation for the diffusion of a continuous-time nonlinear system with discrete-time measurement updates. Filtering of the state is accomplished for a convex set of distributions simultaneously, and closed-form representations of the resulting sets of means and covariances are generated.

The naive credal classifier

Convex sets of probability distributions are also called credal sets. They generalize probability theory by relaxing the requirement that probability values be precise. Classification, i.e. assigning class labels to instances described by a set of attributes, is an important domain of application of Bayesian methods, where the naive Bayes classifier has a surprisingly good performance. This paper proposes a new method of classification which involves extending the naive Bayes classifier to credal sets. Exact and effective solution procedures for naive credal classification are derived, and the related dominance criteria are discussed. Credal classification appears as a new method, based on more realistic assumptions and in the direction of more reliable inferences.

A Non-Specificity Measure for Convex Sets of Probability Distributions

In belief functions, there are two types of uncertainty which are due to lack of knowledge: randomness and non-specificity. In this paper, we present a non-specificity measure for convex sets of probability distributions that generalizes Dubois and Prade's non-specificity measure in the Dempster-Shafer theory of evidence.

A NON-SPECIFICITY MEASURE FOR CONVEX SETS OF PROBABILITY DISTRIBUTIONS

In belief functions, there are two types of uncertainty which are due to lack of knowledge: randomness and non-specificity. In this paper, we present a non-specificity measure for convex sets of probability distributions that generalizes Dubois and Prade's non-specificity measure in the Dempster-Shafer theory of evidence.

A survey of concepts of independence for imprecise probabilities

Our aim in this paper is to clarify the notion of independence for imprecise probabilities. Suppose that two marginal experiments are each described by an imprecise probability model, i.e., by a convex set of probability distributions or an equivalent model such as upper and lower probabilities or previsions. Then there are several ways to define independence of the two experiments and to construct an imprecise probability model for the joint experiment. We survey and compare six definitions of independence. To clarify the meaning of the definitions and the relationships between them, we give simple examples which involve drawing balls from urns. For each concept of independence, we give a mathematical definition, an intuitive or behavioural interpretation, assumptions under which the definition is justified, and an example of an urn model to which the definition is applicable. Each of the independence concepts we study appears to be useful in some kinds of application. The concepts of strong independence and epistemic independence appear to be the most frequently applicable.

Bayesian network implementation of Levi's epistemic utility decision theory

Isaac Levi has proposed an epistemic decision rule that requires two convex sets of probability distributions: a set of credal probability distributions that represent a decision agent's state of knowledge, and a set of information-determining distributions that represent the decision agent's assessment of the informational value of various hypotheses. In this paper, we investigate the feasibility of using Bayesian network structures, in which conditional probability distributions are computed using local computations and conditional independence relationships, to implement Levi's decision rule. We find that Bayesian network update algorithms do not in general result in convex sets of distributions; however, Bayesian networks can compute sets of a posteriori extremal distributions from sets of a priori and conditional extremal distributions. We also show that Levi's decision rule gives the same answer when applied to arbitrary sets of credal and information-determining distributions as it gives when applied to the convex closure of those sets of distributions. Thus, implementation of Levi's decision rule using Bayesian network structures is feasible.

Non-homogeneous de Finetti-type theorems

De Finetti-type theorems in a more general sense state that a certain convex set of probability distributions on an infinite product space, defined by symmetry conditions, is a simplex, and give a description, of its extreme points. In the classical examples these are homogeneous product measures. Motivated by work of Diaconiset al.(4) we describe a general method to obtain certain nonhomogeneous product measures as extreme solutions to the defining symmetry equations, and give examples.

Set-values filtering and smoothing

A theory of discrete-time optimal filtering and smoothing based on convex sets of probability distributions is presented. Rather than propagating a single conditional distribution as does conventional Bayesian estimation, a convex set of conditional distributions is evolved. For linear Gaussian systems, the convex set can be generated by a set of Gaussian distributions with equal covariance with means in a convex region of state space. The conventional point-valued Kalman filter is generated to a set-valued Kalman filter consisting of equations of evolution of a convex set of conditional means and a conditional covariance. The resulting estimator is an exact solution to the problem of running an infinity of Kalman filters and fixed-interval smoothers, each with different initial conditions. An application is presented to illustrate and interpret the estimator results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>