Probabilistic Reasoning Systems Research Articles

The application of probabilistic programming in statistics

This article focuses on three aspects of Probabilistic reasoning systems for applications, including Predict future events, Infer the cause of events, and learn from past events to better predict future events. In addition, the article explains that the data will become more accurate as the number of samples increases, and finally, the article describes the application of Turing-Completed Systems in statistics, a kind of probabilistic programming system. This article starts with a soccer competition as an example, then will explain the concepts in depth behind the example.

Using diverging predictions from classical and quantum models to dissociate between categorization systems

Quantum probability theory has successfully provided accurate descriptions of behavior in the areas of judgment and decision making, and here we apply the same principles to two category learning tasks, one task using information-integration categories and the other using rule-based categories. Since information-integration categories lack verbalizable descriptions, unlike rule-based ones, we assert that an information-integration categorization decision results from an intuitive probabilistic reasoning system characterized by quantum probability theory, whereas a rule-based categorization decision results from a logical, rational probabilistic reasoning system characterized classical probability theory. In our experiment, participants learn to categorize simple, visual stimuli as members of either category S or category K during an acquisition phase, and then rate the likelihood on a scale of 0 to 5 that a stimulus belongs to one category and subsequently perform the same likelihood rating for the other category during a transfer phase. Following the principle of complementarity in quantum theory, we expect the category likelihood ratings to exhibit order effects in the information-integration task, but not in the rule-based task. In the information-integration task, we found definitive order effects in the likelihood ratings. But, in the rule-based task, we found that the order effects in the likelihood ratings are not significant.

Probabilistic reasoning system for social influence analysis in online social networks

People interact with other people in their daily life, either for work or for personal reasons. These interactions are often complex. Thus, interactions that an individual has with other individuals, to some extent, influence the decisions they make. There have been many efforts to uncover, explore, and measure the concept of social influence. Thus, modeling influence is an open and challenging problem where most evaluation models focus on online social networks. However, they fail to characterize some properties of social influence. To address the limitations of the previous approaches, we propose a novel Probabilistic Reasoning system for social INfluence analysis (PRIN) to examine the social influence process and elucidate the factors that affect it in an attempt to explain this phenomenon. In this paper, we present a model that quantitatively measures social influence in online social networks. Experiments on a real social network such as Twitter demonstrate that the proposed model significantly outperforms traditional feature engineering-based approaches. This suggests the effectiveness of this novel model when modeling and predicting social influence.

On Testing of Uniform Samplers

Recent years have seen an unprecedented adoption of artificial intelligence in a wide variety of applications ranging from medical diagnosis, automobile industry, security to aircraft collision avoidance. Probabilistic reasoning is a key component of such modern artificial intelligence systems. Sampling techniques form the core of the state of the art probabilistic reasoning systems.
 The divide between the existence of sampling techniques that have strong theoretical guarantees but fail to scale and scalable techniques with weak or no theoretical guarantees mirrors the gap in software engineering between poor scalability of classical program synthesis techniques and billions of programs that are routinely used by practitioners. One bridge connecting the two extremes in the context of software engineering has been program testing. In contrast to testing for deterministic programs, where one trace is sufficient to prove the existence of a bug, in case of samplers one sample is typically not sufficient to prove non-conformity of the sampler to the desired distribution. This makes one wonder whether it is possible to design testing methodology to test whether a sampler under test generates samples close to a given distribution.
 The primary contribution of this paper is an affirmative answer to the above question when the given distribution is a uniform distribution: We design, to the best of our knowledge, the first algorithmic framework, Barbarik, to test whether the distribution generated is ε−close or η−far from the uniform distribution. In contrast to the sampling techniques that require an exponential or sub-exponential number of samples for sampler whose support can be represented by n bits, Barbarik requires only O(1/(η−ε)4) samples. We present a prototype implementation of Barbarik and use it to test three state of the art uniform samplers over the support defined by combinatorial constraints. Barbarik can provide a certificate of uniformity to one sampler and demonstrate nonuniformity for the other two samplers.
 Erratum: This research is supported in part by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: [AISG-RP-2018-005])

Inference in multi-agent causal models

In this article, we demonstrate the usefulness of causal Bayesian networks as probabilistic reasoning systems. The biggest advantage of causal Bayesian networks over traditional probabilistic Bayesian networks is that they sometimes allow to perform causal inference, i.e. the calculation of the causal effect of one variable on other variables. We treat a state-of-the-art algorithm for performing causal inference that is based on a new factorization of the joint probability distribution and is a systematic approach for the calculation due to Tian and Pearl. We elaborate on the problems that can arise when working with a centralized approach and discuss how a decentralized cooperative multi-agent approach might overcome some of these problems. The main contribution of this article is the introduction of multi-agent causal models as a way to overcome the problems in a centralized setting. They are an extension of causal Bayesian networks to a distributed setting consisting of a number of agents each having access to an overlapping set of the variables. We extend a state-of-the-art causal inference algorithm for this particular domain. We will show that our approach is as powerful in computing causal effects as the centralized algorithm.

The Relational Structure of Belief Networks

This paper demonstrates the relational structure of belief networks by establishing an extended relational data model which can be applied to both belief networks and relational applications. It is demonstrated that a Markov network can be represented as a generalized acyclic join dependency (GAJD) which is equivalent to a set of conflict-free generalized multivalued dependencies (GMVDs). A Markov network can also be characterized by an entropy function, which greatly facilitates the manipulation of GMVDs. These results are extensions of results established in relational theory. It is shown that there exists a complete set of inference rules for the GMVDs. This result is important from a probabilistic perspective. All the above results explicitly demonstrate that there is a unified model for relational database and probabilistic reasoning systems. This is not only important from a theoretical point of view in that one model has been developed for a number of domains, but also from a practical point of view in that one system can be implemented for both domains. This implemented system can take advantage of the performance enhancing techniques developed in both fields. Thereby, this paper serves as a theoretical foundation for harmonizing these two important information domains.

On the implication problem for probabilistic conditional independency

The implication problem is to test whether a given set of independencies logically implies another independency. This problem is crucial in the design of a probabilistic reasoning system. We advocate that Bayesian networks are a generalization of standard relational databases. On the contrary, it has been suggested that Bayesian networks are different from the relational databases because the implication problem of these two systems does not coincide for some classes of probabilistic independencies. This remark, however, does not take into consideration one important issue, namely, the solvability of the implication problem. In this comprehensive study of the implication problem for probabilistic conditional independencies, it is emphasized that Bayesian networks and relational databases coincide on solvable classes of independencies. The present study suggests that the implication problem for these two closely related systems differs only in unsolvable classes of independencies. This means there is no real difference between Bayesian networks and relational databases, in the sense that only solvable classes of independencies are useful in the design and implementation of these knowledge systems. More importantly, perhaps, these results suggest that many current attempts to generalize Bayesian networks can take full advantage of the generalizations made to standard relational databases.

Partitioning the Attribute Set for a Probabilistic Reasoning System

The ability of a computerized system to model the reasoning process of humans has become an important area of research. This research considers a probabilistic reasoning system for applications that require decision making under uncertain conditions. The reasoning system captures the uncertainty associated with different feasible outcomes, and based on historical data, provides users with a measure of this uncertainty. To make accurate predictions, the scheme requires that predictive variables that are not conditionally independent of each other given the outcome be grouped into compound attributes for the purpose of estimating probabilities. These compound attributes partition the entire set of predictive attributes into disjoint sets. An important design issue, then, is that the appropriate partitioning scheme be obtained before the reasoning scheme is used in practice. We formulate the problem of finding the optimal partitioning scheme, and present five different (although related) heuristic techniques to induce partitions from historical cases. Using simulated data, all five techniques are shown to capture accurately underlying dependencies across attributes when a reasonable amount of historical data is available for analysis. In situations where few historical cases are available, the induced structures are less accurate. In such situations, the performance of induced structures for making probability predictions is nevertheless found to be as good as that when using the true structure. Finally, we test for external validity by applying the techniques on a real-world dataset on credit applications for a bank. We show, using this dataset, that (i) the classificatory performance of the reasoning system using structures generated by the heuristic techniques is as good as the performance of the widely used decision tree induction algorithm C4.5, and (ii) the induced structures are able to provide reliable probability estimates for making decisions in environments with asymmetric misclassification costs.

An Extended Relational Data Model For Probabilistic Reasoning

Probabilistic methods provide a formalism for reasoning about partial beliefs under conditions of uncertainty. This paper suggests a new representation of probabilistic knowledge. This representation encompasses the traditional relational database model. In particular, it is shown that probabilistic conditional independence is equivalent to the notion of generalized multivalued dependency. More importantly, a Markov network can be viewed as a generalized acyclic join dependency. This linkage between these two apparently different but closely related knowledge representations provides a foundation for developing a unified model for probabilistic reasoning and relational database systems.

A probabilistic reasoning model: Formulation and control strategy

It has been recognized that past experiences of a decision maker often plays a pivotal role in solving new problem instances. Therefore, the ability to model human reasoning processes has become an important subject of research in recent years. In many applications, the reasoning process must deal with uncertainty inherent in the problem domain. This research addresses the issue of supporting the model formulation and data acquisition processes for situations that (i) operate under uncertain conditions, and (ii) utilize evidential information that is gathered in stages. A theoretical framework is presented for the probabilistic formulation of the reasoning process that incorporates past experiences. The model is validated by testing its performance on simulated data, and is shown to work well when a sufficiently large number of cases are available for estimating probabilities. The probabilistic reasoning system can revise beliefs in an intuitively appealing and theoretically sound manner when information is acquired in an incremental fashion. Two dynamic information gathering strategies are discussed for such a reasoning system, one using information theoretic techniques, and the other using decision theoretic techniques.

Artificial intelligence: a modern approach

The long-anticipated revision of this #1 selling book offers the most comprehensive, state of the art introduction to the theory and practice of artificial intelligence for modern applications. Intelligent Agents. Solving Problems by Searching. Informed Search Methods. Game Playing. Agents that Reason Logically. First-order Logic. Building a Knowledge Base. Inference in First-Order Logic. Logical Reasoning Systems. Practical Planning. Planning and Acting. Uncertainty. Probabilistic Reasoning Systems. Making Simple Decisions. Making Complex Decisions. Learning from Observations. Learning with Neural Networks. Reinforcement Learning. Knowledge in Learning. Agents that Communicate. Practical Communication in English. Perception. Robotics. For computer professionals, linguists, and cognitive scientists interested in artificial intelligence.

Probabilistic reasoning in an augmented fact-proposition space and its applications

A simple and versatile probabilistic reasoning scheme is presented. Based on an augmentation of a multi-dimensional inference space indexed by a Cartesian product of the fact and proposition sets, the scheme simplifies the processes involved in the representation and computation of a probabilistic reasoning system. In the augmented space, a set of auxiliary fields is utilized in addition to the fact-proposition relations to manipulate the uncertainty and incompleteness of the information presented. The scheme enhances the functionality of a probabilistic reasoning and facilitates the building of practical reasoning systems. The utilization of the augmented space in reasoning is illustrated by two problems in computer-vision applications.

Information processing with fuzzy logic-Piero Bonissone

This is a review of a video on the foundations of approximate reasoning systems. The outline found in the visual materials accompanying the video refers to the lecture as fuzzy information systems (approximate reasoning systems). Almost half of the tape is devoted to probabilistic systems, but the presentation tends to justify the need for fuzzy methods in information processing, and so the title can be accepted as a final goal of the tape. The videotape and the accompanying material are intended to provide a basic introduction to the concept of knowledge-based systems in general, to introduce probabilistic and possibilistic reasoning systems, to highlight the main distinction between them, and to give the basis for compiling rule-based systems and fuzzy logic controllers. Examples are provided to illustrate the concepts and limitations of systems. The reviewer says that the tape is good for a first view of the field, or for a broad review of the concepts. It also provides a good perspective on how fuzzy techniques go into knowledge-based systems.