Most Probable Explanation Research Articles

Neural Network Approximators for Marginal MAP in Probabilistic Circuits

Probabilistic circuits (PCs) such as sum-product networks efficiently represent large multi-variate probability distributions. They are preferred in practice over other probabilistic representations, such as Bayesian and Markov networks, because PCs can solve marginal inference (MAR) tasks in time that scales linearly in the size of the network. Unfortunately, the most probable explanation (MPE) task and its generalization, the marginal maximum-a-posteriori (MMAP) inference task remain NP-hard in these models. Inspired by the recent work on using neural networks for generating near-optimal solutions to optimization problems such as integer linear programming, we propose an approach that uses neural networks to approximate MMAP inference in PCs. The key idea in our approach is to approximate the cost of an assignment to the query variables using a continuous multilinear function and then use the latter as a loss function. The two main benefits of our new method are that it is self-supervised, and after the neural network is learned, it requires only linear time to output a solution. We evaluate our new approach on several benchmark datasets and show that it outperforms three competing linear time approximations: max-product inference, max-marginal inference, and sequential estimation, which are used in practice to solve MMAP tasks in PCs.

Analysis of factors affecting the severity of marine accidents using a data-driven Bayesian network

A data-driven Bayesian network model (BN) is used to analyse the relationship between the severity of marine accidents and relevant Accident Influential Factors (AIFs). Firstly, based on the marine accident investigation reports involving 1,294 ships from 2000 to 2019, the severity grades of marine accidents are classified, and a database of factors affecting the severity of marine accidents is established. Secondly, a Tree Augmented Naive Bayesian algorithm (TAN) is used to establish a data-driven BN model, and the established database of AIFs is analysed by data training and machine learning to reveal the influence of related factors on the severity of the accident and the mechanism of action. Finally, the sensitivity analysis and verification of the model are conducted. Through the analysis of the Most Probable Explanation (MPE), it explains the possible configurations in different scenarios and identifies the related potential risks. This study finds that “accident type”, “engine power”, “gross tonnage”, “ship type” and “location” are the five most important AIFs of three accident severity grades. “Capsizing/sinking”, “hull/machinery damage” and “collision” that are most likely to lead to “very serious accidents”. Further, the possibility of fishing boats or other small ships leading to “very serious accidents” is also higher than that of other types of ships. The results of this study can help to analyse and predict marine accidents and ensure the safe navigation of ships and hence benefit such maritime stakeholders as safety authorities and ship owners.

MAP Inference for Probabilistic Logic Programming

AbstractIn Probabilistic Logic Programming (PLP) the most commonly studied inference task is to compute the marginal probability of a query given a program. In this paper, we consider two other important tasks in the PLP setting: the Maximum-A-Posteriori (MAP) inference task, which determines the most likely values for a subset of the random variables given evidence on other variables, and the Most Probable Explanation (MPE) task, the instance of MAP where the query variables are the complement of the evidence variables. We present a novel algorithm, included in the PITA reasoner, which tackles these tasks by representing each problem as a Binary Decision Diagram and applying a dynamic programming procedure on it. We compare our algorithm with the version of ProbLog that admits annotated disjunctions and can perform MAP and MPE inference. Experiments on several synthetic datasets show that PITA outperforms ProbLog in many cases.

Incorporation of human factors into maritime accident analysis using a data-driven Bayesian network

A data-driven Bayesian network (BN) is used to investigate the effect of human factors on maritime safety through maritime accident analysis. Its novelties consist of (1) manual collection and analysis of the primary data representing frequencies of risk factors directly derived from maritime accident reports, (2) incorporation of human factors into causational analysis with respect to different maritime accident types, and (3) modelling by a historical accident data-driven approach, to generate new insights on critical human factors contributing to different types of accidents. The modelling of the interdependency among the risk influencing factors is structured by Tree Augmented Network (TAN), and validated by both sensitivity analysis and past accident records. Our findings reveal that the critical risk factors for all accident types are ship age, ship operation, voyage segment, information, and vessel condition. More importantly, the findings also present the differentiation among the vital human factors against different types of accidents. Most probable explanation (MPE) is used to provide a specific scenario in which the beliefs are upheld, observing the most probable configuration. The work pioneers the analysis of various impacts of human factors on different maritime accident types. It helps provide specific recommendations for the prevention of a particular type of accidents involving human errors.

Discriminative structure learning of sum–product networks for data stream classification

Sum–product network (SPN) is a deep probabilistic representation that allows for exact and tractable inference. There has been a trend of online SPN structure learning from massive and continuous data streams. However, online structure learning of SPNs has been introduced only for the generative settings so far. In this paper, we present an online discriminative approach for SPNs for learning both the structure and parameters. The basic idea is to keep track of informative and representative examples to capture the trend of time-changing class distributions. Specifically, by estimating the goodness of model fitting of data points and dynamically maintaining a certain amount of informative examples over time, we generate new sub-SPNs in a recursive and top-down manner. Meanwhile, an outlier-robust margin-based log-likelihood loss is applied locally to each data point and the parameters of SPN are updated continuously using most probable explanation (MPE) inference. This leads to a fast yet powerful optimization procedure and improved discrimination capability between the genuine class and rival classes. Empirical results show that the proposed approach achieves better prediction performance than the state-of-the-art online structure learner for SPNs, while promising order-of-magnitude speedup. Comparison with state-of-the-art stream classifiers further proves the superiority of our approach.

Evaluating Seismic Soil Liquefaction Potential Using Bayesian Belief Network and C4.5 Decision Tree Approaches

Liquefaction is considered a damaging phenomenon of earthquakes and a major cause of concern in civil engineering. Therefore, its predictory assessment is an essential task for geotechnical experts. This paper investigates the performance of Bayesian belief network (BBN) and C4.5 decision tree (DT) models to evaluate seismic soil liquefaction potential based on the updated and relatively large cone penetration test (CPT) dataset (which includes 251 case histories), comparing them to a simplified procedure and an evolutionary-based approach. The BBN model was developed using the K2 machine learning algorithm and domain knowledge (DK) with data fusion methodology, while the DT model was created using a C4.5 algorithm. This study shows that the BBN model is preferred over the others for evaluation of seismic soil liquefaction potential. Owing to its overall performance, simplicity in practice, data-driven characteristics, and ability to map interactions between variables, the use of a BBN model in assessing seismic soil liquefaction is quite promising. The results of a sensitivity analysis show that ‘equivalent clean sand penetration resistance’ is the most significant factor affecting liquefaction potential. This study also interprets the probabilistic reasoning of the robust BBN model and most probable explanation (MPE) of seismic soil liquefied sites, based on an engineering point of view.

Deep Convolutional Sum-Product Networks

We give conditions under which convolutional neural networks (CNNs) define valid sum-product networks (SPNs). One subclass, called convolutional SPNs (CSPNs), can be implemented using tensors, but also can suffer from being too shallow. Fortunately, tensors can be augmented while maintaining valid SPNs. This yields a larger subclass of CNNs, which we call deep convolutional SPNs (DCSPNs), where the convolutional and sum-pooling layers form rich directed acyclic graph structures. One salient feature of DCSPNs is that they are a rigorous probabilistic model. As such, they can exploit multiple kinds of probabilistic reasoning, including marginal inference and most probable explanation (MPE) inference. This allows an alternative method for learning DCSPNs using vectorized differentiable MPE, which plays a similar role to the generator in generative adversarial networks (GANs). Image sampling is yet another application demonstrating the robustness of DCSPNs. Our preliminary results on image sampling are encouraging, since the DCSPN sampled images exhibit variability. Experiments on image completion show that DCSPNs significantly outperform competing methods by achieving several state-of-the-art mean squared error (MSE) scores in both left-completion and bottom-completion in benchmark datasets.

Accelerating exact and approximate inference for (distributed) discrete optimization with GPUs

Discrete optimization is a central problem in artificial intelligence. The optimization of the aggregated cost of a network of cost functions arises in a variety of problems including (W)CSP, DCOP, as well as optimization in stochastic variants such as the tasks of finding the most probable explanation (MPE) in belief networks. Inference-based algorithms are powerful techniques for solving discrete optimization problems, which can be used independently or in combination with other techniques. However, their applicability is often limited by their compute intensive nature and their space requirements. This paper proposes the design and implementation of a novel inference-based technique, which exploits modern massively parallel architectures, such as those found in Graphical Processing Units (GPUs), to speed up the resolution of exact and approximated inference-based algorithms for discrete optimization. The paper studies the proposed algorithm in both centralized and distributed optimization contexts. The paper demonstrates that the use of GPUs provides significant advantages in terms of runtime and scalability, achieving up to two orders of magnitude in speedups and showing a considerable reduction in execution time (up to 345 times faster) with respect to a sequential version.

Bayesian network-based formulation and analysis for toll road utilization supported by traffic information provision

Congestion pricing has been proposed and investigated as an effective means of optimizing traffic assignment, alleviating congestion, and enhancing traffic operation efficiencies. Meanwhile, advanced traffic information dissemination systems, such as Advanced Traveler Information System (ATIS), have been developed and deployed to provide real-time, accurate, and complete network-wide traffic information to facilitate travelers’ trip plans and routing selections. Recent advances in ATIS technologies, especially telecommunication technology, allow dynamic, personalized, and multimodal traffic information to be disseminated and impact travelers’ choices of departure times, alternative routes, and travel modes in the context of congestion pricing. However, few studies were conducted to determine the impact of traffic information dissemination on toll road utilizations. In this study, the effects of the provisions of traffic information on toll road usage are investigated and analyzed based on a stated preference survey conducted in Texas. A Bayesian Network (BN)-based approach is developed to discover travelers’ opinions and preferences for toll road utilization supported by network-wide traffic information provisions. The probabilistic interdependencies among various attributes, including routing choice, departure time, traffic information dissemination mode, content, coverage, commuter demographic information, and travel patterns, are identified and their impacts on toll road usage are quantified. The results indicate that the BN model performs reasonably well in travelers’ preference classifications for toll road utilization and knowledge extraction. The BN Most Probable Explanation (MPE) measurement, probability inference and variable influence analysis results illustrate travelers using highway advisory radio and internet as their primary mode of receiving traffic information are more likely to comply with routing recommendations and use toll roads. Traffic information regarding congested roads, road hazard warnings, and accident locations is of great interest to travelers, who tend to acquire such information and use toll roads more frequently. Travel time formation for home-based trips can considerably enhance travelers’ preferences for toll road usage. Female travelers tend to seek traffic information and utilize toll roads more frequently. As expected, the information provided at both pre-trip and en-route stages can positively influence travelers’ preferences for toll road usage. The proposed methodology and research findings advance our previous study and provide insight into travelers’ behavioral tendencies concerning toll road utilization in support of traffic information dissemination.

Extracting recurrent scenarios from narrative texts using a Bayesian network: Application to serious occupational accidents with movement disturbance

A probabilistic approach has been developed to extract recurrent serious Occupational Accident with Movement Disturbance (OAMD) scenarios from narrative texts within a prevention framework. Relevant data extracted from 143 accounts was initially coded as logical combinations of generic accident factors. A Bayesian Network (BN)-based model was then built for OAMDs using these data and expert knowledge. A data clustering process was subsequently performed to group the OAMDs into similar classes from generic factor occurrence and pattern standpoints. Finally, the Most Probable Explanation (MPE) was evaluated and identified as the associated recurrent scenario for each class. Using this approach, 8 scenarios were extracted to describe 143 OAMDs in the construction and metallurgy sectors. Their recurrent nature is discussed.Probable generic factor combinations provide a fair representation of particularly serious OAMDs, as described in narrative texts. This work represents a real contribution to raising company awareness of the variety of circumstances, in which these accidents occur, to progressing in the prevention of such accidents and to developing an analysis framework dedicated to this kind of accident.

Multi-dimensional classification with Bayesian networks

Multi-dimensional classification aims at finding a function that assigns a vector of class values to a given vector of features. In this paper, this problem is tackled by a general family of models, called multi-dimensional Bayesian network classifiers (MBCs). This probabilistic graphical model organizes class and feature variables as three different subgraphs: class subgraph, feature subgraph, and bridge (from class to features) subgraph. Under the standard 0–1 loss function, the most probable explanation (MPE) must be computed, for which we provide theoretical results in both general MBCs and in MBCs decomposable into maximal connected components. Moreover, when computing the MPE, the vector of class values is covered by following a special ordering (gray code). Under other loss functions defined in accordance with a decomposable structure, we derive theoretical results on how to minimize the expected loss. Besides these inference issues, the paper presents flexible algorithms for learning MBC structures from data based on filter, wrapper and hybrid approaches. The cardinality of the search space is also given. New performance evaluation metrics adapted from the single-class setting are introduced. Experimental results with three benchmark data sets are encouraging, and they outperform state-of-the-art algorithms for multi-label classification.

Memory intensive AND/OR search for combinatorial optimization in graphical models

In this paper we explore the impact of caching during search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depth-first AND/OR Branch-and-Bound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching scheme similar to good and no-good recording. Furthermore, we present best-first search algorithms for traversing the same underlying AND/OR search graph and compare both algorithms empirically. We focus on two common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in belief networks and solving Weighted CSPs (WCSP). In an extensive empirical evaluation we demonstrate conclusively the superiority of the memory intensive AND/OR search algorithms on a variety of benchmarks.

A machine learning approach to algorithm selection for $\mathcal{NP}$ -hard optimization problems: a case study on the MPE problem

Given one instance of an \(\mathcal{NP}\) -hard optimization problem, can we tell in advance whether it is exactly solvable or not? If it is not, can we predict which approximate algorithm is the best to solve it? Since the behavior of most approximate, randomized, and heuristic search algorithms for \(\mathcal{NP}\) -hard problems is usually very difficult to characterize analytically, researchers have turned to experimental methods in order to answer these questions. In this paper we present a machine learning-based approach to address the above questions. Models induced from algorithmic performance data can represent the knowledge of how algorithmic performance depends on some easy-to-compute problem instance characteristics. Using these models, we can estimate approximately whether an input instance is exactly solvable or not. Furthermore, when it is classified as exactly unsolvable, we can select the best approximate algorithm for it among a list of candidates. In this paper we use the MPE (most probable explanation) problem in probabilistic inference as a case study to validate the proposed methodology. Our experimental results show that the machine learning-based algorithm selection system can integrate both exact and inexact algorithms and provide the best overall performance comparing to any single candidate algorithm.

Complexity Results and Approximation Strategies for MAP Explanations

MAP is the problem of finding a most probable instantiation of a set of variables given evidence. MAP has always been perceived to be significantly harder than the related problems of computing the probability of a variable instantiation Pr, or the problem of computing the most probable explanation (MPE). This paper investigates the complexity of MAP in Bayesian networks. Specifically, we show that MAP is complete for NP^PP and provide further negative complexity results for algorithms based on variable elimination. We also show that MAP remains hard even when MPE and Pr become easy. For example, we show that MAP is NP-complete when the networks are restricted to polytrees, and even then can not be effectively approximated. Given the difficulty of computing MAP exactly, and the difficulty of approximating MAP while providing useful guarantees on the resulting approximation, we investigate best effort approximations. We introduce a generic MAP approximation framework. We provide two instantiations of the framework; one for networks which are amenable to exact inference Pr, and one for networks for which even exact inference is too hard. This allows MAP approximation on networks that are too complex to even exactly solve the easier problems, Pr and MPE. Experimental results indicate that using these approximation algorithms provides much better solutions than standard techniques, and provide accurate MAP estimates in many cases.

Complexity of probabilistic reasoning in directed-path singly-connected Bayes networks

Directed-path (DP) singly-connected Bayesian networks are an interesting special case that, in particular, includes both polytrees and two-level networks. We analyze the computational complexity of these networks. The prediction problem is shown to be easy, as standard message passing can perform correct updating. However, diagnostic reasoning is hard even for DP singly-connected networks. In addition, finding the most-probable explanation (MPE) is hard, even without evidence. Finally, complexity of nearly DP singly-connected networks is analyzed.

Mini-buckets

This article presents a class of approximation algorithms that extend the idea of bounded-complexity inference, inspired by successful constraint propagation algorithms, to probabilistic inference and combinatorial optimization. The idea is to bound the dimensionality of dependencies created by inference algorithms. This yields a parameterized scheme, called mini-buckets , that offers adjustable trade-off between accuracy and efficiency. The mini-bucket approach to optimization problems, such as finding the most probable explanation (MPE) in Bayesian networks, generates both an approximate solution and bounds on the solution quality. We present empirical results demonstrating successful performance of the proposed approximation scheme for the MPE task, both on randomly generated problems and on realistic domains such as medical diagnosis and probabilistic decoding.

An Evolutionary Computing Approach to Probabilistic Reasoning on Bayesian Networks

Bayesian belief networks can be used to represent and to reason about complex systems with uncertain or incomplete information. Bayesian networks are graphs capable of encoding and quantifying probabilistic dependence and conditional independence among variables. Diagnostic reasoning, also referred to as abductive inference, determining the most probable explanation (MPE), or finding the maximum a posteriori instantiation (MAP), involves determining the global most probable system description given the values of any subset of variables. In some cases abductive inference can be performed with exact algorithms using distributed network computations, but the problem is NP-hard, and complexity increases significantly with the presence of undirected cycles, the number of discrete states per variable, and the number of variables in the network. This paper describes an approximate method composed of a graph-based evolutionary algorithm that uses nonbinary alphabets, graphs instead of strings, and graph operators to perform abductive inference on multiply connected networks for which systematic search methods are not feasible. The motivation, basis, and adequacy of the method are discussed, and experimental results are presented.

A multilevel GMRF-based approach to image segmentation and restoration

Abstract In this paper, the Gibbs-Markov approach is extended to integration of observations provided by virtual sensors and organized according to a hierarchical taxonomy. The proposed extension is applied to image restoration and segmentation. A model of coupled Gibbs-Markov random fields (GMRFs) is presented, which involves performing restoration and labeling at two abstraction levels, i.e., the image (pixel) level and the region level. The maximum a posteriori (MAP) approach usually applied as an estimation criterion for single-level GMRFs is shown to be a special case of the most probable explanation (MPE) criterion, which is valid for multilevel GMRFs. A stochastic distributed optimization algorithm is used to reach the solution.

The hydrolysis of chloroform and carbon tetrachloride by anion-exchange resins

Chloroform in acetone or ethanol solvent was rapidly hydrolysed to carbon monoxide and formate by Amberlite XE-78 in the OH-form. Hydroxyl sites on the resin were converted into chloride. The apparent loss in capacity of the resin which occurs during the reaction cannot be wholly accounted for by production of formate. The most probable explanation for the phenomenon is the adsorption of carbon trichloride anions at resin sites. Carbon tetrachloride in ketonic solvents also gave carbon monoxide and formate in the presence of the OH-form of the resin, but in alcohol or ether solvent carbon dioxide and phosgene were formed.