Multi-dimensional Bayesian Network Classifiers Research Articles

Multi-dimensional Bayesian network classifiers for partial label ranking

The label ranking problem consists in learning preference models from training datasets labeled with (possibly incomplete) rankings of the class labels. The goal is then to predict a ranking for a given unlabeled instance. This work focuses on a more general interpretation where both the training dataset and the prediction given as output allow tied class labels, i.e., there is no particular preference between them. This problem is known as the partial label ranking problem. This paper tackles the partial label ranking problem by transforming the ranking with ties into a set of discrete variables representing the preference relations (ranked ahead, tied with, and ranked behind) between each pair of class labels. The posterior probabilities for each pair are then used to fill the values of a preference matrix. This preference matrix is the basis for solving the rank aggregation problem required to obtain the output ranking with ties. This paper aims to exploit the resemblance of this problem with multi-label and multi-dimensional classification by studying the use of Bayesian network classifiers to compute the posterior probabilities for the new class structure, i.e., pairs of class labels. In particular, binary relevance with naive Bayes and averaged one-dependence estimators between the new class structure are used to solve the partial label ranking problem. Furthermore, bivariate relationships between all the pairs of class labels are considered. However, the complexity of the model grows significantly, which makes it necessary to reduce the number of allowed bivariate relationships between pairs. Thus, a feature selection method is included to select the more relevant subset of bivariate relationships. The experimental evaluation shows that our proposals are competitive in accuracy with the current instance-based and decision tree induction algorithms. Moreover, they outperform the existing mixture-based probabilistic graphical models, while the algorithms proposed are much faster.

Patient specific prediction of temporal lobe epilepsy surgical outcomes.

Drug-resistant temporal lobe epilepsy (TLE) is the most common type of epilepsy for which patients undergo surgery. Despite the best clinical judgment and currently available prediction algorithms, surgical outcomes remain variable. We aimed to build and to evaluate the performance of multidimensional Bayesian network classifiers (MBCs), a type of probabilistic graphical model, at predicting probability of seizure freedom after TLE surgery. Clinical, neurophysiological, and imaging variables were collected from 231TLE patients who underwent surgery at the University of California, San Francisco (UCSF) or the Montreal Neurological Institute (MNI) over a 15-year period. Postsurgical Engel outcomes at year 1 (Y1), Y2, and Y5 were analyzed as primary end points. We trained an MBC model on combined data sets from both institutions. Bootstrap bias corrected cross-validation (BBC-CV) was used to evaluate the performance of the models. The MBC was compared with logistic regression and Cox proportional hazards according to the area under the receiver-operating characteristic curve (AUC). The MBC achieved an AUC of 0.67 at Y1, 0.72 at Y2, and 0.67 at Y5, which indicates modest performance yet superior to what has been reported in the state-of-the-art studies to date. The MBC can more precisely encode probabilistic relationships between predictors and class variables (Engel outcomes), achieving promising experimental results compared to other well-known statistical methods. Multisite application of the MBC could further optimize its classification accuracy with prospective data sets. Online access to the MBC is provided, paving the way for its use as an adjunct clinical tool in aiding pre-operative TLE surgical counseling.

Multi-dimensional Bayesian network classifiers: A survey

Multi-dimensional classification is a cutting-edge problem, in which the values of multiple class variables have to be simultaneously assigned to a given example. It is an extension of the well known multi-label subproblem, in which the class variables are all binary. In this article, we review and expand the set of performance evaluation measures suitable for assessing multi-dimensional classifiers. We focus on multi-dimensional Bayesian network classifiers, which directly cope with multi-dimensional classification and consider dependencies among class variables. A comprehensive survey of this state-of-the-art classification model is offered by covering aspects related to their learning and inference process complexities. We also describe algorithms for structural learning, provide real-world applications where they have been used, and compile a collection of related software.

Tractability of most probable explanations in multidimensional Bayesian network classifiers

Multidimensional Bayesian network classifiers have gained popularity over the last few years due to their expressive power and their intuitive graphical representation. A drawback of this approach is that their use to perform multidimensional classification, a generalization of multi-label classification, can be very computationally demanding when there are a large number of class variables. Thus, a key challenge in this field is to ensure the tractability of these models during the learning process.In this paper, we show how information about the most common queries of multidimensional Bayesian network classifiers affects the complexity of these models. We provide upper bounds for the complexity of the most probable explanations and marginals of class variables conditioned to an instantiation of all feature variables. We use these bounds to propose efficient strategies for bounding the complexity of multidimensional Bayesian network classifiers during the learning process, and provide a simple learning method with an order-based search that guarantees the tractability of the returned models. Experimental results show that our approach is competitive with other methods in the state of the art and also ensures the tractability of the learned models.

Balanced sensitivity functions for tuning multi-dimensional Bayesian network classifiers

Multi-dimensional Bayesian network classifiers are Bayesian networks of restricted topological structure, which are tailored to classifying data instances into multiple dimensions. Like more traditional classifiers, multi-dimensional classifiers are typically learned from data and may include inaccuracies in their parameter probabilities. We will show that the graphical properties and dedicated use of these classifiers induce higher-order sensitivity functions of a highly constrained functional form in these parameters. We then introduce the concept of balanced sensitivity function in which multiple parameters are functionally related by the odds ratios of their original and new values, and argue that these functions provide for a suitable heuristic for tuning multi-dimensional classifiers with guaranteed bounds on the effects on their output probabilities. We demonstrate the practicability of our heuristic by means of a classifier for a real-world application in the veterinary field.

A hybrid method for learning multi-dimensional Bayesian network classifiers based on an optimization model

Bayesian networks, which have a solid mathematical basis as classifiers, take the prior information of samples into consideration. They have gained considerable popularity for solving classification problems. However, many real-world applications can be viewed as classification problems in which instances have to be assigned to a set of different classes at the same time. To address this problem, multi-dimensional Bayesian network classifiers (MBCs), which organize class and feature variables as three subgraphs, have recently been proposed. Because each subgraph has different structural restrictions, three different learning algorithms are needed. In this paper, we present for the first time an MBC learning algorithm based on an optimization model (MBC-OM) that is inspired by the constraint-based Bayesian network structure learning method. MBC-OM uses the chi-squared statistic and mutual information to estimate the dependence coefficients among variables, and these are used to construct an objective function as an overall measure of the dependence for a classifier structure. Therefore, the problem of searching for an optimal classifier becomes one of finding the maximum value of the objective function in feasible fields. We prove the existence and uniqueness of the numerical solution. Moreover, we validate our method on five benchmark data sets. Experimental results are competitive, and outperform state-of-the-art algorithms for multi-dimensional classification.

PREDICTING THE EQ-5D FROM THE PARKINSON'S DISEASE QUESTIONNAIRE PDQ-8 USING MULTI-DIMENSIONAL BAYESIAN NETWORK CLASSIFIERS

The impact of the Parkinson's disease and its treatment on the patients' health-related quality of life can be estimated either by means of generic measures such as the european quality of Life-5 Dimensions (EQ-5D) or specific measures such as the 8-item Parkinson's disease questionnaire (PDQ-8). In clinical studies, PDQ-8 could be used in detriment of EQ-5D due to the lack of resources, time or clinical interest in generic measures. Nevertheless, PDQ-8 cannot be applied in cost-effectiveness analyses which require generic measures and quantitative utility scores, such as EQ-5D. To deal with this problem, a commonly used solution is the prediction of EQ-5D from PDQ-8. In this paper, we propose a new probabilistic method to predict EQ-5D from PDQ-8 using multi-dimensional Bayesian network classifiers. Our approach is evaluated using five-fold cross-validation experiments carried out on a Parkinson's data set containing 488 patients, and is compared with two additional Bayesian network-based approaches, two commonly used mapping methods namely, ordinary least squares and censored least absolute deviations, and a deterministic model. Experimental results are promising in terms of predictive performance as well as the identification of dependence relationships among EQ-5D and PDQ-8 items that the mapping approaches are unable to detect.

Assisting in search heuristics selection through multidimensional supervised classification: A case study on software testing

A fundamental question in the field of approximation algorithms, for a given problem instance, is the selection of the best (or a suitable) algorithm with regard to some performance criteria. A practical strategy for facing this problem is the application of machine learning techniques. However, limited support has been given in the literature to the case of more than one performance criteria, which is the natural scenario for approximation algorithms. We propose multidimensional Bayesian network (mBN) classifiers as a relatively simple, yet well-principled, approach for helping to solve this problem. Precisely, we relax the algorithm selection decision problem into the elucidation of the nondominated subset of algorithms, which contains the best. This formulation can be used in different ways to elucidate the main problem, each of which can be tackled with an mBN classifier. Namely, we deal with two of them: the prediction of the whole nondominated set and whether an algorithm is nondominated or not. We illustrate the feasibility of the approach for real-life scenarios with a case study in the context of Search Based Software Test Data Generation (SBSTDG). A set of five SBSTDG generators is considered and the aim is to assist a hypothetical test engineer in elucidating good generators to fulfil the branch testing of a given programme.

Research on Multi-Dimensional Bayesian Network Classifiers Based on ICA Dimension Reduction

Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models proposed to solve classification problems. However, in data analysis and preprocessing tasks, one is often confronted with the problem of selecting features from very high dimensional data. To resolve this problem, the covariance analysis and the FastICA algorithm are applied to decrease the dimension and remove redundant information. And then, we only need to construct class subgraph and bridge subgraph of the MBC model with algorithm and mutual information from the processed data, since the new feature variables satisfy independence assumption. The experiment was tested on three benchmark data sets. The theoretically and experimental results show that our method outperforms other state-of-the-art algorithms for multi-dimensional classification in accuracy.

Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers

Our aim is to use multi-dimensional Bayesian network classifiers in order to predict the human immunodeficiency virus type 1 (HIV-1) reverse transcriptase and protease inhibitors given an input set of respective resistance mutations that an HIV patient carries. Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models especially designed to solve multi-dimensional classification problems, where each input instance in the data set has to be assigned simultaneously to multiple output class variables that are not necessarily binary. In this paper, we introduce a new method, named MB-MBC, for learning MBCs from data by determining the Markov blanket around each class variable using the HITON algorithm. Our method is applied to both reverse transcriptase and protease data sets obtained from the Stanford HIV-1 database. Regarding the prediction of antiretroviral combination therapies, the experimental study shows promising results in terms of classification accuracy compared with state-of-the-art MBC learning algorithms. For reverse transcriptase inhibitors, we get 71% and 11% in mean and global accuracy, respectively; while for protease inhibitors, we get more than 84% and 31% in mean and global accuracy, respectively. In addition, the analysis of MBC graphical structures lets us gain insight into both known and novel interactions between reverse transcriptase and protease inhibitors and their respective resistance mutations. MB-MBC algorithm is a valuable tool to analyze the HIV-1 reverse transcriptase and protease inhibitors prediction problem and to discover interactions within and between these two classes of inhibitors.

Supervised pre-processing approaches in multiple class variables classification for fish recruitment forecasting

A multi-species approach to fisheries management requires taking into account the interactions between species in order to improve recruitment forecasting of the fish species. Recent advances in Bayesian networks direct the learning of models with several interrelated variables to be forecasted simultaneously. These models are known as multi-dimensional Bayesian network classifiers (MDBNs). Pre-processing steps are critical for the posterior learning of the model in these kinds of domains. Therefore, in the present study, a set of ‘state-of-the-art’ uni-dimensional pre-processing methods, within the categories of missing data imputation, feature discretization and feature subset selection, are adapted to be used with MDBNs. A framework that includes the proposed multi-dimensional supervised pre-processing methods, coupled with a MDBN classifier, is tested with synthetic datasets and the real domain of fish recruitment forecasting. The correctly forecasting of three fish species (anchovy, sardine and hake) simultaneously is doubled (from 17.3% to 29.5%) using the multi-dimensional approach in comparison to mono-species models. The probability assessments also show high improvement reducing the average error (estimated by means of Brier score) from 0.35 to 0.27. Finally, these differences are superior to the forecasting of species by pairs.

Using Multidimensional Bayesian Network Classifiers to Assist the Treatment of Multiple Sclerosis

Multiple sclerosis is an autoimmune disorder of the central nervous system and potentially the most common cause of neurological disability in young adults. The clinical disease course is highly variable and different multiple sclerosis subtypes can be defined depending on the progression of the severity of the disease. In the early stages, the disease subtype is unknown, and there is no information about how the severity is going to evolve. As there are different treatment options available depending on the progression of the disease, early identification has become highly relevant. Thus, given a new patient, it is important to diagnose the disease subtype. Another relevant information to predict is the expected time to reach a severity level indicating that assistance for walking is required. Given that we have to predict two correlated class variables: disease subtype and time to reach certain severity level, we use multidimensional Bayesian network classifiers because they can model and exploit the relations among both variables. Besides, the obtained models can be validated by the physicians using their expert knowledge due to the interpretability of Bayesian networks. The learning of the classifiers is made by means of a novel multiobjective approach which tries to maximize the accuracy of both class variables simultaneously. The application of the methodology proposed in this paper can help a physician to identify the expected progression of the disease and to plan the most suitable treatment.

Markov blanket-based approach for learning multi-dimensional Bayesian network classifiers: An application to predict the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39)

Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models recently proposed to deal with multi-dimensional classification problems, where each instance in the data set has to be assigned to more than one class variable. In this paper, we propose a Markov blanket-based approach for learning MBCs from data. Basically, it consists of determining the Markov blanket around each class variable using the HITON algorithm, then specifying the directionality over the MBC subgraphs. Our approach is applied to the prediction problem of the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39) in order to estimate the health-related quality of life of Parkinson’s patients. Fivefold cross-validation experiments were carried out on randomly generated synthetic data sets, Yeast data set, as well as on a real-world Parkinson’s disease data set containing 488 patients. The experimental study, including comparison with additional Bayesian network-based approaches, back propagation for multi-label learning, multi-label k-nearest neighbor, multinomial logistic regression, ordinary least squares, and censored least absolute deviations, shows encouraging results in terms of predictive accuracy as well as the identification of dependence relationships among class and feature variables.

Multi-dimensional classification with Bayesian networks

Multi-dimensional classification aims at finding a function that assigns a vector of class values to a given vector of features. In this paper, this problem is tackled by a general family of models, called multi-dimensional Bayesian network classifiers (MBCs). This probabilistic graphical model organizes class and feature variables as three different subgraphs: class subgraph, feature subgraph, and bridge (from class to features) subgraph. Under the standard 0–1 loss function, the most probable explanation (MPE) must be computed, for which we provide theoretical results in both general MBCs and in MBCs decomposable into maximal connected components. Moreover, when computing the MPE, the vector of class values is covered by following a special ordering (gray code). Under other loss functions defined in accordance with a decomposable structure, we derive theoretical results on how to minimize the expected loss. Besides these inference issues, the paper presents flexible algorithms for learning MBC structures from data based on filter, wrapper and hybrid approaches. The cardinality of the search space is also given. New performance evaluation metrics adapted from the single-class setting are introduced. Experimental results with three benchmark data sets are encouraging, and they outperform state-of-the-art algorithms for multi-label classification.

The Pathwidth and Treewidth of Cographs

It is shown that the pathwidth of a cograph equals its treewidth, and a linear time algorithm to determine the pathwidth of a cograph and build a corresponding path-decomposition is given.