Multidimensional Classification Problems Research Articles

Receiver Operating Characteristic Optimization Based on Convex Hull and Evolutionary Algorithm

Classification is basically a Multi-objective problem. The efficiency of classification vastly depends on performance of a classifier which can be evaluated on the basis of Receiver Operating Characteristic (ROC) graph, Area under Curve (AUC), and selection of different threshold values are generally used as a tool. In machine learning, generally, 2-D classifiers are available that deal with bi-objective problems where overlapping of class may occur i.e. sensitivity and specificity may overlap. Recently, multi-class classification in which classes are mutually exclusive is in research trends along with the evolutionary algorithm. The application of ROC graph is extended to evaluate multi-dimensional classification as it is cost sensitive learning. The goal of this paper is to gather recent achievements in the field at one place to analyze the classifier performance for multi-dimensional classification problems using convex hull and evolutionary algorithms. In this paper, we tried to cover all the existing recent advance techniques in maximization of ROC and proposed a convex hull and evolutionary algorithm based new model for ROC maximization.

Constraint-based and hybrid structure learning of multidimensional continuous-time Bayesian network classifiers

Learning the structure of continuous-time Bayesian networks directly from data has traditionally been performed using score-based structure learning algorithms. Only recently has a constraint-based method been proposed, proving to be more suitable under specific settings, as in modelling systems with variables having more than two states. As a result, studying diverse structure learning algorithms is essential to learn the most appropriate models according to data characteristics and task-related priorities, such as learning speed or accuracy. This article proposes alternative algorithms for learning multidimensional continuous-time Bayesian network classifiers, introducing, for the first time, constraint-based and hybrid algorithms for these models. Nevertheless, these contributions also apply to the simpler one-dimensional classification problem for which only score-based solutions exist in the literature. More specifically, the aforementioned constraint-based structure learning algorithm is first adapted to the supervised classification setting. Then, a novel algorithm of this kind, specifically tailored for the multidimensional classification problem, is presented to improve the learning times for the induction of multidimensional classifiers. Finally, a hybrid algorithm is introduced, attempting to combine the strengths of the score- and constraint-based approaches. Experiments with synthetic and real-world data are performed not only to validate the capabilities of the proposed algorithms but also to conduct a comparative study of the available competitors.

Introducing multi-dimensional hierarchical classification: Characterization, solving strategies and performance measures

Classification problems where there exist multiple class variables that need to be jointly predicted are known as Multi-dimensional classification problems. If the labels of these class variables are organized as hierarchies, we can take advantage of specific strategies designed for the Hierarchical classification paradigm. In this paper we present the Multi-dimensional hierarchical classification (MDHC) paradigm, a result of the combination of Multi-dimensional and Hierarchical classification paradigms. We propose four MDHC learning strategies which are designed to exploit the particularities of this new paradigm, combining characteristics of Multi-dimensional and Hierarchical classification strategies. Along with these strategies, we present a framework for classifier comparison in which we use a set of performance measures specifically designed for MDHC, and a procedure to create MDHC synthetic scenarios. Using this framework and the performance measures presented, we study how characteristics of the MDHC problems influence the performance of the different MDHC strategies proposed, and compare them to other non-MDHC strategies.

Combining natural language processing and multidimensional classifiers to predict and correct CMMS metadata

Computerized maintenance management systems (CMMSs) contain valuable data on the maintenance operations in an organization. A large part of these data consists of unstructured, written texts contained in failure notifications which are generated each time an unexpected failure occurs, enriched with structured metadata consisting of a number of labels that allow to categorize the failures, such as the type of failure, its cause or the corrective action that was taken. In this paper, we show that natural language processing techniques can be used to predict the structured metadata based on the unstructured text and even identify mislabeled notifications or ambiguous labels. Specific attention is given to the complexity that arises from the highly technical nature of the texts combined with a telegraphic writing style and heavy use of sentence fragments and abbreviations. Moreover, it is shown that exploiting dependencies between different components of the metadata, and regarding the prediction problem as a multidimensional classification problem, can improve the reliability of the predicted labels. We illustrate and test our label prediction pipeline on the CMMS data of a large pharmaceutical company.

Decomposition-Based Classifier Chains for Multi-Dimensional Classification

In multi-dimensional classification, the semantics of objects are characterized by multiple class variables from different dimensions. To model the dependencies among class variables, one natural strategy is to build a number of multiclass classifiers in a chaining structure, one per dimension, where the subsequent classifiers on the chain augment the feature space with all labeling information used by the preceding classifiers. However, it is shown that this strategy cannot compete with existing state-of-the-art approaches via comparative studies. One possible reason is that inaccurate predictions of preceding classifiers would degenerate the performance of subsequent ones. Besides, it is more difficult to learn a multiclass classifier than a binary one with the same accuracy, and better performance can be expected if the multi-dimensional classification problem can be solved by building multiple binary classifiers in a chaining structure. Based on these conjectures, this article proposes an approach, which builds a chain of binary classifiers to solve the multi-dimensional classification problem with the help of one-versus-one decomposition. To address the issue that different one-versus-one decomposed problems involve different training examples, the feature space is augmented with the binary predictions of preceding classifiers on the chain to train the subsequent ones. To alleviate the effect of the specified chaining order, the ensemble version of the proposed approach is further investigated. Comparative studies over 20 benchmark datasets clearly show the superiority of the proposed approach against the state-of-the-art multi-dimensional classification baselines.

Multidimensional continuous time Bayesian network classifiers

The multidimensional classification of multivariate time series deals with the assignment of multiple classes to time-ordered data described by a set of feature variables. Although this challenging task has received almost no attention in the literature, it is present in a wide variety of domains, such as medicine, finance or industry. The complexity of this problem lies in two nontrivial tasks, the learning with multivariate time series in continuous time and the simultaneous classification of multiple class variables that may show dependencies between them. These can be addressed with different strategies, but most of them may involve a difficult preprocessing of the data, high space and classification complexity or ignoring useful interclass dependencies. Additionally, no attention has been given to the development of new multidimensional classifiers of time series based on probabilistic graphical models, even though transparent models can facilitate further understanding of the domain. In this paper, a novel probabilistic graphical model is proposed, which is able to classify a discrete multivariate temporal sequence into multiple class variables while modeling their dependencies. This model extends continuous time Bayesian networks to the multidimensional classification problem, which are able to explicitly represent the behavior of time series that evolve over continuous time. Different methods for the learning of the parameters and structure of the model are presented, and numerical experiments on synthetic and real-world data show encouraging results in terms of performance and learning time with respect to independent classifiers, the current alternative approach under the continuous time Bayesian network paradigm.

Multi-Dimensional Classification with Super-Classes

The multi-dimensional classification problem is a generalization of the recently-popularized task of multi-label classification, where each data instance is associated with multiple class variables. There has been relatively little research carried out specific to multi-dimensional classification and, although one of the core goals is similar (modeling dependencies among classes), there are important differences; namely a higher number of possible classifications. In this paper we present method for multi-dimensional classification, drawing from the most relevant multi-label research, and combining it with important novel developments. Using a fast method to model the conditional dependence between class variables, we form super-class partitions and use them to build multi-dimensional learners, learning each super-class as an ordinary class, and thus explicitly modeling class dependencies. Additionally, we present a mechanism to deal with the many class values inherent to super-classes, and thus make learning efficient. To investigate the effectiveness of this approach we carry out an empirical evaluation on a range of multi-dimensional datasets, under different evaluation metrics, and in comparison with high-performing existing multi-dimensional approaches from the literature. Analysis of results shows that our approach offers important performance gains over competing methods, while also exhibiting tractable running time.

Multidimensional hierarchical classification

Hierarchical classification can be seen as a multidimensional classification problem where the objective is to predict a class, or set of classes, according to a taxonomy. There have been different proposals for hierarchical classification, including local and global approaches. Local approaches can suffer from the inconsistency problem, that is, if a local classifier has a wrong prediction, the error propagates down the hierarchy. Global approaches tend to produce more complex models. In this paper, we propose an alternative approach inspired in multidimensional classification. It starts by building a multi-class classifier per each parent node in the hierarchy. In the classification phase, all the local classifiers are applied simultaneously to each instance, providing a probability for each class in the taxonomy. Then the probability of the subset of classes, for each path in the hierarchy, is obtained by combining the local classifiers results. The path with highest probability is returned as the result for all the levels in the hierarchy. As an extension of the proposal method, we also developed a new technique, based on information gain, to classifies at different levels in the hierarchy. The proposed method was tested on different hierarchical classification data sets and was compared against state-of-the-art methods, resulting in superior predictive performance and/or efficiency to the other approaches in all the datasets.

Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers

Our aim is to use multi-dimensional Bayesian network classifiers in order to predict the human immunodeficiency virus type 1 (HIV-1) reverse transcriptase and protease inhibitors given an input set of respective resistance mutations that an HIV patient carries. Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models especially designed to solve multi-dimensional classification problems, where each input instance in the data set has to be assigned simultaneously to multiple output class variables that are not necessarily binary. In this paper, we introduce a new method, named MB-MBC, for learning MBCs from data by determining the Markov blanket around each class variable using the HITON algorithm. Our method is applied to both reverse transcriptase and protease data sets obtained from the Stanford HIV-1 database. Regarding the prediction of antiretroviral combination therapies, the experimental study shows promising results in terms of classification accuracy compared with state-of-the-art MBC learning algorithms. For reverse transcriptase inhibitors, we get 71% and 11% in mean and global accuracy, respectively; while for protease inhibitors, we get more than 84% and 31% in mean and global accuracy, respectively. In addition, the analysis of MBC graphical structures lets us gain insight into both known and novel interactions between reverse transcriptase and protease inhibitors and their respective resistance mutations. MB-MBC algorithm is a valuable tool to analyze the HIV-1 reverse transcriptase and protease inhibitors prediction problem and to discover interactions within and between these two classes of inhibitors.

Markov blanket-based approach for learning multi-dimensional Bayesian network classifiers: An application to predict the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39)

Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models recently proposed to deal with multi-dimensional classification problems, where each instance in the data set has to be assigned to more than one class variable. In this paper, we propose a Markov blanket-based approach for learning MBCs from data. Basically, it consists of determining the Markov blanket around each class variable using the HITON algorithm, then specifying the directionality over the MBC subgraphs. Our approach is applied to the prediction problem of the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39) in order to estimate the health-related quality of life of Parkinson’s patients. Fivefold cross-validation experiments were carried out on randomly generated synthetic data sets, Yeast data set, as well as on a real-world Parkinson’s disease data set containing 488 patients. The experimental study, including comparison with additional Bayesian network-based approaches, back propagation for multi-label learning, multi-label k-nearest neighbor, multinomial logistic regression, ordinary least squares, and censored least absolute deviations, shows encouraging results in terms of predictive accuracy as well as the identification of dependence relationships among class and feature variables.

Hidden Markov models-based 3D MRI brain segmentation

This paper introduces a 3D MRI segmentation algorithm based on Hidden Markov Models (HMMs). The mathematical models for the HMM that forms the basis of the segmentation algorithm for both the continuous and discrete cases are developed and contrasted with Hidden Markov Random Field in terms of complexity and extensibility to larger fields. The presented algorithm clearly demonstrates the capacity of HMM to tackle multi-dimensional classification problems. The HMM-based segmentation algorithm was evaluated through application to simulated brain images from the McConnell Brain Imaging Centre, Montréal Neurological Institute, McGill University as well as real brain images from the Internet Brain Segmentation Repository (IBSR), Harvard University. The HMM model exhibited high accuracy in segmenting the simulated brain data and an even higher accuracy when compared to other techniques applied to the IBSR 3D MRI data sets. The achieved accuracy of the segmentation results is attributed to the HMM foundation and the utilization of the 3D model of the data. The IBSR 3D MRI data sets encompass various levels of difficulty and artifacts that were chosen to pose a wide range of challenges, which required handling of sudden intensity variations and the need for global intensity level correction and 3D anisotropic filtering. During segmentation, each class of MR tissue was assigned to a separate HMM and all of the models were trained using the discriminative MCE training algorithm. The results were numerically assessed and compared to those reported using other techniques applied to the same data sets, including manual segmentations establishing the ground truth for real MR brain data. The results obtained using the HMM-based algorithm were the closest to the manual segmentation ground truth in terms of an objective measure of overlap compared to other methods.