RAndom k-labELsets Research Articles

Local positive and negative correlation-based k-labelsets for multi-label classification

Abstract Multi-label classification has attracted great attention from different domains. RAkEL (Random k-label sets) is an effective high-order multi-label learning approach. However, this method exploits label correlations randomly, which cannot make full use of label correlations, causing poor performance of sub-classifiers. In the paper, a novel ensemble method is proposed for multi-label classification. Instead of randomly selecting small-size subsets, this paper constructs a set of k-labelset based on local positive and negative pairwise label correlations, which are captured by KNN and matrix similarity measure. Furthermore, considering the possible missing or noisy labels in predictions, a rectification mechanism based on local label correlations is established to lessen global errors, which enhances the accuracy of the classification procedure. The experimental results over eight benchmark datasets based on multiple evaluation metrics show that these two strategies can achieve relatively better performance compared with the RAkEL method and other state-of-the-art approaches.

Study of data transformation techniques for adapting single-label prototype selection algorithms to multi-label learning

In this paper, the focus is on the application of prototype selection to multi-label data sets as a preliminary stage in the learning process. There are two general strategies when designing Machine Learning algorithms that are capable of dealing with multi-label problems: data transformation and method adaptation. These strategies have been successfully applied in obtaining classifiers and regressors for multi-label learning. Here we investigate the feasibility of data transformation in obtaining prototype selection algorithms for multi-label data sets from three prototype selection algorithms for single-label. The data transformation methods used were: binary relevance, dependent binary relevance, label powerset, and random k-labelsets. The general conclusion is that the methods of prototype selection obtained using data transformation are not better than those obtained through method adaptation. Moreover, prototype selection algorithms designed for multi-label do not do an entirely satisfactory job, because, although they reduce the size of the data set, without affecting significantly the accuracy, the classifier trained with the reduced data set does not improve the accuracy of the classifier when it is trained with the whole data set.

A non-intrusive load monitoring system using multi-label classification approach

This paper proposes an experimental design process for the application of energy disaggregation using multi-label classification. The classification approach has recently shown to be a suitable representative model for treating data disaggregation in which the primary task is to identify or predict multiple load usage from aggregate data. The experiments were conducted by collecting basic electrical parameters from individual circuit branches of the load distribution in a house. Sets of data were analyzed through machine learning process to select the optimal set of parameters, learning algorithm and model parameter so that the system resulted from the learning process could deliver the optimal predictive performance for appliance loads. By taking the electrical parameters of current (I), real power (P), reactive power (Q), and power factor (PF) at every one-minute and employing RAkEL (RAndom k-labELsets) with Decision Tree as the multi-label classification algorithm together with the right model parameter configuration. F-score and prediction accuracy were evaluated as the predictive performance which found to be 97% and 99%, respectively, for high power appliances (water heater, Air-conditioner); 59% and 93%, respectively for lightings; finally, 75% and 92%, respectively for plug-outlet utilities.

Progressive random k-labelsets for cost-sensitive multi-label classification

In multi-label classification, an instance is associated with multiple relevant labels, and the goal is to predict these labels simultaneously. Many real-world applications of multi-label classification come with different performance evaluation criteria. It is thus important to design general multi-label classification methods that can flexibly take different criteria into account. Such methods tackle the problem of cost-sensitive multi-label classification (CSMLC). Most existing CSMLC methods either suffer from high computational complexity or focus on only certain specific criteria. In this work, we propose a novel CSMLC method, named progressive random k-labelsets (PRAkEL), to resolve the two issues above. The method is extended from a popular multi-label classification method, random k-labelsets, and hence inherits its efficiency. Furthermore, the proposed method can handle arbitrary example-based evaluation criteria by progressively transforming the CSMLC problem into a series of cost-sensitive multi-class classification problems. Experimental results demonstrate that PRAkEL is competitive with existing methods under the specific criteria they can optimize, and is superior under other criteria.

Ensemble methods for multi-label classification

Ensemble methods have been shown to be an effective tool for solving multi-label classification tasks. In the RAndom k-labELsets (RAKEL) algorithm, each member of the ensemble is associated with a small randomly-selected subset of k labels. Then, a single label classifier is trained according to each combination of elements in the subset. In this paper we adopt a similar approach, however, instead of randomly choosing subsets, we select the minimum required subsets of k labels that cover all labels and meet additional constraints such as coverage of inter-label correlations. Construction of the cover is achieved by formulating the subset selection as a minimum set covering problem (SCP) and solving it by using approximation algorithms. Every cover needs only to be prepared once by offline algorithms. Once prepared, a cover may be applied to the classification of any given multi-label dataset whose properties conform with those of the cover. The contribution of this paper is two-fold. First, we introduce SCP as a general framework for constructing label covers while allowing the user to incorporate cover construction constraints. We demonstrate the effectiveness of this framework by proposing two construction constraints whose enforcement produces covers that improve the prediction performance of random selection by achieving better coverage of labels and inter-label correlations. Second, we provide theoretical bounds that quantify the probabilities of random selection to produce covers that meet the proposed construction criteria. The experimental results indicate that the proposed methods improve multi-label classification accuracy and stability compared to the RAKEL algorithm and to other state-of-the-art algorithms.

Multilabel classification using error-correcting codes of hard or soft bits.

We formulate a framework for applying error-correcting codes (ECCs) on multilabel classification problems. The framework treats some base learners as noisy channels and uses ECC to correct the prediction errors made by the learners. The framework immediately leads to a novel ECC-based explanation of the popular random k-label sets (RAKEL) algorithm using a simple repetition ECC. With the framework, we empirically compare a broad spectrum of off-the-shelf ECC designs for multilabel classification. The results not only demonstrate that RAKEL can be improved by applying some stronger ECC, but also show that the traditional binary relevance approach can be enhanced by learning more parity-checking labels. Our research on different ECCs also helps to understand the tradeoff between the strength of ECC and the hardness of the base learning tasks. Furthermore, we extend our research to ECC with either hard (binary) or soft (real-valued) bits by designing a novel decoder. We demonstrate that the decoder improves the performance of our framework.

Random k-Labelsets for Multilabel Classification

A simple yet effective multilabel learning method, called label powerset (LP), considers each distinct combination of labels that exist in the training set as a different class value of a single-label classification task. The computational efficiency and predictive performance of LP is challenged by application domains with large number of labels and training examples. In these cases, the number of classes may become very large and at the same time many classes are associated with very few training examples. To deal with these problems, this paper proposes breaking the initial set of labels into a number of small random subsets, called labelsets and employing LP to train a corresponding classifier. The labelsets can be either disjoint or overlapping depending on which of two strategies is used to construct them. The proposed method is called RAkEL (RAndom k labELsets), where k is a parameter that specifies the size of the subsets. Empirical evidence indicates that RAkEL manages to improve substantially over LP, especially in domains with large number of labels and exhibits competitive performance against other high-performing multilabel learning methods.