An Efficient Stacking Model of Multi-Label Classification Based on Pareto Optimum

Abstract

Nowadays, multi-label data are ubiquitous in real-world applications, in which each instance is associated with a set of labels. Multi-label learning has attracted significant attentions from researchers and plenty of algorithms have been proposed. Among those algorithms binary relevance (BR) is a widely used framework for multi-label classification. It constructs binary classifiers for each label by means of one-vs-rest style. BR approach is a simple and straight forward way of problem transformation for multi-label learning, but it ignores label correlations totally. Stacking based BR is a feasible way to tackle this problem. The key issue of stacking based BR is how to select label subset to extend the original features for each label. Existing methods of stacking based BR usually select identical label subset for all labels. It may be suboptimal as each label has its own most related label subset. In this paper, a novel stacking based method is introduced to utilize label correlations based on Pareto Optimum for improving the performance of BR. Our method builds a stack of two layers of BR classifiers. At the first layer, a group of binary classifiers are constructed, one for a label. At the second layer, for each label we employ Pareto Optimum to select most related label subset, then augment the original features by the selected label subset. The final binary classifiers for each label are constructed based on their corresponding reconstructed feature space. Comparing to other well-established stacking multi-label learning algorithms in terms of different multi-label classification criteria, experimental results on several multi-label benchmark datasets testify the superiority of the proposed methods.