Multi-label classification by polytree-augmented classifier chains with label-dependent features

Lu Sun,Mineichi Kudo

doi:10.1007/s10044-018-0711-6

Abstract

Multi-label classification faces several critical challenges, including modeling label correlations, mitigating label imbalance, removing irrelevant and redundant features, and reducing the complexity for large-scale problems. To address these issues, in this paper, we propose a novel method—polytree-augmented classifier chains with label-dependent features—that models label correlations through flexible polytree structures based on low-dimensional label-dependent feature spaces learned by a two-stage feature selection approach. First, a feature weighting approach is applied to efficiently remove irrelevant features for each label and mitigate the effect of label imbalance. Second, a polytree structure is built in the label space using estimated conditional mutual information. Third, an appropriate label-dependent feature subset is found by taking account of label correlations in the polytree. Extensive empirical studies on six synthetic datasets and 12 real-world datasets demonstrate the superior performance of the proposed method. In addition, by incorporating the proposed two-stage feature selection approach, the multi-label classifiers with label-dependent features achieve on average 9.4% performance improvement in Exact-Match compared with the original classifiers.

Highlights

In recent years we have witnessed the increasing demand of multi-label classification (MLC) in a wide range of applications, such as text categorization, semantic image annotation, bioinformatics analysis and audio emotion detection, for which numerous machine learning techniques have been designed and successfully utilized
We develop a two-stage feature selection approach for Classifier Chains (CC)-based methods based on the simple filter algorithm, in order to find label-dependent, equivalently class-dependent, features [1] and save label correlations during feature selection
Polytree-Augmented Classifier Chains (PACC)-Label-Dependent Feature (LDF) was compared with three popular MLC methods

Summary

Introduction

In recent years we have witnessed the increasing demand of multi-label classification (MLC) in a wide range of applications, such as text categorization, semantic image annotation, bioinformatics analysis and audio emotion detection, for which numerous machine learning techniques have been designed and successfully utilized. Unlike traditional multi-class single-label classification, where each instance is associated with only a single label, the task of MLC is to assign a label subset to an unseen instance. The existing MLC methods fall into two broad categories: problem transformation and algorithm adaptation [35]. Problem transformation strategy typically transforms an MLC problem into a set of single-label classification problems, and learns a family of classifiers for modeling the single-label memberships. Algorithm adaptation strategy induces conventional machine learning algorithms in the multi-label settings. A number of MLC methods adopting one of the above two strategies have been developed and succeeded in dealing with various multi-label problems

Objectives

Results

Conclusion