Exploiting local label correlation from sample perspective for multi-label classification via three-way decision theory

Abstract

In multi-label classification, the expansion of output dimension seriously interferes learning performance, and even fails to build a joint prediction model. In order to restrain the proliferation of multi-label classifier’s hypothesis space, the current works focus on the application of global positive label correlation. However, the “black or white” mechanism ignore other possible forms of label correlation, such as negative or neutral correlation. By introducing the doctrine of the mean, three-way decision (3WD) theory provides a solution for in-depth research on local label correlation, and aims to handle the uncertainty of multi-label learning tasks. In this paper, a novel learning algorithm for multi-label joint classification, namely ML-3WD, is proposed by considering the 3WD label correlation from the perspective of samples. According to the weights of different features on any label, the comprehensive loss of each sample to three action strategies can be measured. Obviously, the 3WD rules for any label variable in multi-label output space is obtained. By aggregating the cutting thresholds between different labels, the division principles of 3WD label correlation are further established. Given any multi-label sample, the local fuzzy membership to co-occurrence or mutual state for label pair is examined based on kernelized fuzzy rough sets. The 3WD local label relevance of each sample is confirmed, that is, positive, negative or neutral. The global application strategy for multi-label classification is utilized to avoid over-fitting induced by local mining strategy. Based on the integral mean of the distribution of 3WD local label relevance in multi-label sample space, two different versions of empirical label relevance are constructed. By constraining the relative position between sub-separation hyperplanes, the 3WD label correlation distribution-based model for multi-label joint classification is designed. The experiment results on fifteen real world multi-label datasets reflect that our algorithm achieves good classification ability and versatility. The impact of core parameters on learning performance is also dissected.