A stacking weighted k-Nearest neighbour with thresholding

Abstract

• Stacking of binary classifiers to consider label correlations. • Thresholding the decision boundary. • Feature weighting in order to correlation adjustment. • Dealing with imbalanced data problem by optimizing F -measure. • Using gradient ascent method. In Multi-label problems, each instance is associated with a set of predefined labels. Binary Relevance, as a common approach, uses one binary classifier for each label and ignores the probable dependencies between some labels. Stacked-Binary Relevance (SBR) was proposed to consider the label dependencies, by augmenting a second layer of binary models using the predicted labels of the first level binary models as additional features. By reusing all predicted labels, SBR implicitly assumes full dependencies between labels, which is not usually a true assumption in the real world. Moreover, SBR uses a constant threshold in decision functions of the binary models, while adjusting the threshold for each label specially for imbalanced ones can improve the performance. This paper proposes a k -Nearest Neighbor stacking method that adjusts the thresholds in decision functions of the binary classifiers and uses a feature-weighted distance measure to reduce the effect of irrelevant labels in stacking. The method can leverage positive/negative and symmetric/asymmetric label dependencies expressed as feature weights. Also, it can tackle the main shortcomings of SBR (revealed in the existence of irrelevant labels and imbalanced data). Using 22 multi-label datasets, the proposed method is assessed and outperforms state-of-the-art methods presented in the literature.