Multi-label classification with weighted classifier selection and stacked ensemble

Abstract

Multi-label classification has attracted increasing attention in various applications, such as medical diagnosis and semantic annotation. With such trend, a large number of ensemble approaches have been proposed for multi-label classification tasks. Most of these approaches construct the ensemble members by using bagging schemes, but few stacked ensemble approaches are developed. Existing research on stacked ensemble approaches remains active, but several issues remain such as (1) little has been done to learn the weights of classifiers for classifier selection; (2) the relationship between pairwise label correlations and multi-label classification performance has not been investigated sufficiently. To address these issues, we propose a novel stacked ensemble approach that simultaneously exploits label correlations and the process of learning weights of ensemble members. In our approach, first, a weighted stacked ensemble with sparsity regularization is developed to facilitate classifier selection and ensemble members construction for multi-label classification. Second, in order to improve the classification performance, the pairwise label correlations are further considered for determining weights of these ensemble members. Finally, we develop an optimization algorithm based on both of the accelerated proximal gradient and the block coordinate descent techniques to achieve the optimal ensemble solution efficiently. Extensive experiments on publicly available datasets and real Cardiovascular and Cerebrovascular Disease datasets demonstrate that our proposed algorithm outperforms related state-of-the-art methods from perspectives of benchmarking and real-world applications.