Exploring Multi-Objective Optimization for Multi-Label Classifier Ensembles

Abstract

Multi-label classification deals with the task of predicting multiple class labels for a given sample. Several performance metrics are designed in the literature to measure the quality of any multi-label classification technique. In general existing multi-label classification approaches focus on optimizing only a single performance measure. The current work builds on the hypothesis that a weighted ensemble of multiple multi-label classifiers will lead to obtain improved results. The appropriate weight combinations for combining the outputs of multiple classifiers can be selected after simultaneously optimizing different multi-label classification metrics like micro F1, hamming loss, 0/1 loss, accuracy, etc. The problem is posed as a multi-objective optimization problem where a set of 13 conflicting objective functions are simultaneously optimized using the search capability of a multi-objective genetic algorithm based technique, namely NSGA-II. The weights of votes for a classifier for different class labels vary over a range quantifying the degrees of confidence of a classifier for different class labels. Several base classifiers are utilized for solving the problem of multi-objective multi-label classification. The effectiveness of the proposed multi-objective based multi-label classifier ensemble approach is shown for 10 data sets of varying complexities, and comparisons are reported for several existing techniques. Obtained experimental results clearly illustrate the efficacy of the proposed technique.