Improving performance of multi-label classification using ensemble of feature selection and outlier detection

Abstract

Nowadays classification has become one of the most common techniques in machine learning. In classification, there are two types of data; single-label and multi-label. In multilabel datasets, one sample can have multiple labels at the same time. In recent years, classification of multi-label data has gained a lot of attention. Multi-label classification algorithms can be divided into 3 main parts: problem transformation methods, algorithm adaptation methods and ensemble methods. In problem transformation methods, classification of multi-label data is transformed to other fields. In algorithm adaptation methods, common single-label classification algorithms are changed so that they can deal with multi-label data. In third category, algorithms of two previous categories are combined together. Despite of many different proposed algorithms in this field, improvement of methods in terms of evaluation metrics has always been a challenge. Also, there is a lack of systems which can self-improve the base classifier. Thus, in this paper we try to present a novel ensemble system which can improve any classifier. The presented system has a novel structure which consists of two tree ensembles and each one has its own specific function. One of them has the task of removing noisy and outlier data with a novel approach and the other one has the task of removing noisy and redundant features. In one group some random samples are selected and in the other one, some random features are selected. If the evaluation metrics of the created child are improved, the algorithm can go to the next step and create its own child and if not, the parents create another child. Lastly, the results of these groups are combined together. The conducted experiments on 10 various datasets and 5 evaluation metrics show the superiority of the proposed method.