Clustering Based Data Preprocessing Technique to Deal with Imbalanced Dataset Problem in Classification Task

Abstract

Dealing with imbalanced dataset is an important data preprocessing activity in the classification task. Several solutions such as data preprocessing using re-sampling, cost based learning, ensemble learning techniques have been proposed in the past to deal with imbalanced dataset problem. In this paper, we propose a new clustering based oversampling and undersampling (CBOUS) data preprocessing technique to deal with imbalanced dataset problem in the classification task. In order to analyze and compare performance of the proposed data preprocessing technique with the existing techniques, we have used three different classifiers namely C4.5, k Nearest Neighbor, Logistic Regression, and 37 different datasets. Area under receiver operating characteristic curve is used as a measure to assess performance of the classifiers. The experimental result shows that the proposed clustering based oversampling and undersampling technique achieves better classifier performance.