Modeling using support vector machines on imbalanced data: A case study on the prediction of the sightings of Irrawaddy dolphins

Abstract

Support vector machines (SVMs) is a powerful machine learning algorithm for classification particularly in medical, image processing and text analysis related studies. Nonetheless, its application in ecology is scarce. This study aims to demonstrate and compare the classification performance of SVMs models developed with weights and models developed with adoption of systematic random under-sampling technique in predicting a one-class independent dataset. The data used is a typical imbalanced real-world data with 700 data points where only 11% are sighted data points. Conversely, the one-class independent real-world dataset, with twenty data points, used for prediction consists of sighted data only. Both datasets are characterized with seven attributes. The results show that the former models have reported overall accuracy ranged between 87.62% and 90% with G-mean between 0% and 30.07% (0% to 9.09% sensitivity and 97.34% to 100% specificity) while the ROC-AUC values ranged between 75.92% and 88.78%. The latter models have reported overall accuracy ranged between 67.39% and 78.26% with G-mean between 66.51% and 76.30% (78.26% to 95.65% sensitivity and 52.17% to 60.87% specificity) while the ROC-AUC values ranged between 72.59% and 85.82%. Nevertheless, the former models could barely predict the independent dataset successfully. Majority of the models fail to predict a single sighted data point and the best prediction accuracy reported is 30%. The classification performance of the latter models is surprisingly encouraging where majority of the models manage to achieve more than 30% prediction accuracy. In addition, many of the models are capable to attain 65% prediction accuracy, more than double the performance of the former models. Current study thus suggests that, where highly imbalanced ecology data is concerned, modeling using SVMs adopting systematic random under-sampling technique is a more promising mean than w-SVM in obtaining much rewarding classification results for a one-class independent dataset.