Finding the Best Performance of Bayesian and Naïve Bayes Models in Fraudulent Firms Classification through Varying Threshold

Abstract

—Fraud detection is the first step to preventing fraud committed by both individuals and organizations. The development of a high-performance classification model to detect fraud is an interesting topic in machine learning modeling. A finding of the best Bayesian and Naive Bayes classification models is a crucial issue because both models are simple and easily applied models in the fields of life and social sciences. This study aims to obtain the best performance of classification models developed based on probability concepts, namely Bayesian and Naive Bayes models. Adding a threshold value to the decision-making criteria of the two models is an effort expected to can find models that perform superiorly. Data on the auditing of fraudulent firms containing of 775 firms from various business sectors in Australia is used as a case study. The testing data consisting of 100 instances were taken by cluster random sampling with a proportion of 61 non-fraudulent and 39 fraudulent firms and the remaining instances as the training data. The best Bayesian model has an average accuracy of 84% obtained at a threshold value of 0.22. While the best Naive Bayes model has an average accuracy of 94% which is obtained in the 15 threshold values. Adding the threshold value has a significant impact on the performance of the Bayesian model, which can increase the average accuracy from 36% to 84%. On the other hand, the average accuracy of the Naive Bayes model only increased by 1%, from 93% to 94%. Performance measures Sensitivity, Specificity, F1 score, and ROC curve of the Naive Bayes model are also superior to the Bayesian model.