Analyzing the Impact of the Threshold on Machine Learning Models for Credit Risk Prediction Using Business Intelligence

Abstract

Making decisions based on predictions generated from machine learning models requires users to have a clear understanding of the mechanisms and logic behind every prediction. From one side, business users must be convinced in the ability of the models to generate correct predictions. Predictive power, expressed by the different performance measures, is not sufficient for building trust and acceptance of machine learning models. Business users need additional techniques and tools for model interpretation and evaluation of the effects from decisions based on machine learning predictions. In this research paper we propose an approach for analyzing the impact of different thresholds for converting probabilities into predictions for binomial classification machine learning models applicable for credit risk prediction in Peer-to-Peer Lending platforms. We define a set of measures to explore global and local impact on decision-making process and present different scenarios in a Business Intelligence application built in an analytical and business intelligent platform. Based on the presented results we can draw conclusions that when choosing the best model and threshold users should consider a broad set of measures not only for model accuracy, but also should consider misclassification costs, financial results, asset portfolio structure, etc.