Default Prediction Research Articles

Personal Loan Default Prediction Analysis based on TabNet and Logistic Regression

Abstract. With the continuous development of the global economy, the demand for loans from individuals and enterprises is growing. However, loan defaults have gradually become a major challenge facing the financial industry. Loan defaults not only directly affect the profitability of financial institutions, but may also trigger systemic risks and pose a potential threat to the entire economic system. Therefore, improving the accuracy of loan default prediction is crucial for financial institutions to effectively manage credit risks. This study built a system for personal loan default prediction by combining TabNet with the Logistic regression model. Through feature engineering, this study extracts potential credit risk features by utilizing datasets from personal and Internet loans. The accuracy of default prediction is improved by combining TabNet's feature learning capabilities with Logistic Regression's interpretability. An AUC value of 0.89 was achieved by the integrated model, which achieved a notable performance. The results indicate that a system that relies on machine learning to predict defaults can significantly enhance the quality of credit approval decisions and lower the likelihood of bad debts for financial institutions. Future studies could aim to optimize the feature selection process, experiment with more advanced machine learning algorithms, or apply the model to diverse loan datasets, thereby enhancing both its generalization and accuracy. In conclusion, this study offers a novel approach to loan default prediction, demonstrating significant practical value and providing substantial support for the risk management and decision-making processes of financial institutions.

Research on Credit Default Prediction Model Based on TabNet-Stacking.

With the development of financial technology, the traditional experience-based and single-network credit default prediction model can no longer meet the current needs. This manuscript proposes a credit default prediction model based on TabNeT-Stacking. First, use the PyTorch deep learning framework to construct an improved TabNet structure. The multi-population genetic algorithm is used to optimize the Attention Transformer automatic feature selection module. The particle swarm algorithm is used to optimize the hyperparameter selection and achieve automatic parameter search. Finally, Stacking ensemble learning is used, and the improved TabNet is used to extract features. XGBoost (eXtreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine), CatBoost (Category Boosting), KNN (K-NearestNeighbor), and SVM (Support Vector Machine) are selected as the first-layer base learners, and XGBoost is used as the second-layer meta-learner. The experimental results show that compared with original models, the credit default prediction model proposed in this manuscript outperforms the comparison models in terms of accuracy, precision, recall, F1 score, and AUC (Area Under the Curve) of credit default prediction results.

Advancing Financial Resilience: A Systematic Review of Default Prediction Models and Future Directions in Credit Risk Management

This research presents a systematic review of a substantial body of high-quality research articles on Default Prediction Models published from 2015 to 2024. It is a comprehensive analysis of a DPM wide spectrum approaches including Textual Models, Systematic Review Studies, Hybrid Models, Intelligent Models and Statistical Models. The reason behind this study is rooted in the critical need to mitigate and understand the credit default risk that poses a significant threat to financial stability worldwide. By employing an evidence-based approach and methodological rigorously, this research critically evaluates the gaps, effectiveness and evolution in existing DPM methodologies. It is not only synthesized the current landscape of DPM study but also identified the direction for the future research, by offering novel insights and bridging theoretical gaps for enhancing the strategies of credit risk management. This study stands out by focusing on high citation research from top tier publishers, ensuring the quality and relevance of its analysis. The findings of this study have profound implications for stakeholders across the financial sector, including bankers, investors, regulatory bodies, and researchers. It aims to advance financial stability by providing a comprehensive overview of DPM advancements and pointing towards areas that require further exploration. By doing so, it contributes significantly to the development of more effective and sophisticated DPM strategies, thereby enhancing the robustness of financial institutions against potential defaults.

Feature selection in P2P lending for default prediction using grey wolf optimization and machine learning

Feature selection in P2P lending for default prediction using grey wolf optimization and machine learning

Using machine learning to investigate the determinants of loan default in P2P lending: Are there differences between before and during COVID-19?

The coronavirus disease (COVID-19) has led to a persistent increase in the volatility of the credit market and triggered a series of financial distress and bankruptcy. To investigate whether there are differences in loan default determinants before and during COVID-19 and to identify the most effective predictors of loan default during COVID-19, this study employs machine learning methods to establish a comprehensive loan default prediction model for Peer-to-peer (P2P) lending based on four perspectives: loan characteristics, credit transaction history, personal information, and macroeconomic environment. The results show that the EXtreme Gradient Boosting (XGBoost) outperforms the other models and that credit transaction history plays a vital role in forecasting loan default over the two periods. We also find discrepancies between the effects of consumer price index, purchasing manager’ index, and the number of bidders on loan default before and during the pandemic. Our study contributes to related research fields on loan default prediction by identifying loan default determinants that are more applicable to unstable periods and investigating the impact of COVID-19 on default predictions. Meanwhile, our findings can provide P2P lending investors, platforms, and policymakers with practical implications to reduce uncertainty and losses that result from similar black swan events.

Neyman-Pearson Multi-class Classification via Cost-sensitive Learning

Most existing classification methods aim to minimize the overall misclassification error rate. However, in applications such as loan default prediction, different types of errors can have varying consequences. To address this asymmetry issue, two popular paradigms have been developed: the Neyman-Pearson (NP) paradigm and the cost-sensitive (CS) paradigm. Previous studies on the NP paradigm have primarily focused on the binary case, while the multi-class NP problem poses a greater challenge due to its unknown feasibility. In this work, we tackle the multi-class NP problem by establishing a connection with the CS problem via strong duality and propose two algorithms. We extend the concept of NP oracle inequalities, crucial in binary classifications, to NP oracle properties in the multi-class context. Our algorithms satisfy these NP oracle properties under certain conditions. Furthermore, we develop practical algorithms to assess the feasibility and strong duality in multi-class NP problems, which can offer practitioners the landscape of a multi-class NP problem with various target error levels. Simulations and real data studies validate the effectiveness of our algorithms. To our knowledge, this is the first study to address the multi-class NP problem with theoretical guarantees. The proposed algorithms have been implemented in the R package npcs, which is available on CRAN.

Corporate bond default prediction using bilateral topic information of credit rating reports

The default of corporate bonds can result in large financial losses as well as irreparable harm to investors’ trust and the economy as a whole, which implies that the identification of corporate bond default must be done promptly and properly. Current studies mainly rely on accounting and/or macroeconomic data and use the credit rank (CR) to disclose the credit status of corporate bonds in the default prediction task. However, the textual data of credit rating reports (CRRs) contain richer and more comprehensive information and are neglected in related work. In this paper, we propose a novel framework that draws on the unstructured data in CRR to predict the default of corporate bonds. We extract the rating opinion sentences (categorized as positive and negative) from the collected CRR files and use latent Dirichlet allocation (LDA) models to mine topic information. The bilateral topic information of positive and negative opinions can reflect the anti-risk ability and potential risk of corporate bonds, respectively, based on which the constructed topic features are used for default prediction. Results on real-world Chinese corporate bonds dataset show that the bilateral topic information of CRR can significantly improve the predicting power of models (LR, SVM, KNN and MLP) under three performance metrics (AUC, KS and H-measure). By analyzing the ranking of topic features using SHAP value, the proposed framework can explain the factors that affect bond defaults, which can provide a basis for the decision-making of investment behavior.

Credit card default prediction using ML and DL techniques

The banking sector is widely acknowledged for its intrinsic unpredictability and susceptibility to risk. Bank loans have emerged as one of the most recent services offered over the past several decades. Banks typically serve as intermediaries for loans, investments, short-term loans, and other types of credit. The usage of credit cards is experiencing a steady increase, thereby leading to a rise in the default rate that banks encounter. Although there has been much research investigating the efficacy of conventional Machine Learning (ML) models, there has been relatively less emphasis on Deep Learning (DL) techniques. The application of DL approaches to credit card default prediction has not been extensively researched despite their considerable potential in numerous fields. Moreover, the current literature frequently lacks particular information regarding the DL structures, hyperparameters, and optimization techniques employed. To predict credit card default, this study evaluates the efficacy of a DL model and compares it to other ML models, such as Decision Tree (DT) and Adaboost. The objective of this research is to identify the specific DL parameters that contribute to the observed enhancements in the accuracy of credit card default prediction. This research makes use of the UCI ML repository to access the credit card defaulted customer dataset. Subsequently, various techniques are employed to preprocess the unprocessed data and visually present the outcomes through the use of exploratory data analysis (EDA). Furthermore, the algorithms are hypertuned to evaluate the enhancement in prediction. We used standard evaluation metrics to evaluate all the models. The evaluation indicates that the AdaBoost and DT exhibit the highest accuracy rate of 82 ​% in predicting credit card default, surpassing the accuracy of the ANN model, which is 78 ​%.

Loan Defaults Prediction Based on Stacked Models Trained by Personalized Features

Training one machine learning model with features that all clients have will result in a waste of features, which is likely to adversely affect the model’s performance. To solve the problem, the study attempts a new method, which it to train an individual stacked model for each loan client based on personalised features. Data used contains information of about fifteen million loan applicants, their default status, and 468 features in all. 41 of the features that can be quantitively analysed are selected according to the feature importance output by a Random Forest model. Default prediction of every client is made by a stacked model trained with all selected features he/she has. The stacked model consists of two layers, in which a Light Gradient-Boosting Machine (LGBM) classifier is the base learner, and a Logistic Regression model is the meta learner. As the defaulters account for only 3.14%, which is significantly unbalanced, Area Under the Curve (AUC) and F1 scores are employed to evaluate the method, instead of accuracy. Test results show that models trained by personalised features perform better than the ones trained by shared features. Additionally, the stacked model outperforms individual Logistic Regression model, but performs nearly the same as individual LGBM Classifier. In detailed, the stacked models trained with personalised features result in AUC=0.772 and F1=0.188. Due to data unbalance, although the method’s F1 score is relatively low, it’s considered to be passable. In the future, stacked models combining different models will be attempted.

Attention-based dynamic multilayer graph neural networks for loan default prediction

Whereas traditional credit scoring tends to employ only individual borrower- or loan-level predictors, it has been acknowledged for some time that connections between borrowers may result in default risk propagating over a network. In this paper, we present a model for credit risk assessment leveraging a dynamic multilayer network built from a Graph Neural Network and a Recurrent Neural Network, each layer reflecting a different source of network connection. We test our methodology in a behavioural credit scoring context using a dataset provided by U.S. mortgage financier Freddie Mac, in which different types of connections arise from the geographical location of the borrower and their choice of mortgage provider. The proposed model considers both types of connections and the evolution of these connections over time. We enhance the model by using a custom attention mechanism that weights the different time snapshots according to their importance. After testing multiple configurations, a model with GAT, LSTM, and the attention mechanism provides the best results. Empirical results demonstrate that, when it comes to predicting probability of default for the borrowers, our proposed model brings both better results and novel insights for the analysis of the importance of connections and timestamps, compared to traditional methods.

설명 가능한 인공지능을 활용한 은행대출연체 예측 모델 연구

[Purpose] This study aims to create a loan default prediction model using machine learning, a core concept of AI technology, and explainable AI (XAI) techniques, and to compare and evaluate it against highly interpretable models such as logistic regression. [Methodology] This study utilizes the data of household loan customers of Local Bank A in South Korea from December 2020 to June 2022 to generate and compare the logistic regression model and the XGBoost model. We used SHAP, one of the XAI techniques, to explain the XGBoost model. [Findings] The XGBoost model shows better prediction performance than the logistic regression model. By applying SHAP to the XGBoost model, we identify key predictors： credit rating, average bank deposit balance over the past six months, card loan risk rating, K rating agency’s multiple debt risk index, average core deposit balance over the past six months, and the current interest rate of the loan account. [Implications] The machine learning model (XGBoost) combined with XAI shows better prediction performance than the logistic regression model, and the key predictors identified by the two models are similar. Therefore, the combined model could serve as an alternative to the logistic regression model commonly used in business research.

Credit Card Client’s Payment Prediction for Next Month Using Machine Learning Algorithms

With the quick growth of the credit card system, there is a rising number of misconduct rates on credit card loans, which creates a financial risk for commercial banks. Thus, successful resolutions of the risks are significant for the sound advancement of the industry in the long term. Numerous financial banks and organizations become more and more attentive to the issue of credit card default because it brings about a high probability of financial risks. Credit risk plays a significant part in the financial business. One of the main functions of a bank is to issue loans, credit cards, investment mortgages, and other credit. One of the most popular financial services offered by banks in recent years has been the credit card. With its constant rise in risk factors, the banking industry is perhaps the most fragile and volatile in the world. Credit risk remains a crucial element for financial institutions that have experienced losses amounting to hundreds of millions of dollars as a result of their incapacity to retrieve the funds disbursed to clients. In the banking industry, it is now vital to forecast whether a borrower will be able to repay the loan. In this paper, we applied different machine learning classifiers, including Random Forest, K Nearest Neighbor, Logistic Regression, Decision Tree, Decision Tree with AdaBoosting, and Random Forest with AdaBoosting, to build a credit default prediction model. The results show that the AdaBoosting model achieved better accuracy than the other machine learning algorithms. Our proposed technique can support financial organizations in controlling, identifying, and monitoring credit risk, and it can identify credit card clients who pay the loan in the next month.

Firm Default Prediction by GNN with Gravity-Model Informed Neighbor Node Sampling

AbstractFirm default prediction is important in credit risk management and understanding economic trends. Both practitioners and academic researchers have long studied it. While traditional statistical methods such as discriminant analysis and logistic regression have been used recently, machine learning and deep learning methods have been widely applied. The graph neural network (GNN) is one of the latest applications of deep-learning approaches. With the use of GNNs, it is possible to reflect the non-linear relationships of features among neighboring companies around the target company, whereas ordinary machine learning and deep learning methods focus only on the features of the target company. However, when handling large-scale graphs such as inter-firm networks, it is difficult to apply vanilla GNNs naively. Although uniform neighbor node sampling is commonly used for large-scale graphs, to the best of our knowledge, no research has focused on better sampling methods for GNN applications for default prediction. From the practical viewpoint, it means which companies should be considered with priority for firm default prediction. In this study, we propose a novel gravity model-informed neighbor sampling method based on the estimated transaction volume by utilizing knowledge from econophysics. The scope of this research is to determine whether we can improve default predictions by considering neighboring companies with larger transaction amounts compared to ordinary uniform sampling. We also verified that the proposed method improves the prediction performance and stability compared to GNNs with other sampling techniques and other machine learning methods using real large-scale inter-firm network data.

Predicting Mortgage Loan Defaults Using Machine Learning Techniques

Mortgage default prediction is always on the table for financial institutions. Banks are interested in provision planning, while regulators monitor systemic risk, which this sector may possess. This research is focused on predicting defaults on a one-year horizon using data from the Ukrainian credit registry applying machine-learning methods. This research is useful for not only academia but also policymakers since it helps to assess the need for implementation of macroprudential instruments. We tested two data balancing techniques: weighting the original sample and synthetic minority oversampling technique and compared the results. It was found that random forest and extreme gradient-boosting decision trees are better classifiers regarding both accuracy and precision. These models provided an essential balance between actual default precision and minimizing false defaults. We also tested neural networks, linear discriminant analysis, support vector machines with linear kernels, and decision trees, but they showed similar results to logistic regression. The result suggested that real gross domestic product (GDP) growth and debt-service-to-income ratio (DSTI) were good predictors of default. This means that a realistic GDP forecast as well as a proper assessment of the borrower’s DSTI through the loan history can predict default on a one-year horizon. Adding other variables such as the borrower’s age and loan interest rate can also be beneficial. However, the residual maturity of mortgage loans does not contribute to default probability, which means that banks should treat both new borrowers equally and those who nearly repaid the loan.

Consensus credit ratings: a view from banks

While the production of credit ratings has long been limited mainly to rating agencies (CRAs), recent years have seen the growing popularity of consensus credit ratings crowdsourced from banks (i.e., bank ratings). We provide the first comprehensive examination of the properties and informativeness of bank ratings relative to CRA ratings. We find that bank ratings often deviate from CRA ratings, with over 60% of firm-months having different bank and CRA ratings. These deviations contain useful information. Bank ratings improve out-of-sample prediction of defaults and CRA rating revisions and explain the cross-section of credit spreads. However, bank ratings do not improve out-of-sample prediction of credit excess returns, indicating that current prices incorporate bank rating information. Overall our findings suggest that bank ratings are a useful supplement to traditional credit ratings.

Refining Credit Risk Analysis- Integrating Bayesian MCMC with Hamiltonian Monte Carlo

The accurate prediction of loan defaults is paramount for financial institutions to enhance decision-making processes, optimize loan approval rates, and mitigate associated risks. This study develops a predictive model utilizing Bayesian Markov Chain Monte Carlo (MCMC) techniques to forecast potential loan defaults. Employing a comprehensive dataset of 255,000 borrower profiles, which include detailed borrower characteristics and loan information, the model integrates advanced statistical methods to assess and interpret the factors influencing loan defaults. The Bayesian framework allows for robust uncertainty quantification and model complexity management, making it particularly suitable for the nuanced nature of credit risk assessment. Results from the model demonstrate a compelling accuracy rate, substantially aligning with industry benchmarks while providing deeper insights into the probability of default as influenced by various borrower attributes. This research underscores the efficacy of Bayesian MCMC modelling in financial risk management and offers a scalable approach for financial institutions aiming to refine their credit evaluation strategies.

Research on User Default Prediction Algorithm Based on Adjusted Homogenous and Heterogeneous Ensemble Learning

In the field of risk assessment, the traditional econometric models are generally used to assess credit risk. And with the introduction of the “dual-carbon” goals to promote the development of a low-carbon economy, the scale of green credit in China has rapidly expanded. But with the advent of the big data era, due to the poor interpretability of a traditional single machine learning model, it is difficult to capture nonlinear relationships, and there are shortcomings in prediction accuracy and robustness. This paper selects the adjusted ensemble learning model based on the homogeneous and heterogeneous factors for user default prediction, which can efficiently process large quantities of high-dimensional data. This article adjusts each model to adapt to the task and innovatively compares various models. In this paper, the missing value filling method, feature selection, and ensemble model are studied and discussed, and the optimal ensemble model is obtained. When comparing the predictions of single models and ensemble models, the accuracy, sensitivity, specificity, F1-Score, Kappa, and MCC of Categorical Features Gradient Boosting (CatBoost) and Random undersampling Boosting (RUSBoost) all reach 100%. The experimental results prove that the algorithm based on adjusted homogeneous and heterogeneous ensemble learning can predict the user default efficiently and accurately. This paper also provides some references for establishing a risk assessment index system.

Application of Machine Learning in Loan Default Prediction

Application of Machine Learning in Loan Default Prediction

A hierarchical attention-based feature selection and fusion method for credit risk assessment

A stable financial environment is requisite for the continuous growth of the E-business market, emphasizing the importance of credit risk assessment. Generally, credit risk assessment involves evaluating the creditworthiness of the economic entity and predicting the probability of defaulting on their financial obligations. Existing works mainly focus on exploring more features from various sources to help default prediction, but tend to overlook the feature cost and feature fusion problem. In this paper, we proposed a novel credit risk assessment model based on the hierarchical attention method, characterizing the ability to manage feature acquisition cost and adaptively integrate multi-view features. Particularly, to fulfill the utility of multi-source features, three segmenting standards are proposed to categorize multi-source features into different views according to their characteristics and statistical attributes. Then, a feature-level attention mechanism is proposed to estimate feature importance and guide feature selection, while a view-level attention mechanism is proposed to aggregate view representations and produce default predictions. The proposed method is capable of enhancing crucial features in a hierarchical way and undertaking feature selection to meet feature acquisition cost, thereby boosting the model performance with feature cost awareness. Experimental results show that the proposed method can approximate the best performance with 5% of selected features, and can also achieve a 1.1% AUC improvement compared with the best baseline method over the full feature set, demonstrating our effectiveness of enhancing default prediction performance while reducing feature costs.

Analysis of Credit Default Prediction Based on Logistic Regression, Random Forest and KNN Model

Analysis of Credit Default Prediction Based on Logistic Regression, Random Forest and KNN Model