Multi-view representation learning with Kolmogorov-Smirnov to predict default based on imbalanced and complex dataset

Abstract

Existing solutions focus on improving overall accuracy for imbalanced and complex loan datasets, resulting in a lower precise recall for default samples. To embrace these challenges, based on peer-to-peer loan application information, we proposed a multi-view representation learning with Kolmogorov-Smirnov (KS) to effectively organize these complex data and predict default. Firstly, the features were automatically represented as multi-views based on their discreteness and correlation difference. Then, a corresponding multi-view deep neural network (MV-DNN) was developed to obtain knowledge in a multi-view way. Here, we firstly designed different view learning layers to obtain knowledge in corresponding views. Subsequently, to interact with the knowledge in different views, an information fusion layer was developed to fuse the acquired information. To face the challenge from imbalanced data distribution, the KS was set as evaluation metric to assist in training MV-DNN to improve the distinguishing ability for two classes of samples. The experimental results show compared with the MV-DNNs based on random and k-means multi-view strategies, and other advanced models, our method could provide optimal comprehensive performance and the most stable multi-view organizing results. Furthermore, we also verified the KS is the key component to assist the model in dealing with the imbalanced dataset.