AutoEIS: Automatic feature embedding, interaction and selection on default prediction

Abstract

Deep models have shown the effectiveness in various areas, e.g., finance, healthcare and recommendation system. Among them, default prediction is a major application in the financial field. However, there are still problems remained, like insufficient learning for numerical feature encoding and difficulty of explicitly modelling high-order feature interactions. To address these issues, we propose an automatic end-to-end deep learning framework named AutoEIS. In this framework, we embed categorical and numerical features with distinct strategies. In particular, we design a Multi-field-aware Mixture-of-Experts (MfMoE) structure for numerical value embedding, which can simultaneously learn the single-field and global-field information. Then, by organically integrating attention mechanism, weighted-average aggregation and bilinear interaction, we can effectively generate significant high-order explicit interactions. Moreover, we integrate a DNN block to further capture the complex relationships among different variables. Comprehensive experiments on two real-world datasets of about 30,000 samples show the superiority of AutoEIS on default prediction, boosting average AUC and KS metrics over the best classical baseline by 0.49% and 3.5%, and the best of deep baselines by 0.48% and 2.55%. Furthermore, as a model-agnostic strategy, we can generalize MfMoE to other deep models like DeepFM, thereby boosting their performance.