Rectified Encoder Network for High-Dimensional Imbalanced Learning

Abstract

Many existing works have studied the learning on imbalanced data, however, it is still very challenging to handle high-dimensional imbalanced data. One key challenge of learning on imbalanced data is that most learning models usually have a bias towards the majority and its performance will deteriorate in the presence of underrepresented data and severe class distribution skews. One solution is to synthesize the minority data to balance the class distribution, but it may lead to more overlapping, especially in the high-dimensional setting. To alleviate the above challenges, in this paper, we present a novel Rectified Encoder Network (REN) for high-dimensional imbalanced learning tasks. The main contribution is that: (1) To deal with high-dimensionality, REN encodes high-dimensional imbalanced data into low dimensional latent codes as a latent representation. (2) To obtain a discriminative representation, we introduce a Rectifier to match the latent codes with our proposed Predefined Codes, which disentangles the overlapping among classes. (3) During rectification, in the Predefined Latent Distribution, we can efficiently identify and generate informative samples to maintain the balance of class distribution, so that the minority classes will not be neglected. The experimental results on several high-dimensional and image imbalanced data sets indicate that our REN obtains good representation code for classification and visualize the reason why REN gets better performance in high-dimensional imbalanced learning.