Generalized Correntropy based deep learning in presence of non-Gaussian noises

Abstract

Abstract Deep learning algorithms are the hottest topics in machine learning area lately. Although deep learning has made great progress in many domains, the robustness of learning systems with deep architectures is still rarely studied and needs further investigation. For instance, the impulsive noises (or outliers) are pervasive in real world data and can badly influence the mean square error (MSE) cost function based deep learning algorithms. Correntropy based loss function, which uses Gaussian kernel, is widely utilized to reject the above noises, however, the effect is not satisfactory. Therefore, generalized Correntropy (GC) is put forward to further improve the robustness, which uses generalized Gaussian density (GGD) function as kernel. GC can achieve extra flexibility through the GC parameters, which control the behavior of the induced metric, and shows a markedly better robustness than Correntropy. Motivated by the enhanced robustness of GC, we propose a new robust algorithm named generalized Correntropy based stacked autoencoder (GC-SAE), which is developed by combining the GC and stacked autoencoder (SAE). The new algorithms can extract useful features from the data corrupted by impulsive noises (or outliers) in a more effective way. The good robustness of the proposed method is confirmed by the experimental results on MNIST benchmark dataset. Furthermore, we show how our model can be applied for robust network classification, based on Moore network data of 377,526 samples with 12 classes.