Hybrid Pre-Training Strategy for Deep Denoising Neural Networks and Its Application in Machine Fault Diagnosis

Abstract

Removing or at least attenuating the effect of noise is of significance for machine fault diagnosis under noisy measuring environments. In this paper, deep denoising neural networks are developed to decrease the influence of noise on the modeling accuracy of deep neural networks (DNN). Among the network, a reproducing kernel Hilbert space (RKHS) based denoising layer is designed as the first layer to reduce noise. However, due to the unstable gradients problem of the earlier layers in DNN, weights to-be-optimized in the first denoising layer are tough to update. In addition, the overfitting problem caused by multiple layers also affects the generalization ability of the model. To overcome these dilemmas, a novel hybrid pre-training strategy for deep denoising neural networks is proposed. In this strategy, motivated by the greedy algorithm, a shallow supervised network is firstly adopted to pre-train the weight of the denoising layer for alleviating the learning difficulty of the first layers in deep networks. Secondly, feature extraction layers with a large number of layers are pre-trained by an autoencoder-based unsupervised network. After that, weights of the denoising and feature extraction layers are transferred to a deep classification network, and the entire network is finally fine-tuned via supervised learning. The hybrid pre-training strategy for deep denoising neural networks is applied to machine fault diagnosis. Experiment studies verify the effectiveness of the proposed hybrid pre-training strategy in intelligent fault diagnosis of machinery under noisy measuring environments.