Elastic SDAE: An Adaptive Noise Selection for Stacked Denoising Auto encoder

Abstract

This paper proposes an Elastic Stacked Denoising Autoencoder model, an upgraded model of a stacked autoencoder algorithm. It works on the basis of reconstruction through learning from the input towards generating the same as the output. But, the selection of noise levels in the autoencoder is determined as fixed-parameter throughout the learning. The proposed model addresses this limitation based on the principle of annealing (ElasticSDAE), a novel method of adaptively obtaining the noise level. This is achieved by first computing the average noise level for each epoch using a linear average noise level function based on the principle of annealing; and second calculating the noise level for each input neuron based on the average noise level, and the contribution of the input neuron to the activation of hidden neurons (which depend on the input neuron’s value and the weights). Thus, the network includes a combination of features at multiple scales. The experimental results show that our proposed ElasticSDAE performed better than SDAE and other unsupervised feature learning methods.