A Hybrid Regularized Multilayer Perceptron for Input Noise Immunity

Abstract

The immunity of multilayer perceptron (MLP) is less effective towards input noise. In this article, we have focused on the robustness of MLP with respect to input noise where noise can be additive or multiplicative. Here, we have proposed a DropConnect based regularized MLP to reduce the coadaptation among the neurons of the hidden layer. At first, we have empirically and statistically shown that by reducing coadaptation among the hidden neurons, an MLP can achieve better noise immunity. We have also empirically shown that an MLP with input noise injection and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2}$</tex-math></inline-formula> regularizer is an effective approach to improve its noise immunity. However, the results indicate that it does not adjust the coadaptation among the hidden neurons. Therefore, for further improvement, we have proposed a hybrid regularized MLP (HRMLP), where DropConnect is combined with the noise injection and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2}$</tex-math></inline-formula> regularizer. In addition to input noise, we have also verified the robustness of HRMLP with respect to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {20\%}$</tex-math></inline-formula> outliers in the data set. To justify the effectiveness of the proposed HRMLP, we have compared it with MLP, MLP with noise injection, MLP with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2}$</tex-math></inline-formula> regularizer, MLP with noise injection and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2}$</tex-math></inline-formula> regularizer, and MLP with DropConnect along with two state-of-the-art works based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {20}$</tex-math></inline-formula> standard data sets. The experimental results for both noisy inputs and outliers confirm that the performance of HRMLP is significant compared to other methods.