Discriminative Regularized Input Manifold for multilayer perceptron

Abstract

Multilayer perceptron (MLP) fails to discriminate the ambiguous inputs belonging to the overlapping regions of multiple classes, resulting in misclassification. To classify the input samples accurately according to their classes, removing the ambiguity that occurred due to the sharing of common input space is important. In this article, a novel neural network framework, called Discriminative Regularized Input Manifold MLP (DRIM-MLP) is proposed to reduce this ambiguity and improve the classification accuracy. The proposed framework consists of two different feed forward networks that are trained simultaneously: (i) DRIM and (ii) MLP. The proposed DRIM learns the input distribution during its training and the learnt information is incorporated during the training of MLP to reduce the ambiguity. Simultaneously, the class information learnt by MLP is incorporated into the DRIM to learn discriminative information of the input distribution. The hidden layer output of DRIM estimates (i) the input of DRIM using the weights of hidden and output layers of DRIM and (ii) the class output of MLP using the weights of hidden and output layers of MLP. Here, the estimated class output, learnt by the DRIM that contains information of the input distribution, is used as a regularizer for the MLP to minimize the difference between the estimated class output of DRIM and the estimated class output of MLP itself. The hidden layer of MLP also estimates: (i) the class output of MLP using the weights between hidden and output layers of MLP and (ii) the input of DRIM using the weights between hidden and output layers of DRIM. Here, the estimated input learnt by MLP that contains class information is used as a regularizer for DRIM to minimize the difference between the estimated input of DRIM itself and the estimated input of MLP. These two regularizers are respectively called the Input manifold Regularizer (ImR) and the Discriminative Regularizer (DiscR). The experimental results based on ten standard data sets strongly support the effectiveness of the proposed DRIM-MLP compared to conventional MLP, auto encoder based MLP (AE-MLP), denoising auto encoder based MLP (DAE-MLP), AE-MLP with KLD, DAE-MLP with KLD along with two recent works of state-of-the-art.