Autoeconder-Based Excessive Information Generation for Improving and Interpreting Multi-layered Neural Networks

Abstract

The present paper aims to propose a new type of learning method to increase information content in input patterns with multiple steps to be used in supervised learning. Unsupervised pre-training to train multi-layered neural networks turned out to be not so effective as has been expected, because connection weights obtained by the unsupervised learning tend to lose their original characteristics immediately in supervised training. To keep original information by unsupervised learning, we here try to increase information in input patterns as much as possible to overcome the vanishing information problem. In particular, for acquiring detailed information more appropriately, we gradually increases detailed information through multiple steps. We applied the method to the actual real data set of the eye-tracking, and two step information augmentation approach was taken. The results confirmed that generalization performance could be improved. In addition, we could interpret the importance of input variables more easily by treating all connection weights collectively.