Selective enhancement learning in competitive learning

Abstract

In this paper, we propose a new information-theoretic method to explicitly interpret final representations created by learning. The new method, called “selective enhancement learning,” aims at producing explicit representation with fewer input variables. The variable selection is performed by information enhancement in which with a specific and enhanced variable, mutual information, is measured. As this information grows larger, the importance of the variable increases. With selected and important variables, a network is retrained by free energy minimization. In this free energy minimization, we can obtain connection weights by considering the importance of specific variables. When we applied the method to the Senate problem, experimental results showed that clear representations could be obtained with a smaller number of variables. This tendency was more explicit when the network was large.