Enhancing and Relaxing Competitive Units for Feature Discovery

Abstract

In this paper, we propose a new information-theoretic method called enhancement and relaxation to discover main features in input patterns. We have so far shown that competitive learning is a process of mutual information maximization between input patterns and connection weights. However, because mutual information is an average over all input patterns and competitive units, it is not adequate for discovering detailed information on the roles of elements in a network. To extract information on the roles of elements in a networks, we enhance or relax competitive units through the elements. Mutual information should be changed by these processes. The change in information is called enhanced information. The enhanced information can be used to discover features in input patterns, because the information includes detailed information on elements in a network. We applied the method to the symmetry data, the well-known Iris problem and the OECD countries classification. In all cases, we succeeded in extracting the main features in input patterns.