Double Competition for Information-Theoretic SOM

Ryotaro Kamimura

doi:10.14569/ijarai.2014.031104

Abstract

In this paper, we propose a new type of informationtheoretic method for the self-organizing maps (SOM), taking into account competition between competitive (output) neurons as well as input neurons. The method is called ”double competition”, as it considers competition between outputs as well as input neurons. By increasing information in input neurons, we expect to obtain more detailed information on input patterns through the information-theoretic method. We applied the informationtheoretic methods to two well-known data sets from the machine learning database, namely, the glass and dermatology data sets. We found that the information-theoretic method with double competition explicitly separated the different classes. On the other hand, without considering input neurons, class boundaries could not be explicitly identified. In addition, without considering input neurons, quantization and topographic errors were inversely related. This means that when the quantization errors decreased, topographic errors inversely increased. However, with double competition, this inverse relation between quantization and topographic errors was neutralized. Experimental results show that by incorporating information in input neurons, class structure could be clearly identified without degrading the map quality to severely.

Highlights

We have introduced an information-theoretic method considering information in input neurons to realize competitive learning as well as the self-organizing maps
When mutual information is maximized between neurons and input patterns, just one neuron wins the competition
Mutual information maximization corresponds to competitive learning

Summary

Introduction

The present paper aims to show that the concept of competition among components in neural networks should be extended to all components of neural networks. Many methods have been developed to realize competition in neural networks. Output neurons compete with each other to represent input patterns. If a neuron wins the competition, it tries to represent input patterns as efficiently as possible. We have mentioned that competition can be realized in any component of neural networks. In addition to output neurons, we can consider input neurons in competitive neural networks. We can imagine a case where output as well as input neurons compete with other to represent input patterns. The goal of the present paper is to show that the extension of competition into input neurons can improve the performance of neural networks

Objectives

Results

Conclusion