Information maximization with Gaussian activation functions to generate explicit self-organizing maps

Abstract

We propose a new information theoretic method to produce explicit self-organizing maps. Competition is realized by maximizing mutual information between input patterns and competitive units. Competitive unit outputs are computed by the Gaussian function of distance between input patterns and competitive units. By the property of this Gaussian function, as distance becomes smaller, a neuron tends to fire strongly. Cooperation processes are realized by taking into account the firing rates of neighboring neurons. We applied our method to uniform distribution learning and road classification. Experimental results confirmed that cooperation processes could significantly increase information content in input patterns. When cooperative operations are not effective in increasing information, mutual information as well as entropy maximization is used to increase information. Experimental results especially showed that entropy maximization could be used to increase information and to give clearer self-organizing maps, because competitive units are forced to use equally on average.