Variational Approach to Unsupervised Learning Algorithms of Neural Networks

Abstract

Variational approach in pattern recognition basing on a mean risk functional is applied to unsupervised learning algorithms for neural networks. The paper summarizes main statements and results of the approach: a structure of mean risk functionals relevant for the problem of grouping the multivariate data; optimum conditions; recurrent algorithms of stochastic approximation type for minimization of the functionals; and conditions ensuring convergence of the algorithms to appropriate solutions. Basing on this theory a general scheme is proposed and it allows to obtain neural networks whose adaptation laws arise as the recurrent unsupervised learning algorithms for suitably selected mean risk functionals. The well known Grossberg networks, counter-propagation networks, and Kohonen self-organizing networks are among them. Moreover the new variants of unsupervised-learning neural structures can be derived as it is demonstrated in the paper. The developed mathematical means can serve for theoretical justification of using such neural networks in data processing systems. © 1997 Elsevier Science Ltd. All Rights Reserved.