Robustness of artificial metaplasticity learning algorithm

Abstract

Artificial metaplasticity learning algorithm is inspired by the biological metaplasticity property of neurons and Shannon׳s information theory. It is based on the bio-inspired hypothesis that neurons do not learn in the same amount (metaplasticity of biological learning) from unfrequent patterns than from common ones, as the former are expected to contain more information than the latter (information entropy concept). On MLPs, the artificial metaplasticity can be formulated as an improvement in regular backpropagation algorithm by using a variable learning rate affecting all the weights in each iteration step and so resembling heterosynaptic plasticity of biological neurons. The variable rate involves statistical inference on the training set and it is common to successfully assume Gaussian distribution for the training patterns. Nevertheless, Gaussian assumption may diverge from the real one and using statistical information extracted from the training patterns may be necessary. In this research, robustness to significative variations on Gaussian assumption is evaluated using input sets generated with different probability distributions. For the cases where Gaussian assumption shows to degrade learning, a general algorithm is applied. This algorithm takes advantage of the inherent statistical inference performed by the MLP through the a posteriori probabilities of input patterns estimation provided by its outputs. The generality of this last algorithm for any input distribution is then demonstrated.