The learning of moment neuronal networks

Abstract

We consider how to train the recently developed moment neuronal networks (MNN) with the IF model activated by the renewal process synaptic inputs, which encodes the input information into the mean and variance of the interspike intervals. Here we apply an error minimization technique to train the networks and mathematically derive a corresponding back-propagation learning rule based on the error regression, which seeks to minimize the error combination of the output means and variances of networks. As a result, we present a more biologically plausible so-called second order spiking perceptron (SOSP). We show through various examples that such system, even a single neuron, is able to perform various complex non-linear tasks like the XOR problem which is impossible to be solved by traditional single-layer perceptrons. Moreover such perceptron can be trained to implement a practical and classical dynamic learning task: simulating the trajectory of an arm movement. Among including the second order statistics in computations, such perceptron offers a significant advantage over their predecessors, in that it involves the learning accuracy of output variance by directly introducing the variance term in the error presentation. Thus we can train not only the output bias (mean error) but also the output noises (variances), and also can reach the trade-off between the output bias and output variance by the adjustment of the penalty factor in error function, due to a specific learning task. Indeed, this is the most important advantage over other kinds of spiking neural networks in existent.