Spontaneous scale-free structure of spike flow graphs in recurrent neural networks

Abstract

Listen

Translate article

In this paper we introduce a simple and mathematically tractable model of an asynchronous spiking neural network which to some extent generalizes the concept of a Boltzmann machine. In our model we let the units contain a certain (possibly unbounded) charge, which can be exchanged with other neurons under stochastic dynamics. The model admits a natural energy functional determined by weights assigned to neuronal connections such that positive weights between two units favor agreement of their states whereas negative weights favor disagreement. We analyze energy minima (ground states) of the presented model and the graph of charge transfers between the units in the course of the dynamics where each edge is labeled with the count of unit charges (spikes) it transmitted. We argue that for independent Gaussian weights in low enough temperature the large-scale behavior of the system admits an accurate description in terms of a winner-take-all type dynamics which can be used for showing that the resulting graph of charge transfers, referred to as the spike flow graph in the sequel, has scale-free properties with power law exponent gamma=2. Whereas the considered neural network model may be perceived to some extent simplistic, its asymptotic description in terms of a winner-take-all type dynamics and hence also the scale-free nature of the spike flow graph seem to be rather universal as suggested both by a theoretical argument and by numerical evidence for various neuronal models. As establishing the presence of scale-free self-organization for neural models, our results can also be regarded as one more justification for considering neural networks based on scale-free graph architectures.