Constructing multilayered neural networks with sparse, data-driven connectivity using biologically-inspired, complementary, homeostatic mechanisms

Abstract

The immense complexity of the brain requires that it be built and controlled by intrinsic, self-regulating mechanisms. One such mechanism, the formation of new connections via synaptogenesis, plays a central role in neuronal connectivity and, ultimately, performance. Adaptive synaptogenesis networks combine synaptogenesis, associative synaptic modification, and synaptic shedding to construct sparse networks. Here, inspired by neuroscientific observations, novel aspects of brain development are incorporated into adaptive synaptogenesis. The extensions include: (i) multiple layers, (ii) neuron survival and death based on information transmission, and (iii) bigrade growth factor signaling to control the onset of synaptogenesis in succeeding layers and to control neuron survival and death in preceding layers. Also guiding this research is the assumption that brains must achieve a compromise between good performance and low energy expenditures. Simulations of the network model demonstrate the parametric and functional control of both performance and energy expenditures, where performance is measured in terms of information loss and classification errors, and energy expenditures are assumed to be a monotonically increasing function of the number of neurons. Major insights from this study include (a) the key role a neural layer between two other layers has in controlling synaptogenesis and neuron elimination, (b) the performance and energy-savings benefits of delaying the onset of synaptogenesis in a succeeding layer, and (c) how the elimination of neurons in a preceding layer provides energy savings, code compression, and can be accomplished without significantly degrading information transfer or classification performance.