Neural network with multiple connection weights

Abstract

Biological studies have shown that the interaction between neurons are based on neurotransmitters, which transmit signals between neurons, and that one neuron sends information to another neuron by releasing a number of different neurotransmitters, which play different roles. Motivated by this biological discovery, a novel neural networks model is proposed by extending the dimension of connection weights from one to multiple, i.e. there are multiple not only one connections between each two units. The number of dimensions of connection weight represents the number of categories of neurotransmitters and different components of the weight correspond to different neurotransmitters. In order to make these neurotransmitters collaborate and compete appropriately, the input and output for each unit in our proposed model have been heuristically defined. From the biological perspective, the proposed neural network is much closer to biological neural network. From the viewpoint of new model structure, the characteristic that the activation of each hidden unit is based on several filters, can improve the interpretability of features learned by the proposed neural network. Experimental results on MNIST, NORB and several other data sets have demonstrated that the performances of traditional neural networks can be improved by extending the dimension of connection weight between units, and the idea of multiple connection weights provides a new paradigm for the design of neural networks.