Neural networks based on vectorized neurons

Abstract

As the main research content of artificial intelligence, the artificial neural network has been widely concerned because of its excellent performance in the fields such as computer vision and natural language processing since it was proposed in the 1940s. The neuron model of the traditional neural network was proposed by McCulloch and Pitts in 1943 (MP neurons), But MP neurons is too simple to representing biological neurons. Based on this, this paper studies the attention mechanism and proposes vectorized neuron and its activation function. Firstly, we propose vectorized neurons, then use the attention mechanism to dynamically generate connection weights between vectorized neurons. Nextly, we construct a new type of neural network with vectorized neurons, which we called neural functional group (NFG). Finally, we tested the proposed neural functional group model on two tasks: image classifcation and few-shot learning. The vectorized neuron can be conditionally activated through its activation function. Besides, the vectorized neuron has the potential of representing complex biological neurons, which is difficult for MP neuron. The experimental results show that it can achieve higher accuracy with fewer parameters than convolutional neural networks (CNN) and capsule networks in image classication task; it also competitive to CNN based feature extractor in few-shot learning task.