Convolutional Neural Network with Multi-Valued Neurons

Abstract

In this paper, a convolutional neural network (CNN) based on multi-valued neurons (MVNs) with complex-valued weights is presented. Convolutional neural networks are known as one of the best tools for solving such problems as image and speech recognition. The vast majority of researches and users so far employ a classical CNN, which operates with real-valued inputs/outputs and is based on classical neurons with a sigmoidal activation function. Recently complex-valued CNNs were introduced, but their neurons employ an activation function, which is a complex-valued generalization of a sigmoidal one. While such complex-valued neurons are more flexible when they learn and make it possible to process complex-valued data, their generalization capability is basically not higher than the one of real-valued neurons. At the same time there exists a complex-valued neuron (a multi-valued neuron – MVN) with a phase dependent activation function whose functionality is higher than the one of neurons with a sigmoidal activation function. It is also known that a multilayer neural network with multi-valued neurons (MLMVN) outperforms a classical multilayer peceptron in terms of speed of learning and generalization capability. Hence our goal was to develop a convolutional neural network based on multi-valued neurons (CNNMVN).We consider in detail a learning algorithm for CNNMVN, which is derivative free as well as the one for MLMVN. We also suggested the max pool operation for complex numbers. These results are illustrated by simulations showing that CNNMVN with even a minimal convolutional topology is able to solve image recognition problems with pretty high accuracy.