Convolutional neural networks with multi-valued neurons

Abstract

Convolutional neural network (CNN) has been successfully used in many fields including image recognition. CNN is composed of input, convolution, pooling, hidden and output layers, and the weights and biases between layers except the ones between convolution and pooling layers are acquired by learning. In comparison to the conventional neural networks, the learning cost of CNN is higher, and the learning time is longer especially when hidden layer(s) are added. Recently, complex- and quaternion-valued neural networks have drawn much attention. In complex-valued neural networks, inputs, weights, biases and outputs are complex numbers, and in quaternion-valued neural networks, these parameters are quaternions. It has been shown that both methods exhibit excellent accuracy in various applications such as classification and function approximation problems with less computational burden. In this study, we propose CNNs with complex- and quaternion-valued neurons where complex and quaternion numbers are used between pooling, hidden and output layers. We here show that CNNs with complex- and quaternion-valued neurons have higher learning ability in handwritten digit image classification with the MNIST data than the real-valued counterpart.