An optical neural chip for implementing complex-valued neural network

The optical implementation of neural networks is proposed to have advantages over electronic implementations with lower power consumption and higher computation speed. However, most optical neural networks (ONNs) utilize conventional real-valued frameworks that are designed for digital computers, forfeiting many advantages of optical computing such as efficient complex-valued operations. Complex-valued neural networks are advantageous to their real-valued counterparts by offering rich representation space, fast convergence, and strong generalizations. We propose and demonstrate an ONN that implements truly complex-valued neural networks, achieving high accuracy and strong learning capability in many benchmark tasks [1]. On the other hand, efficiently training ONNs remains a formidable challenge, due to the difficulty in obtaining gradient information from a physical device. We propose an efficient on-chip training protocol for ONNs and demonstrate it by several practical tasks [2]. The protocol is gradient-free and physical agnostic, and is applicable for various types of chip structures, especially those that cannot be analytically decomposed and characterized. The protocol is robust to experimental perturbations like imperfect phase detection and photodetection noise. Our results present a promising avenue towards deep complex networks with smaller chip size, stronger performance, and flexible reconfiguration to realistic applications (e.g., facial recognition, natural language processing, and autonomous vehicles).

Complex numbers are used to represent data in many practical applications such as in telecommunications, image processing, and speech recognition. In this work, we examine the efficiency of complex-valued neural networks and compare that with their real-valued counterpart. Specifically, we examine the performance of neural network with Multi Layer Multi-Valued Neuron (MLMVN) for classification on several benchmark datasets such as Iris and MNIST datasets. It is shown that in applications where complex numbers occur naturally, complex-valued neural networks such as MLMVN network could offer advantages such as more efficient embedding and processing of information over their real-valued counterparts. It is also observed that complex-valued neural networks have a tendency of overfitting especially in applications involving large datasets. Potential solution to the overfitting problem has been discussed.

The optical implementation of neural networks is proposed to have advantages over electronic implementations with lower power consumption and higher computation speed. However, most optical neural networks (ONNs) utilize conventional real-valued frameworks that are designed for digital computers, forfeiting many advantages of optical computing such as efficient complex-valued operations. Complex-valued neural networks are advantageous to their real-valued counterparts by offering rich representation space, fast convergence, and strong generalizations. We propose and demonstrate an ONN that implements truly complex-valued neural networks, achieving high accuracy and strong learning capability in many benchmark tasks.1 On the other hand, efficiently training ONNs remains a formidable challenge, due to the difficulty in obtaining gradient information from a physical device. We propose an efficient on-chip training protocol for ONNs and demonstrate it by several practical tasks.2 The protocol is gradient-free and physical agnostic, and is applicable for various types of chip structures, especially those that cannot be analytically decomposed and characterized. The protocol is robust to experimental perturbations like imperfect phase detection and photodetection noise. Our results present a promising avenue towards deep complex networks with smaller chip size, stronger performance, and flexible reconfiguration to realistic applications (e.g., facial recognition, natural language processing, and autonomous vehicles).

Automatic gender recognition is an emerging problem in computer visions. An accurate gender recognition system can be used to reduce the search space in face recognition system for about half. However, since there is no definitive features of sexual dimorphism on human face that can be applied to all kind of face shapes from any race and age, it needs more studies to optimize the recognition system. Thus, this research investigates the implementation of complex-valued neural network as a classifier to recognize human gender which is based on face image. The experiment is also aimed to study the comparison between complex-valued and real-valued neural network. The methods proposed in this paper include image processing, feature extraction, and classification. After the face image processed by using local binary pattern to accentuate face texture and gradient filter to define face outline, the features of the face image is extracted by using histogram of oriented gradient. Then, the dimension of the resulted vectors is reduced by using principal component analysis. The final feature vectors is then used in neural network training and neural network testing processes. This paper shows investigation results in implementing the methods for some measurement parameters. The accuracy level of real-valued neural network system is a bit lower than the average accuracy level of complex-valued one, with 78.2% and 80.2% average accuracy rate, respectively. The results also show that complex-valued neural network system can achieve convergency rate four times faster than the real-valued neural network, which implies on the training process time.

Since the last decade, several complex-valued neural networks have been developed and applied in various research areas. As an extension of real-valued recurrent neural networks, complex-valued recurrent neural networks use complex-valued states, connection weights, or activation functions with much more complicated properties than real-valued ones. This paper presents several sufficient conditions derived to ascertain the existence of unique equilibrium, global asymptotic stability, and global exponential stability of delayed complex-valued recurrent neural networks with two classes of complex-valued activation functions. Simulation results of three numerical examples are also delineated to substantiate the effectiveness of the theoretical results.

This paper discusses the track control problem of complex-valued neural networks (CVNNs) with time delay. By defining the suitable norm, we directly investigate globally asymptotically stable of CVNNs. With the Hanaly’s inequality and analysis techniques, we derive a sufficient condition which can make the state of CVNNs globally exponentially track the given reference trace. At last, one illustrative example verifies our result.

Complex-valued convolutional neural networks (CVCNN) have better performance than real-valued neural networks in the field of terahertz imaging. In this paper, a complex-valued neural network is innovatively applied to terahertz image classification task in a vector network analyzer (VNA) imaging system. The complex-valued CNN (CVCNN) processing framework for terahertz image classification is proposed. Terahertz image datasets are constructed using MINIST handwritten datasets and PSF which was measured from our transmission system. Compared to CNN, CVCNN has a better accuracy rate, and it is significantly less vulnerable to over-fitting. Phase information can be used well at the same time, which is impossible for the CNN. The method of training data generation is given, and some specific implementation details are given. the superiority of the method in this paper is verified by using simulated and measured data obtained from 200Ghz image system.

In this paper, we develop a new approach to image recognition. This approach is based on the analysis of frequency domain features (namely Fourier transform phases corresponding to certain frequencies) using the multilayer neural network with multi-valued neurons (MLMVN). MLMVN is a powerful complex-valued feedforward neural network, which has shown its high efficiency in solving various classification, prediction, and intelligent image filtering problems. As a complex-valued neural network, MLMVN has an ability to treat the phase information properly, completely preserving a circular nature of phase. At the same time it is known that phases contain all information about image edges, their location and spatial orientation. This means that the phase information can be used for image recognition. Particularly, this is the case when it is necessary to recognize objects whose size is known and fixed and it is possible to use phases corresponding to certain frequencies found based on the Nyquist-Shannon theorem as features for recognition. These phases can then be analyzed using MLMVN. We illustrate this approach using the famous MNIST image dataset, which for a 100% recognition rate was achieved. Phases corresponding only to the three lowest frequencies (1 to 3) and just a single hidden layer MLMVN are enough to achieve this result. A batch learning algorithm (MLMVN-SM-LLS) is employed to train the network.

In this paper a method related to extracting white blood cells (WBCs) from blood microscopic images and recognizing them and counting each kind of WBCs is presented. In medical science diagnosis by check the number of WBCs and compared with normal number of them is a new challenge and in this context has been discussed it. After reviewing the methods of extracting WBCs from hematology images, because of high applicability of artificial neural networks (ANNs) in classification we decided to use this effective method to classify WBCs, and because of high speed and stable convergence of complex-valued neural networks (CVNNs) compare to the real one, we used them to classification purpose. In the method that will be introduced, first the white blood cells are extracted by RGB color system's help. In continuance, by using the features of each kind of globules and their color scheme, a normalized feature vector is extracted, and for classifying, it is sent to a complex-valued back-propagation neural network. And at last, the results are sent to the output in the shape of the quantity of each of white blood cells. Despite the low quality of the used images, our method has high accuracy in extracting and recognizing WBCs by CVNNs, and because of this, certainly its result on high quality images will be acceptable. Learning time of complex-valued neural networks, that are used here, was significantly less than real-valued neural networks.

This paper presents some theoretical results on dynamical behavior of complex-valued neural networks with discontinuous neuron activations. Firstly, we introduce the Filippov differential inclusions to complex-valued differential equations with discontinuous right-hand side and give the definition of Filippov solution for discontinuous complex-valued neural networks. Secondly, by separating complex-valued neural networks into real and imaginary part, we study the existence of equilibria of the neural networks according to Leray---Schauder alternative theorem of set-valued maps. Thirdly, by constructing appropriate Lyapunov function, we derive the sufficient condition to ensure global asymptotic stability of the equilibria and convergence in finite time. Numerical examples are given to show the effectiveness and merits of the obtained results.

Complex-valued neural networks, which are extensions of ordinary neural networks, have been studied as interesting models by many researchers. Especially, complex-valued Hopfield neural networks (CHNNs) have been used to process multilevel data, such as gray-scale images. CHNNs with Hermitian connection weights always converge using asynchronous update. The noise tolerance of CHNNs deteriorates extremely as the resolution increases. Noise tolerance is one of the most controversial problems for CHNNs. It is known that rotational invariance reduces noise tolerance. In this brief, we propose symmetric CHNNs (SCHNNs), which have symmetric connection weights. We define their energy function and prove that the SCHNNs always converge. In addition, we show that the SCHNNs improve noise tolerance through computer simulations and explain this improvement from the standpoint of rotational invariance.

Quasi-projective synchronization of fractional-order complex-valued recurrent neural networks

Applications of complex-valued neural networks (CVNNs) have expanded widely in recent years-in particular in radar and coherent imaging systems. In general, the most important merit of neural networks lies in their generalization ability. This paper compares the generalization characteristics of complex-valued and real-valued feedforward neural networks in terms of the coherence of the signals to be dealt with. We assume a task of function approximation such as interpolation of temporal signals. Simulation and real-world experiments demonstrate that CVNNs with amplitude-phase-type activation function show smaller generalization error than real-valued networks, such as bivariate and dual-univariate real-valued neural networks. Based on the results, we discuss how the generalization characteristics are influenced by the coherence of the signals depending on the degree of freedom in the learning and on the circularity in neural dynamics.

Currently, dealing directly with in-phase and quadrature time series data using the deep learning method is widely used in signal modulation classification. However, there is a relative lack of methods that consider the complex properties of signals. Therefore, to make full use of the inherent relationship between in-phase and quadrature time series data, a complex-valued hybrid neural network (CV-PET-CSGDNN) based on the existing PET-CGDNN network is proposed in this paper, which consists of phase parameter estimation, parameter transformation, and complex-valued signal feature extraction layers. The complex-valued signal feature extraction layers are composed of complex-valued convolutional neural networks (CNN), complex-valued gate recurrent units (GRU), squeeze-and-excite (SE) blocks, and complex-valued dense neural networks (DNN). The proposed network can improve the extraction of the intrinsic relationship between in-phase and quadrature time series data with low capacity and then improve the accuracy of modulation classification. Experiments are carried out on RML2016.10a and RML2018.01a. The results show that, compared with ResNet, CLDNN, MCLDNN, PET-CGDNN, and CV-ResNet models, our proposed complex-valued neural network (CVNN) achieves the highest average accuracy of 61.50% and 62.92% for automatic modulation classification, respectively. In addition, the proposed CV-PET-CSGDNN has a significant improvement in the misjudgment situation between 64QAM, 128QAM, and 256QAM compared with PET-CGDNN on RML2018.01a.

Convolutional neural network (CNN) has been successfully used in many fields including image recognition. CNN is composed of input, convolution, pooling, hidden and output layers, and the weights and biases between layers except the ones between convolution and pooling layers are acquired by learning. In comparison to the conventional neural networks, the learning cost of CNN is higher, and the learning time is longer especially when hidden layer(s) are added. Recently, complex- and quaternion-valued neural networks have drawn much attention. In complex-valued neural networks, inputs, weights, biases and outputs are complex numbers, and in quaternion-valued neural networks, these parameters are quaternions. It has been shown that both methods exhibit excellent accuracy in various applications such as classification and function approximation problems with less computational burden. In this study, we propose CNNs with complex- and quaternion-valued neurons where complex and quaternion numbers are used between pooling, hidden and output layers. We here show that CNNs with complex- and quaternion-valued neurons have higher learning ability in handwritten digit image classification with the MNIST data than the real-valued counterpart.