Complex Backpropagation Algorithm Research Articles

A novel self‐supervised ensemble learning framework for land use and land cover classification of polarimetric synthetic aperture radar images

AbstractClassification with a few samples of training set has been a longstanding issue in the field of polarimetric synthetic aperture radar (PolSAR) image analysis and processing. Aiming at the small number of training samples of the PolSAR image classification task, a novel Self‐Supervised Ensemble Learning Framework (SSELF) is designed. The designed SSELF can automatically extract PolSAR features conducive to PolSAR image classification with a small number of training samples. In addition, it can significantly decrease the dependence of neural network algorithms on large labelled samples of training set. First, utilise the spatial–polarimetric features of PolSAR data perfectly, the EfficientNet‐B0 is presented and utilised as the main section of the Deep Learning (DL) model to extract DL features of PolSAR data. Then, using an optimisation function that constrains the cross‐correlation matrix of various distortions of each sample to the identity matrix, the designed DL model can obtain the effective features of homogeneous samples gathering and heterogeneous samples separating from each other in a self‐supervised manner. Moreover, following the great success of curriculum learning in the area of machine learning, a novel deep curriculum‐learning model is proposed, entitled Deep Curriculum Learning (DCL), to train the DL network in our self‐supervised model. The proposed DCL utilises the entropy‐alpha target decomposition to estimate the degree of complexity of each PolSAR image patch before applying it to the EfficientNet‐B0. Also, an accumulative mini‐batch pacing function is used to introduce more difficult patches to EfficientNet‐B0 in the training process of the designed self‐supervised model. Furthermore, two EL models, feature‐level and view‐level ensemble, are proposed to increase the feature extraction capability and classification result of PolSAR data by jointly using spatial features at different scales and polarimetric information at different bands. In fact, in the proposed feature‐level ensemble strategy, to improve the classification result, the extracted features of the different scales are used as the input of the designed feature fusion algorithm. Therefore, the extracted features are mapped to the new space with lower dimension. In general, in the proposed view‐level ensemble strategy, first, the Mutual Information (MI) between each feature and the other features calculates, and then based‐on the calculated MI, different group of features are selected as various views. Next, the final classification result of the PolSAR image classification is obtained by using the designed majority vote of the result in each view. It should be noted that unlike existing traditional methods, both amplitude and phase information of SAR images to devolve the training process of the proposed model are used. In addition, all of the parameters of the proposed network are developed to a complex domain. In addition, a complex backpropagation algorithm by using the gradient‐based model is used for training the network. Finally, experimental results on three well‐known PolSAR data sets illustrate that the designed SSELF can extract more discriminant features using the designed DL model and can achieve better classification results than the other six novel DL models in terms of small training samples and adequate labelled samples.

A phenomenological model of whole brain dynamics using a network of neural oscillators with power-coupling

We present a general, trainable oscillatory neural network as a large-scale model of brain dynamics. The model has a cascade of two stages - an oscillatory stage and a complex-valued feedforward stage - for modelling the relationship between structural connectivity and functional connectivity from neuroimaging data under resting brain conditions. Earlier works of large-scale brain dynamics that used Hopf oscillators used linear coupling of oscillators. A distinctive feature of the proposed model employs a novel form of coupling known as power coupling. Oscillatory networks based on power coupling can accurately model arbitrary multi-dimensional signals. Training the lateral connections in the oscillator layer is done by a modified form of Hebbian learning, whereas a variation of the complex backpropagation algorithm does training in the second stage. The proposed model can not only model the empirical functional connectivity with remarkable accuracy (correlation coefficient between simulated and empirical functional connectivity- 0.99) but also identify default mode network regions. In addition, we also inspected how structural loss in the brain can cause significant aberration in simulated functional connectivity and functional connectivity dynamics; and how it can be restored with optimized model parameters by an in silico perturbational study.

A new model based on multi-aspect images and complex-valued neural network for synthetic aperture radar automatic target recognition

ABSTRACT During the last decades, Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has been vastly utilized in the military services and civil studies. Because of the sensibility of the SAR images of the imaging in the azimuth dimension, using the multi-aspect SAR image sequence is more practical than the single-aspect SAR image to achieve the superior classification accuracy in an actual SAR ATR task. At present time, multi-aspect SAR ATR models mostly utilize Recurrent Neural Networks (RNN) that depend on the sequence between samples and so suffer from the lack of information. In a practical work, a huge amount of training set is needed to train a deep learning network accurately, but it is so costly work to extract multi-aspect SAR images. So, in this article, a new model of multi-aspect SAR ATR is proposed by using a self-attention model. The proposed self-attention model is utilized to calculate the internal correlation between the original SAR images. At the same time, to develop the anti-noise capability of the proposed model and decrease the dependency on a huge amount of training data, a Convolutional Auto Encoder (CAE) is designed and utilized in the feature extraction section of the proposed model. On the other hand, unlike existing traditional methods, in this work we use both amplitude and phase information of SAR images to devolve the training process of the proposed model. It should be noted that all of the parameters of the proposed network are developed to a complex domain. In addition a complex backpropagation algorithm by using the gradient based model is used for training the network. In the end, experiments is obtained by using two MSTAR data set (the MSTAR-SOC and EOC). The experimental results prove that the proposed model not only can achieve a high recognitions rate on the case of sufficient training sample but also can obtain an acceptable rate in the case of small training samples in different complex situations. In addition, the simulation results demonstrate that by using the encoder of CAE in the proposed model, the whole configuration of the proposed model achieves the anti-noise capability that is a valuable benefit of any practical SAR ATR task.

Class factorized complex variational auto-encoder for HRR radar target recognition

In the field of radar automatic target recognition (RATR), the high-resolution range profile (HRRP) has received intensive attention. Bar a few exceptions, almost all HRRP-based ATR classification systems ignore the phase of the HRRP when the data is input to the classifier, relying instead only on the magnitude of the complex HRRP samples. This approach ignores the phase of the complex HRRPs, which reduces the information in the signal. In this paper, we develop a novel class factorized complex variational auto-encoder (CFCVAE) to utilize the phase of the high range resolution (HRR) radar echo for recognition. The CFCVAE is a complex-valued neural network (CV-NN) consisting of the encoding and decoding modules. In CFCVAE, the encoding module projects the observed data into the latent space, and then the latent features are fed to the decoding module, which further maps the latent features to data. In particular, the decoding module introduces the class labels to partition the whole observations into some parts, each of which is depicted by a specific class-decoder. Compared with the traditional variational auto-encoder (VAE) containing a single decoder, the CFCVAE can give a more accurate description to the whole dataset via multiple class-decoders, thus improving the characterization ability of features. In addition, based on the class labels, the CFCVAE employs the conditional prior on the latent variable to enhance the discrimination of features. Moreover, a complex backpropagation algorithm is derived for CFCVAE training, and a sample is classified to the class corresponding to the class-decoder with the minimum reconstruction error in the test stage. Experimental evaluations on the measured data indicate that the proposed method indeed achieves very promising target recognition performance.

Complex-Valued Convolutional Neural Network and Its Application in Polarimetric SAR Image Classification

Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy.

Convergence analysis of fully complex backpropagation algorithm based on Wirtinger calculus.

This paper considers the fully complex backpropagation algorithm (FCBPA) for training the fully complex-valued neural networks. We prove both the weak convergence and strong convergence of FCBPA under mild conditions. The decreasing monotonicity of the error functions during the training process is also obtained. The derivation and analysis of the algorithm are under the framework of Wirtinger calculus, which greatly reduces the description complexity. The theoretical results are substantiated by a simulation example.

J1630101 複素BPを用いた階層型ニューラルネットワークによる制御システムに関する一考察([J163]機械の知能化を学際的に考える)

In this paper, a complex multi-layer neural network that is conducted its training by using complex back propagation algorithm is applied to solve inverse kinematics a robot manipulator and to control an inverted pendulum. Computational experiments are carried out to investigate the characteristics of the complex back propagation algorithm. Experimental results of the inverse kinematics show that the performance of complex neural network is a little advantage than a real number neural network in local minimum problem. Experimental results of the inverted pendulum also show that the complex neural network learns faster than the real number neural network and achieves higher success rate in generalization problems.

A robust automatic phase-adjustment method for financial forecasting

In this work we present the robust automatic phase-adjustment (RAA) method to overcome the random walk dilemma for financial time series forecasting. It consists of a hybrid model composed of a qubit multilayer perceptron (QuMLP) with a quantum-inspired evolutionary algorithm (QIEA), which is able to evolve the complete QuMLP architecture and parameters, as well as searches for the best time lags to optimally describe financial phenomena. In the attempt to improve the QuMLP parameters supplied by QIEA, each individual of the QIEA population is further trained by the complex back-propagation (CBP) algorithm. Also, for each forecasting model generated, we use a phase fix procedure to adjust time phase distortions that appear in financial time series. Furthermore, an experimental analysis is conducted with the proposed method through six real world financial time series, and the obtained results are discussed and compared to results found with the best methods recently presented in the literature.

Complex-Valued Multi-Layer Perceptrons – An Application to Polarimetric SAR Data

Multi-Layer Perceptrons (MLPS) are powerful function approximators. In the last decades they were successfully applied to many different regression and classification problems. Their characteristics and convergence properties are well studied and relatively well understood, but they were originally designed to work with real-valued data. The main focus of this paper is the classification of polarimetric synthetic aperture radar (POLSAR) data which are a complex-valued signal. Instead of using an arbitrarily projection of this complex-valued data to the real domain, the paper proposes the usage of complex-valued MLPS (CV-MLPS), which are an extension of MLPS to the complex domain. The paper provides a detailed yet general derivation of the complex backpropagation algorithm and mentions related problems as well as possible solutions. Furthermore, it evaluates the performance of CV-MLPS in a land-cover classification task in POLSAR images under several learning conditions, and compares the proposed classifier with standard methods. The experimental results show that CV-MLPS are successfully applicable to classification tasks in POLSAR data. They show good convergence properties and a better performance if compared to real-valued MLPS.

DAMAGELESS DIGITAL WATERMARKING USING COMPLEXVALUED ARTIFICIAL NEURAL NETWORK

Several high-ranking watermarking schemes using neural networks have been proposed in order to make the watermark stronger to resist attacks. However, the current system only deals with real value data. Once the data become complex, the current algorithms are not capable of handling complex data. In this paper, a distortion-free digital watermarking scheme based on Complex-Valued Neural Network (CVNN) in transform domain is proposed. Fast Fourier Transform (FFT) was used to obtain the complex number (real and imaginary part) of the host image. The complex values form the input data of the Complex Back-Propagation (CBP) algorithm. Because neural networks perform best on detection, classification, learning and adaption, these features are employed to simulate the Safe Region (SR) to embed the watermark, thus, watermark are appropriately mapped to the mid frequency of selected coeffi cients. The algorithm was appraised by Mean Squared Error MSE and Average Difference Indicator (ADI). Implementation results have shown that this watermarking algorithm has a high level of robustness and accuracy in recovery of the watermark.

Complex-Valued Neural Network in Signal Processing: A Study on the Effectiveness of Complex Valued Generalized Mean Neuron Model

A complex valued neural network is a neural network which consists of complex valued input and/or weights and/or thresh- olds and/or activation functions. Complex-valued neural networks have been widening the scope of applications not only in electronics and informatics, but also in social systems. One of the most important applications of the complex valued neural network is in signal processing. In Neural networks, generalized mean neuron model (GMN) is often discussed and studied. The GMN includes a new aggregation function based on the concept of generalized mean of all the inputs to the neuron. This paper aims to present exhaustive results of using Generalized Mean Neuron model in a complex-valued neural network model that uses the back-propagation algorithm (called 'Complex-BP') for learning. Our experiments results demonstrate the effectiveness of a Generalized Mean Neuron Model in a complex plane for signal processing over a real valued neural network. We have studied and stated various observations like effect of learning rates, ranges of the initial weights randomly selected, error functions used and number of iterations for the convergence of error required on a Generalized Mean neural network model. Some inherent properties of this complex back propagation algorithm are also studied and discussed.

Wavelet network based predistortion method for wideband RF power amplifiers exhibiting memory effects

RF power amplifiers (PAs) are usually considered as memoryless devices in most existing predistortion techniques. Nevertheless, in wideband communication systems, PA memory effects can no longer be ignored and memoryless predistortion cannot linearize PAs effectively. After analyzing PA memory effects, a novel predistortion method based on wavelet networks (WNs) is proposed to linearize wideband RF power amplifiers. A complex wavelet network with tapped delay lines is applied to construct the predistorter and then a complex backpropagation algorithm is developed to train the predistorter parameters. The simulation results show that compared with the previously published feed-forward neural network predistortion method, the proposed method provides faster convergence rate and better performance in reducing out-of-band spectral regrowth.

Complex RPROP-algorithm for neural network equalization of GSM data bursts

Neural networks have been studied for channel equalization purposes with quite promising results. However, not a lot of published results are available for their performance in realistic mobile systems, such as Global System for Mobile communications (GSM). In this paper we have studied the use of complex-valued multilayer perceptron (MLP) network for equalization purposes when transmitting data bursts through GSM-channels and through a nonlinear channel. In addition to the conventional complex backpropagation algorithm for the training of the network, we have also presented a complex version of the Resilient PROPagation (RPROP) algorithm. These training methods are then compared and studied using GSM channel models as well as a nonlinear channel model. Performance comparisons are made in terms of bit error rates (BERs) and computational complexity. Results show that the MLP network trained with complex RPROP algorithm achieves approximately as good bit error rates as the MLP network trained with complex backpropagation, but with clearly smaller computational load.

A Real Time Complex EKF Learning Algorithm for Multilayered Neural Networks

The complex backpropagation (BP) algorithm to train the neural networks for complex signals is available in the literature. However, the convergence speed of complex BP is slow. Hence, in this paper, a real time complex extended Kalman filtering (EKF) algorithm which has faster convergence speed than the complex BP algorithm is obtained. An approach using a two step recursion procedure for updating the weights of the complex EKF algorithm has been suggested. Illustrative simulation results are promising and confirm the efficacy of the algorithm.

A data extrapolation algorithm using a complex domain neural network

Many applications make use of the discrete Fourier transform (DFT) during data manipulation. The resolution for such applications is inversely proportional to the available data length used during the DFT. Resolution can be improved by modeling and then implicitly or explicitly extrapolating the known data to increase its effective length prior to the application of the DFT. A new data extrapolation algorithm based on a complex-domain feedforward neural network is detailed in this brief. The complex back-propagation algorithm used to train the network includes adaptive learning and momentum methods normally found in real-valued neural networks. Approaches to increase the extrapolation stability are discussed. The success of the algorithm is demonstrated by using short data sets to reconstruct phantom and medical magnetic resonance images which suffer from severe artifacts when reconstructed by the standard Fourier technique.

On the complex backpropagation algorithm

A recursive algorithm for updating the coefficients of a neural network structure for complex signals is presented. Various complex activation functions are considered and a practical definition is proposed. The method, associated to a mean-square-error criterion, yields the complex form of the conventional backpropagation algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The complex backpropagation algorithm

The backpropagation (BP) algorithm that provides a popular method for the design of a multilayer neural network to include complex coefficients and complex signals so that it can be applied to general radar signal processing and communications problems. It is shown that the network can classify complex signals. The generalization of the BP to deal with complex signals should make it possible to expand the line of applications of this powerful nonlinear signal processing algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>