Modified Backpropagation Algorithm Research Articles

Design and Experimental Validation of an Adaptive Multi-Layer Neural Network Observer-Based Fast Terminal Sliding Mode Control for Quadrotor System

This study considers the numerical design and practical implementation of a new multi-layer neural network observer-based control design technique for unmanned aerial vehicles systems. Initially, an adaptive multi-layer neural network-based Luenberger observer is designed for state estimation by employing a modified back-propagation algorithm. The proposed observer’s adaptive nature aids in mitigating the impact of noise, disturbance, and parameter variations, which are usually not considered by conventional observers. Based on the observed states, a nonlinear dynamic inversion-based fast terminal sliding mode controller is designed to attain the desired attitude and position tracking control. This is done by employing a two-loop control structure. Numerical simulations are conducted to demonstrate the effectiveness of the proposed scheme in the presence of disturbance, parameter uncertainty, and noise. The numerical results are compared with current approaches, demonstrating the superiority of the proposed method. In order to assess the practical effectiveness of the proposed method, hardware-in-loop simulations are conducted by utilizing a Pixhawk 6X flight controller that interfaces with the mission planner software. Finally, experiments are conducted on a real F450 quadrotor in a secured laboratory environment, demonstrating stability and good tracking performance of the proposed MLNN observer-based SMC control scheme.

A Soft Sensor for Multirate Quality Variables Based on MC-CNN.

In recent years, data-driven soft sensor modeling methods have been widely used in industrial production, chemistry, and biochemical. In industrial processes, the sampling rates of quality variables are always lower than those of process variables. Meanwhile, the sampling rates among quality variables are also different. However, few multi-input multi-output (MIMO) sensors take this temporal factor into consideration. To solve this problem, a deep-learning (DL) model based on a multitemporal channels convolutional neural network (MC-CNN) is proposed. In the MC-CNN, the network consists of two parts: the shared network used to extract the temporal feature and the parallel prediction network used to predict each quality variable. The modified BP algorithm makes the blank values generated at unsampled moments not participate in the backpropagation (BP) process during training. By predicting multiple quality variables of two industrial cases, the effectiveness of the proposed method is verified.

A multi-layer photonic spiking neural network with a modified backpropagation algorithm for nonlinear classification

In this work, we construct a fully connected multi-layer photonic spiking neural network (PSNN) based on excitable vertical-cavity surface-emitting lasers with an embedded saturable absorber (VCSELs-SA) equipped with the modified supervised backpropagation (BP) algorithm. Different from the traditional BP algorithm, our proposed modified BP algorithm is based on optical spikes, and its activation function exploits the excitable characteristic of VCSELs-SA. It is demonstrated that this BP PSNN can be competent in linearly inseparable problems by correctly solving the exclusive or (XOR) problem. Thus, a three-layer fully connected BP PSNN is designed to perform the prediction of Human immunodeficiency virus-1 protease (HIV-1 PR) cleavage sites and achieves 91.95% classification accuracy with the modified BP algorithm, which could be a useful tool in helping to find effective inhibitors of HIV PR to fight against acquired immunodeficiency syndrome (AIDS). These results demonstrate the validity of our modified BP algorithm in dealing with nonlinear classification problems in PSNNs, which is helpful for supervised learning in the photonic neuromorphic computing architecture for future neuromorphic applications.

Modified Backpropagation Algorithm with Multiplicative Calculus in Neural Networks

Backpropagation is one of the most widely used algorithms for training feedforward deep neural networks. The algorithm requires a differentiable activation function and it performs computations of the gradient proceeding backwards through the feedforward deep neural network from the last layer through to the first layer. In order to calculate the gradient at a specific layer, the gradients of all layers are combined via the chain rule of calculus. One of the biggest disadvantages of the backpropagation is that it requires a large amount of training time. To overcome this issue, this paper proposes a modified backpropagation algorithm with multiplicative calculus. Multiplicative calculus provides an alternative to the classical calculus and it defines new kinds of derivative and integral forms in multiplicative form rather than addition and subtraction forms. The performance analyzes are discussed in various case studies and the results are given comparatively with classical backpropagation algorithm. It is found that the proposed modified backpropagation algorithm converges in less time to the solution and thus provides fast training in the given case studies. It is also shown that the proposed algorithm avoids the local minima problem.

Neural network disturbance observer with extended weight matrix for spacecraft disturbance attenuation

A modified neural network-based disturbance observer with high-accuracy estimation performance is fabricated for spacecraft attitude control in this paper. To reduce the approximation error of the neural network with a limited number of neurons for complex disturbances, the weight matrix of the neural network is extended into a time series expansion and updated by a modified back propagation algorithm. Thus, the high-order dynamics of the weight matrix can be reconstructed, and the disturbance estimation accuracy is improved. Then, by introducing an integral state of the lumped disturbance into the disturbance model and utilizing it as the virtual disturbance measurement, an improved neural network-based disturbance observer is designed. The proposed observer takes advantages of the nonlinear approximation capability of the neural network and the high-order dynamics estimation ability of the extended observer structure. To realize high-performance attitude control of a spacecraft, a composite controller based on the proposed observer is designed, and then tested by comparative simulations. Simulation results show that, the estimation error of the designed observer can be reduced by an order of magnitude than the traditional observers. It is suitable for high-performance disturbance attenuation of verities of applications.

A Novel Signal Localized Convolution Neural Network for Power Transformer Differential Protection

This paper presents a novel signal localized convolution neural network (SLCNN) for the power transformer differential protection. The distinct signal localization is performed with the convolution process sequentially on the frequency and time coefficients which are obtained from the wavelet decomposition of the differential current signal. The SLCNN is trained with a modified back-propagation algorithm according to SLCNNs architecture. Three power transformer test systems are considered for evaluation of the proposed SLCNN. The training patterns of each transformer are generated for various operating conditions. The SLCNN for each transformer is trained, validated and tested using its corresponding patterns. Then the performance of SLCNN is evaluated through confusion matrix analysis and is also compared with long short-term memory (LSTM) deep neural network, support vector machine (SVM), conventional back-propagation neural network (CBPNN) and conventional biased restraint second harmonic (CBSH) blocking method.

Genetic Algorithm with Radial Basis Mapping Network for the Electricity Consumption Modeling

The modified backpropagation algorithm based on the backpropagation with momentum is used for the parameters updating of a radial basis mapping (RBM) network, where it requires of the best hyper-parameters for more precise modeling. Seeking of the best hyper-parameters in a model it is not an easy task. In this article, a genetic algorithm is used to seek of the best hyper-parameters in the modified backpropagation for the parameters updating of a RBM network, and this RBM network is used for more precise electricity consumption modeling in a city. The suggested approach is called genetic algorithm with a RBM network. Additionally, since the genetic algorithm with a RBM network starts from the modified backpropagation, we compare both approaches for the electricity consumption modeling in a city.

SPIF Quality Prediction Based on Experimental Study Using Neural Networks Approaches

Abstract—This paper deals with the quality prediction of the Single Point Incremental forming (SPIF) process. The quality prediction can be evaluated through five parameters: Roughness surface, thickness, springback, circularity and position errors. Despite the contribution of many researchers on the development of sheet metal forming process, the geometric accuracy of the formed part remains less developed and analyzed. Several parameters are relevant to this inaccuracy namely the complexity of the part geometry, the Elasto-Plastic Material Behavior and tool path strategy. The present work proposes an experimental study for a complex geometry part (double truncated cone) obtained by SPIF. To product a truncated cone, two different trajectories were used: single and alternating directions. While in literature three quality parameters are generally used (roughness surface, thickness and springback) we propose in the paper to predict moreover two other quality parameters which are the circularity and the position errors. To deal with the nonlinearity of the problem we proposed to use an ANN and benefit of its generalization capacities to generate new and unpredictable situations through different input parameters: Strategy tool path, incremental step size, spindle speed, feed rate, and the forming angle. To improve the generalization accuracy of the neural network the modified back propagation algorithm was used in the learning phase of one hidden multilayer neural network. Experimental results show that the new proposed prediction model allows to reach an accurate prediction more than 96.74% with respect to all the quality parameters.

Modified Backpropagation with Added White Gaussian Noise in Weighted Sum for Convergence Improvement

The impact of the noise injection in backpropagation is studied since last two decades. In this manuscript, a brief review is done on noise injection techniques used in the backpropagation. Here, the modified backpropagation algorithm is proposed, in which the white Gaussian noise is added in the weighted sum entity of the backpropagation. The experimentation is carried out on different standard benchmark problems such as 2 bit parity and iris dataset. It is observed that the proposed modified backpropagation requires less number of epochs as compared to the standard backpropagation for the convergence when applied on 2 bit parity and Iris dataset.

Backpropagation algorithm with fractional derivatives

The paper presents a model of a neural network with a novel backpropagation rule, which uses a fractional order derivative mechanism. Using the Grunwald Letnikow definition of the discrete approximation of the fractional derivative, the author proposed the smooth modeling of the transition functions of a single neuron. On this basis, a new concept of a modified backpropagation algorithm was proposed that uses the fractional derivative mechanism both for modeling the dynamics of individual neurons and for minimizing the error function. The description of the signal flow through the neural network and the mechanism of smooth shape control of the activation functions of individual neurons are given. The model of minimization of the error function is presented, which takes into account the possibility of changes in the characteristics of individual neurons. For the proposed network model, example courses of the learning processes are presented, which prove the convergence of the learning process for different shapes of the transition function. The proposed algorithm allows the learning process to be conducted with a smooth modification of the shape of the transition function without the need for modifying the IT model of the designed neural network. The proposed network model is a new tool that can be used in signal classification tasks.

A Feed‐Forward Wavelet Neural Network Adaptive Observer‐Based Fault Detection Technique for Spacecraft Attitude Control Systems

This paper presents a novel neural network-based fault detection technique applicable to a class of nonlinear systems. The adaptive observer was designed for fault detection based on a single hidden layer feed-forward wavelet neural network. In order to guarantee network convergence, the network weights are updated according to a modified back-propagation algorithm, and the Lyapunov function is introduced to ensure stability. The proposed fault detection scheme was tested on the actuators of a typical spacecraft attitude control system, and the results demonstrated the effectiveness and feasibility of the proposed observer in detecting nonlinear system failure.

Drilling Rig Operation Mode Recognition by an Artificial Neuronet

The article proposes a way to develop a drilling rig operation mode classifier specialized to recognize pre-emergency situations appearable in commercial oil-and-gas well drilling. The classifier is based on the theory of image recognition and artificial neuronet taught on real geological and technological information obtained while drilling. To teach the neuronet, a modified backpropagation algorithm that can teach to reach the global extremum of a target function has been proposed. The target function was a relative recognition error to minimize in the teaching. Two approaches to form the drilling rig pre-emergency situation classifier based on a taught neuronet have been considered. The first one involves forming an output classifier of N different signals, each of which corresponds to a single recognizable situation and, and can be formed on the basis of the analysis of M indications, that is using a uniform indication vocabulary for all recognized situations. The second way implements a universal classifier comprising N specialized ones, each of which can recognize a single pre-emergency situation and having a single output.

Robot manipulator identification based on adaptive multiple-input and multiple-output neural model optimized by advanced differential evolution algorithm

This article proposes a novel advanced differential evolution method which combines the differential evolution with the modified back-propagation algorithm. This new proposed approach is applied to train an adaptive enhanced neural model for approximating the inverse model of the industrial robot arm. Experimental results demonstrate that the proposed modeling procedure using the new identification approach obtains better convergence and more precision than the traditional back-propagation method or the lonely differential evolution approach. Furthermore, the inverse model of the industrial robot arm using the adaptive enhanced neural model performs outstanding results.

A Modified Back Propagation Algorithm for Assyrian Optical Character Recognition Based on Moments

Character recognition has been very popular and interested area for researches, and it continues to be a challenging and impressive research topic due to its diverse applicable environment. The optical character recognition has been introduced as a fast and accurate method to convert both existing text images as well as large archives of existing paper documents to editable digital text format. However, existing optical character recognition algorithms suffer from flawed tradeoffs between accuracy and speed, making them less effective and impractical for large and complex documents. This paper describes a suggested method for Assyrian optical character recognition using modified back propagation artificial neural network based on moments. The experimental results show that the proposed method achieves higher recognition accuracy rate in compared with the standard algorithm.

Radial Basis Function Network Learning with Modified Backpropagation Algorithm

Radial Basis Function Network (RBFN) is a class of Artificial Neural Network (ANN) that was used in many classification problems in science and engineering. Backpropagation (BP) algorithm is a learning algorithm that was widely used in ANN. However, BP has major disadvantages of slow error rate convergence and always easily stuck at the local minima. Hence, Modified BP algorithm was proposed in this study to improve the learning speed of RBFN using discretized data. C programming language was used to develop the program for the proposed method. Performance measurement of the method was conducted and the experimental results indicate that our proposed method performs better in error rate convergence and correct classification compared to the result with continuous dataset. T-test statistical analysis was used to check the significance of the results and most were found to be satisfactorily significant. DOI: http://dx.doi.org/10.11591/telkomnika.v13i2.7032

A Modified Back Propagation Algorithm of Neural Network with Global Optimization

The advantages and weakens of traditional BP algorithm is briefly analyzed and an efficient global optimization algorithm is proposed.The basic principle of the algorithm is presented,and a new BP neural network algorithm based on the existing BP algorithm and the new global optimization algorithm is proposed, considering the new global optimization algorithm can solve the problem of local minimum efficiently. To verify the effectiveness of the new BP algorithm,the paper compared the experimental results of various algorithms in solving function fitting problem.

Strength Prediction of Aluminum–Stainless Steel-Pulsed TIG Welding–Brazing Joints with RSM and ANN

Pulsed TIG welding–brazing process was applied to join aluminum with stainless steel dissimilar metals. Major parameters that affect the joint property significantly were identified as pulsed peak current, base current, pulse on time, and frequency by pre-experiments. A sample was established according to central composite design. Based on the sample, response surface methodology (RSM) and artificial neural networks (ANN) were employed to predict the tensile strength of the joints separately. With RSM, a significant and rational mathematical model was established to predict the joint strength. With ANN, a modified back-propagation algorithm consisting of one input layer with four neurons, one hidden layer with eight neurons, and one output layer with one neuron was trained for predicting the strength. Compared with RSM, average relative prediction error of ANN was <10% and it obtained more stable and precise results.

Recurrent Spiking Neural Networks the Third Generation in Identification of Systems

paper the modified identification method for nonlinear systems is proposed based on Recurrent Spiking Neural Networks (RSNN). Spike Response Model (SRM) has been employed in the modification method. The learning of the parameters of RSNN is based on modified backpropagation algorithm which is known as SpikeProp. In the identification of a variety of types of nonlinear systems, a coding equation is applied to convert real numbers into spike times. The RSNN structure is tested for the identification of the nonlinear systems. The simulation results show that the proposed modification method provides a good performance in terms of execution time and minimizing error in the training phase. . KeywordsNeural Networks, Identification, nonlinear systems, SpikeProp

A Novel Topology in Modular Ann Approach for Multi-Modal Concept Identification and Image Retrieval

Concept identification and automated image annotation helps to overcome the non-trivial issue of image retrieval. It is accomplished by decomposing the image at object level and extracting concepts from objects. Several applications like biometrics, geographical information systems, and automatic target recognition possess highly diffused feature vectors. Hence, a conventional Artificial Neural Network (ANN) fails to deliver truthful results. In this work, we have developed a modified back propagation algorithm and have proposed a novel topology in Modular Artificial Neural Network (MANN) to rectify the problem faced in concept identification. In this topology we have trained each module of ANN for one object in one verse all fashion and highly ranked output is taken as classified output. Our argument is supported by combined application of afore-mentioned algorithm and novel topology using MATLAB simulations on two different datasets.

Nonlinear effects compensation in Optical Coherent PDM-QPSK Systems

Nonlinear effects has been appointed as the main limitation in coherent optical fiber transmission. Digital Back-Propagation algorithms and Maximum Likelihood Sequence Estimation are two of the current studied methods to cope with such impairment and extend the systems maximum reach. In this article, we analyzed both methods in a 112 Gb/s Dual Polarization Quadrature Phase Shifting Keying (DP-QPSK) optical coherent system through simulations and experimental results. In order to reduce the huge required computational complexity, a modified back-propagation algorithm is also analyzed.