Dynamic Backpropagation Algorithm Research Articles

A Neural Network NARMA-L2 Tracking Control for Electronic Throttle System*

<div class="section abstract"><div class="htmlview paragraph">In this paper, the Artificial Neural Network (ANN) control strategy based on the Nonlinear Auto Regressive Moving Average-Level 2 (<i>NARMA-L2</i>) technique has been used for tracking control of an electronic throttle body. The <i>NARMA-L2</i> nonlinear plant model is first identified offline by training using a set of input-output data pairs measured at different operating conditions. This data was collected from an actual operation of the throttle body running in a closed-loop control system on a prototype vehicle. The identified <i>NARMA-L2</i> plant model was then inverted and used to force the throttle output position to approximately track any reference inputs with multiple set-point changes at different operating conditions. The <i>NARMA-L2</i> model was reconfigured to be an equivalent model of a feed-forward controller that can cancel not only the actual dynamic behavior of the throttle body but also the nonlinearity effects. This type of controller has great potential to overcome the difficulty of developing an inverse dynamic model of the throttle body [<span class="xref">1</span>] and has the ability to compensate any un-modeled nonlinear dynamic changes that may be excited due to environmental changes. These changes can include load, temperature, humidity, and external disturbances with noise effects that may be introduced into the closed-loop control of the throttle body during real time operation in the vehicle. The resulting closed loop control system used the <i>NARMA-L2</i> architecture that have a multilayer perceptron neural network structure with the dynamic back-propagation algorithm becomes an implicit algebraic model that perfectly tracks any multiple set-point changes at different operating conditions. Testing results from real time implementation are provided to illustrate the new controller performance for tracking controls during actual operation on a prototype vehicle. The identified <i>NARMA-L2</i> neuro-controller shows more robustness and better performs as compared to other controllers developed in previous work [<span class="xref">1</span>-<span class="xref">3</span>].</div><div class="htmlview paragraph">* This work was done while the author was at IAV Automotive Engineering Inc, 15620 Technology Drive, Northville, MI 48168.</div><div class="htmlview paragraph">† Currently working with MAHLE Powertrain LLC, 14900 GalleonCourt, Plymouth, MI 48170.</div></div>

Adaptive Control Using Neural Networks and Approximate Models for Nonlinear Dynamic Systems

In this research, a comparative study of two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive moving average (NARMA-L2), and a feedforward neural network (FFNN) is performed for their ability to provide adaptive control of nonlinear systems. Three dynamical nonlinear systems of different complexity are considered. The aim of this work is to make the output of the plant follow the desired reference trajectory. The problem becomes more challenging when the dynamics of the plants are assumed to be unknown, and to tackle this problem, a multilayer neural network-based approximate model is set up which will work in parallel to the plant and the control scheme. The network parameters are updated using the dynamic backpropagation (BP) algorithm.

Lyapunov stability-Dynamic Back Propagation-based comparative study of different types of functional link neural networks for the identification of nonlinear systems

In this paper, the performance comparison of various types of functional link neural networks (FLNNs) has been done for the nonlinear system identification. The FLNNs being compared in the present study are: trigonometry FLNN, Legendre FLNN (LeFLNN), Chebyshev FLNN, power series FLNN (PSFLNN) and Hermite FLNN. The recursive weights adjustment equations are derived using the combination of Lyapunov stability criterion and dynamic back propagation algorithm. In the simulation study, a total of three nonlinear systems (both static and dynamic systems) are considered for testing and comparing the approximation ability and computational complexity of the above-mentioned FLNNs. From the simulation results, it is observed that the LeFLNN has given better approximation accuracy and PSFLNN offered least computational load as compared to the rest models.

Optimization of a linear controller using dynamic back-propagation

This paper presents the optimization of a linear controller for a DC motor using Dynamic Back-Propagation algorithm. This algorithm is commonly employed for neural networks training; however, it can be used for optimization of a linear controller. The results show a satisfactory controller optimization.

Dynamics identification and control of nonlinear MIMO coupled plant using supervised neural gas and comparison with recurrent neural controller

The dynamics identification and subsequent control of a nonlinear system is not a trivial issue. The application of a neural gas network that is trained with a supervised batch version of the algorithm can produce identification models in a robust way. In this paper, the neural model identifies each local transfer function, demonstrating that the local linear approximation can be done. Moreover, other parameters are analyzed in order to obtain a correct modeling. Furthermore, the algorithm is applied to control a nonlinear multi-input multi-output system composed of tanks. In addition, this plant is a coupled system where the manipulated input variables are influencing all the output variables. The aim of the work is to demonstrate that the supervised neural gas algorithm is able to obtain linear models to be used in a state space design scenario to control nonlinear coupled systems and guarantee a robust control method. The results are compared with the common approach of using a recurrent neural controller trained with a dynamic backpropagation algorithm. Regarding the steady-state errors in disturbance rejection, reference tracking and sensitivity to simple process changes, the proposed approach shows an interesting application to control nonlinear plants.

Dynamic TOPSIS fuzzy cerebellar model articulation controller for magnetic levitation system

This study proposes a fuzzy Cerebellar Model Articulation Controller (CMAC) using a dynamic Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) technique for dealing with the metallic sphere position control of a magnetic levitation system (MLS). The proposed Dynamic TOPSIS F uzzy CMAC (DTFCMAC) incorporates a multi-criteria decision analysis with a fuzzy structure to decrease the computational load for parameter learning and to enhance the fuzzy reasoning inference for a CMAC. The Shannon entropy index is used to derive the objective weights for the evaluation criterion. By combining entropy weight and TOPSIS, the optimal threshold value for suitable firing nodes is determined automatically and easily. In the proposed method, the dynamic back-propagation algorithm is applied to train the proposed DTFCMAC online. Moreover, to guarantee the convergence of output tracking error for periodic command tracking, analytical methods developed from a discrete-type Lyapunov function are used to determine the optimal learning-rate parameters for the proposed DTFCMAC. The proposed DTFCMAC is applied to the MLS, and its performance is verified through simulations and experiments. Our findings indicate that the proposed DTFCMAC control system achieves stability and desired control performance for the MLS.

Temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems

In this paper, a novel temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems is proposed. The proposed structure consists of recurrent hidden neurons having weighted self-feedback loops and a weighted linear feed-through from the input layer directly to the output layer neuron(s). The dynamic back-propagation algorithm is developed and used for updating the parameters of the proposed structure. To improve the performance of learning algorithm, discrete Lyapunov stability method is used to develop an adaptive learning rate scheme. This scheme ensures the faster convergence of the parameters and maintains the stability of the system. A total of 5 complex nonlinear systems are used to test and compare the performance of the proposed network with other neural network structures. The disturbance rejection tests are also carried out to check whether the proposed scheme is able to handle the external disturbance/noise signals effects or not. The obtained results show the efficacy of the proposed method.

Comparative study of neural networks for dynamic nonlinear systems identification

In this paper, a comparative study is performed to test the approximation ability of different neural network structures. It involves three neural networks multilayer feedforward neural network (MLFFNN), diagonal recurrent neural network (DRNN), and nonlinear autoregressive with exogenous inputs (NARX) neural network. Their robustness is also tested and compared when the system is subjected to parameter variations and disturbance signals. Further, dynamic back-propagation algorithm is used to update the parameters associated with these neural networks. Four dynamical systems of different complexities including motor-driven robotic link are considered on which the comparative study is performed. The simulation results show the superior performance of DRNN identification model over NARX and MLFFNN identification models.

Comparative Study of Neural Networks for Control of Nonlinear Dynamical Systems with Lyapunov Stability-Based Adaptive Learning Rates

This paper performs the comparative study of two feed-forward neural networks: radial basis function network (RBFN), multilayer feed-forward neural network (MLFFNN) and a recurrent neural network: nonlinear auto-regressive with exogenous inputs (NARX) neural network for their ability to provide an adaptive control of nonlinear systems. Dynamic back-propagation algorithm is used to derive parameter update equations. To ensure stability and faster convergence, an adaptive learning rate is developed in the sense of discrete Lyapunov stability method. Both parameter variation and disturbance signal cases are considered for checking and comparing the robustness of controller. Three simulation examples are considered for carrying out this study. The results so obtained reveal that RBFN-based controller is performing better than that of NARX- and MLFFNN-based controllers.

Recurrent Neural Network Model for Prediction of Microclimate in Solar Greenhouse

A recurrent neural network(RNN) based dynamic back propagation(BP) algorithm model with historical internal inputs are developed to predict the temperature and humidity of a solar greenhouse in the north of China. Climate data including air and substrate temperature, air humidity, illumination and C02 concentration recorded over eight days were used to build and validate models for climatic prediction. In order to compare the accuracy of predictions, different performance measures, such as average relative error (ARE), mean absolute error (AME) and root mean square error (RSME), were calculated, for using BP and untrained RNN neural networks using the same processing. For the RNN model, a context layer as the last hidden layer output, is input with the next input to the next hidden layer, which is equivalent to the state feedback. The results demonstrate that the RNN-BP model provides reasonably good predictions with the RSME for temperature 0.751 and 0.781 for humidity, including both of the R2 is both above 0.9, which outperforms the compared models tested in this paper.

Diagonal recurrent neural networks for parameters identification of terrain based on wheel–soil interaction analysis

Wheeled mobile robots (WMR) are often applied to travel on outdoor unstructured environment, such as loose soil or variable field terrain. Learning the knowledge of terrain has played a significant role for better mobility and stability of WMR. In this study, a diagonal recurrent neural network-based adaptive method is proposed to identify terrain parameters by the platform of a single driving wheel. According to the classical terramechanics model of wheel---soil interaction, a decoupling simplification model is developed by closed-form analytical equations. Five unknown terrain parameters are divided into two groups and included in two complex nonlinear equations. These parameters are used to compute the model outputs of force and torque of wheel---soil interaction. Dynamic back propagation algorithm is applied to update these parameters for compensating the errors between the prediction of neural network and measurable data in real time. The results of simulation show that the terrain parameters can be obtained and approximate the experimental value of terrain parameters when the predictive errors converge to zero.

Stable adaptive output feedback controller for a class of uncertain non‐linear systems

This study deals with design of an adaptive output feedback tracking controller for a class of non-linear systems with unknown fixed control direction. By using neural networks and deriving adaptive rules based on the steepest descent algorithm, the authors present a stable output feedback control scheme, which is applicable to a wide class of unknown complicated non-linear systems. Therefore an approach based on the dynamic back propagation algorithm is proposed to develop the adaption laws for systems with more general model structure. Using Lyapunov's direct method, uniformly ultimately boundedness of all signals of the closed-loop system is also ensured. Moreover, it is shown that the bounds on the tracking errors depend on the designing parameters. Hence, an arbitrarily small tracking error can be achieved by adjusting the parameters properly. Finally, simulation results performed on a non-affine uncertain non-linear system having internal dynamics are given to demonstrate the effectiveness of the proposed scheme and the theoretical discussions.

A context layered locally recurrent neural network for dynamic system identification

This paper puts forward a novel recurrent neural network (RNN), referred to as the context layered locally recurrent neural network (CLLRNN) for dynamic system identification. The CLLRNN is a dynamic neural network which appears in effective in the input–output identification of both linear and nonlinear dynamic systems. The CLLRNN is composed of one input layer, one or more hidden layers, one output layer, and also one context layer improving the ability of the network to capture the linear characteristics of the system being identified. Dynamic memory is provided by means of feedback connections from nodes in the first hidden layer to nodes in the context layer and in case of being two or more hidden layers, from nodes in a hidden layer to nodes in the preceding hidden layer. In addition to feedback connections, there are self-recurrent connections in all nodes of the context and hidden layers. A dynamic backpropagation algorithm with adaptive learning rate is derived to train the CLLRNN. To demonstrate the superior properties of the proposed architecture, it is applied to identify not only linear but also nonlinear dynamic systems. The efficiency of the proposed architecture is demonstrated by comparing the results to some existing recurrent networks and design configurations. In addition, performance of the CLLRNN is analyzed through an experimental application to a dc motor connected to a load to show practicability and effectiveness of the proposed neural network. Results of the experimental application are presented to make a quantitative comparison with an existing recurrent network in the literature.

A hybrid neural genetic method for load forecasting based on phase space reconstruction

PurposeThe purpose of this paper is to improve forecasting accuracy for short‐term load series.Design/methodology/approachA forecasting method based on chaotic time series and optimal diagonal recurrent neural networks (DRNN) is presented. The input of the DRNN is determined by the embedding dimension of the reconstructed phase space, and adaptive dynamic back propagation (DBP) algorithm is used to train the network. The connection weights of the DRNN are optimized via modified genetic algorithms, and the best results of optimization are regarded as initial weights for the network. The new method is applied to predict the actual short‐term load according to its chaotic characteristics, and the forecasting results also validate the feasibility.FindingsFor the chaos time series, the hybrid neural genetic method based on phase space reconstruction can carry out the short‐term prediction with the higher accuracy.Research limitations/implicationsThe proposed method is not suited to medium and long‐term load forecasting.Practical implicationsThe accuracy of the load forecasting is important to the economic and secure operation of power systems; also, the neural genetic method can improve forecasting accuracy.Originality/valueThis paper will help overcome the defects of traditional neural network and make short‐term load forecasting more accurate and fast.

Recurrent Polynomial Neural Networks for Enhancing Performance of GPS in Electric Systems

Global Positioning System (GPS) is a worldwide satellite system that provides navigation, positioning, and timing for both military and civilian applications. GPS based time reference provides inexpensive but highly-accurate timing and synchronization capability and meets requirements in power system fault location, monitoring, and control. In the present era of restructuring and modernization of electric power utilities, the applications of GIS/GPS technology in power industry are growing and covering several technical and man-agement activities. Because of GPS receiver’s error sources are time variant, it is necessary to remove the GPS measurement noise. This paper presents novel recurrent neural networks called the Recurrent Pi-Sigma Neural Network (RPSNN) and Recurrent Sigma-Pi Neural Network (RSPNN). The proposed NNs have been used as predictor in GPS receivers timing errors. The NNs were trained using the dynamic Back Propagation (BP) algorithm. The actual data collection was used to test the performance of the proposed NNs. The ex-perimental results obtained from a Coarse Acquisition (C/A)-code single-frequency GPS receiver strongly support the potential of the method using RPSNN to give high accurate timing. The GPS timing RMS error reduces from 200 to less than 40 nanoseconds.

Development of a new EDRNN procedure in control of human arm trajectories

In this paper the trajectory tracking control of a human arm moving on the sagittal plane is investigated by an interdisciplinary approach with the combination of neural network mapping, evolutionary computation, and dynamic system control. The arm in the study is described by a musculoskeletal model with two degrees of freedom and six muscles, and the control signal is applied directly in the muscle space. A new control system structure is proposed to manipulate the complicated nonlinear dynamical arm motion. To design the intelligent controller, an evolutionary diagonal recurrent neural network (EDRNN) is integrated with proper performance indices, in which genetic algorithm (GA) and evolutionary program (EP) strategy are effectively integrated with the diagonal recurrent neural network (DRNN). The hybrid GA with EP strategy is applied to optimize the DRNN architecture and an adaptive dynamic back-propagation (ADBP) algorithm with momentum for the multi-input multi-output (MIMO) systems is used to obtain the network weights. The effectiveness of the control scheme is demonstrated through a simulated case study.

Implementation of Artificial Neural Network-Based Tracking Controller for High-Performance Stepper Motor Drives

Two distinct multilayer perception neural networks (NNs) are implemented via laboratory experiment to simultaneously identify and adaptively control the trajectory tracking of a hybrid step motor assumed to operate in a high-performance drives environment. That is, a neural network identifier (NNI) which captures the nonlinear dynamics of the stepper motor drive system (SMDS) over any arbitrary time interval in its range of operation, and a neural network controller (NNC) to provide the necessary control actions as to achieve trajectory tracking of the rotor speed. The exact form of the control law is unknown, and must be estimated by the NNC. Consequently, the NNC is constructed as a nonlinear unknown function depending on the current state of the drive system supplies by the NNI and the reference trajectory we wish the outputs to follow. The two NNs are online trained using dynamic back-propagation algorithm. The composite structure is used as a speed controller for the SMDS. Performance of the composite controller is evaluated through a laboratory experiment. Experimental results show the effectiveness of this approach, and demonstrate the usefulness of the proposed controller in high-performance drives

A NEW VERSION OF ELMAN NEURAL NETWORKS FOR DYNAMIC SYSTEMS MODELING AND CONTROL

Elman network is a class of recurrent neural networks used for function approximation. The main problem of this class is that its structure has a set of global sigmoid functions at its hidden layer. That means that if the operating conditions of a process be identified, are changed the function approximation property of the network is degraded. This paper introduces a new version of the Elman network named Elman Recurrent Wavelet Neural Network (ERWNN). It merges the multi- resolution property of the wavelets and the learning capabilities of the Elman neural network to inherit the advantages of the two paradigms and to avoid their drawbacks. Stability and convergence property is proven for the proposed network. The paper also develops a model reference control scheme using the proposed ERWNN. The proposed scheme belongs to indirect adaptive control schemes. The dynamic back propagation (DBP) algorithm is employed to train both the two networks structured for the indirect control scheme. This paper derives also the plant sensitivity for adjusting the parameters of the developed controller. The advantages of this new version of ERWNN in modeling and controlling time intensive dynamic processes, are reflected in our simulation results.

Implementation of LLCC-Resonant Driving Circuit and Adaptive CMAC Neural Network Control for Linear Piezoelectric Ceramic Motor

In this paper, an adaptive cerebellar-model articulation computer (CMAC) neural network (NN) control system is developed for a linear piezoelectric ceramic motor (LPCM) that is driven by an LLCC-resonant inverter. The motor structure and LLCC-resonant driving circuit of an LPCM are introduced initially. The LLCC-resonant driving circuit is designed to operate at an optimal switching frequency such that the output voltage will not be influenced by the variation of quality factor. Since the dynamic characteristics and motor parameters of the LPCM are highly nonlinear and time varying, an adaptive CMAC NN control system is designed without mathematical dynamic model to control the position of the moving table of the LPCM drive system to achieve high-precision position control with robustness. In the proposed control scheme, the dynamic backpropagation algorithm is adopted to train the CMAC NN online. Moreover, to guarantee the convergence of output tracking error for periodic commands tracking, analytical methods based on a discrete-type Lyapunov function are utilized to determine the optimal learning-rate parameters of the CMAC NN. The effectiveness of the proposed driving circuit and control system is verified by experimental results in the presence of uncertainties, and the advantages of the proposed control system are indicated in comparison with a traditional integral-proportional position control system. Accurate tracking response and superior dynamic performance can be obtained due to the powerful online learning capability of the CMAC NN with optimal learning-rate parameters.

Diagonal recurrent neural networks for a walking robot

Whilst the necessity of finding an intelligent-based controlling method for a two-leg walking robot increases, the balance of the under-actuated leg consisting of two links is emphasized in this study. This is not only a nonlinear structure, but also a single-input double-output system. However, the problem becomes concrete through the proposed diagonal recurrent neural networks (DRNN) method. In this paper, two kinds of DRNN are introduced into the control system. The diagonal recurrent neuroidentifier (DRNI) is selected as an identifier, and the diagonal recurrent neurocontroller (DRNC) is determined as a controller. Additionally, a generalized dynamic backpropagation algorithm (DBP) is also applied to train both DRNC and DRNI. With the simulated results, it is shown that the under-actuated leg is balanced and stabilized by DRNN. This study definitely contributes the intelligent-based as well as the real-time controlled method for a two-leg walking robot with profound insight.