Dynamic Neural Network Structure Research Articles

Dynamic Neural Network Structure: A Review for Its Theories and Applications

Dynamic Neural Network Structure: A Review for Its Theories and Applications

On the dynamic neural network toolbox design for identification, estimation and control

PurposeThere are common problems in the identification of uncertain nonlinear systems, nonparametric approximation, state estimation, and automatic control. Dynamic neural network (DNN) approximation can simplify the development of all the aforementioned problems in either continuous or discrete systems. A DNN is represented by a system of differential or recurrent equations defined in the space of vector activation functions with weights and offsets that are functionally associated with the input data.Design/methodology/approachThis study describes the version of the toolbox, that can be used to identify the dynamics of the black box and restore the laws underlying the system using known inputs and outputs. Depending on the completeness of the information, the toolbox allows users to change the DNN structure to suit specific tasks.FindingsThe toolbox consists of three main components: user layer, network manager, and network instance. The user layer provides high-level control and monitoring of system performance. The network manager serves as an intermediary between the user layer and the network instance, and allows the user layer to start and stop learning, providing an interface to indirectly access the internal data of the DNN.Research limitations/implicationsControl capability is limited to adjusting a small number of numerical parameters and selecting functional parameters from a predefined list.Originality/valueThe key feature of the toolbox is the possibility of developing an algorithmic semi-automatic selection of activation function parameters based on optimization problem solutions.

Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic Nodes.

Large-scale artificial neural networks have many redundant structures, making the network fall into the issue of local optimization and extended training time. Moreover, existing neural network topology optimization algorithms have the disadvantage of many calculations and complex network structure modeling. We propose a Dynamic Node-based neural network Structure optimization algorithm (DNS) to handle these issues. DNS consists of two steps: the generation step and the pruning step. In the generation step, the network generates hidden layers layer by layer until accuracy reaches the threshold. Then, the network uses a pruning algorithm based on Hebb’s rule or Pearson’s correlation for adaptation in the pruning step. In addition, we combine genetic algorithm to optimize DNS (GA-DNS). Experimental results show that compared with traditional neural network topology optimization algorithms, GA-DNS can generate neural networks with higher construction efficiency, lower structure complexity, and higher classification accuracy.

Modeling of heterojunction bipolar transistors based on novel Wiener‐type dynamic neural network

A novel Wiener-type dynamic neural network modeling method for heterojunction bipolar transistors (HBTs) is proposed in this article. The model structure of the novel Wiener-type dynamic neural network applicable to HBT modeling is determined by the vector fitting. For the novel Wiener-type dynamic neural network structure, we also proposed a training algorithm applicable to this novel structure to improve its accuracy. We considered the transmission characteristics of the HBT and use the H-parameter to replace the Y-parameter for small-signal analysis. The DC and small-signal model formulas for HBT are also derived. The proposed novel Wiener-type dynamic neural network modeling method does not require detailed parameter information and the internal design structure of the HBT and can model any type of HBT as long as accurate training data are available. In this paper, an application example on modeling a 3 × 30 μ m 2 InGaP/GaAs HBT is presented, and a combined DC and small-signal model is established to demonstrate the accuracy and feasibility of the proposed novel Wiener-type dynamic neural network modeling method. The modeling method can also be used for computer-aided design, and the established models can be more easily applied to circuit simulation.

Nonlinear Model Predictive Control Using Feedback Linearization for a Pressurized Water Nuclear Power Plant

The present work aims to introduce a nonlinear control scheme that combines intelligent feedback linearization (FBL) and a model predictive control (MPC) for a pressurized water reactor (PWR). The nonlinear plant model that is considered in this study is described by the first-principles approach, and it consists of 38 state variables. First, system identification using a dynamic neural network (DNN) structure is performed to obtain a standard affine nonlinear system. The quasi-Newton algorithm is employed to find the best DNN model. Then, an FBL is formulated to address the nonlinearity of the DNN model. An MPC controller is developed based on the FBL system to improve the system performance. The designed controller is compared with a linear MPC controller that is based on state-space models to evaluate the performance of the proposed controller. The proposed approach improves the load-following operation and offers better disturbance rejection capability than the conventional MPC. In addition, numerical measures are employed to compare and analyze the performances of the two control strategies.

Rule Insertion Technique for Dynamic Cell Structure Neural Network

This paper discusses the idea of capturing an expert’s knowledge in the form of human understandable rules and then inserting these rules into a dynamic cell structure (DCS) neural network. The DCS is a form of self-organizing map that can be used for many purposes, including classification and prediction. This particular neural network is considered to be a topology preserving network that starts with no pre-structure, but assumes a structure once trained. The DCS has been used in mission and safety-critical applications, including adaptive flight control and health-monitoring in aerial vehicles. The approach is to insert expert knowledge into the DCS before training. Rules are translated into a pre-structure and then training data are presented. This idea has been demonstrated using the well-known Iris data set and it has been shown that inserting the pre-structure results in better accuracy with the same training.

Performance of Cellular Neural Network Based Channel Equalizers

Abstract—In this paper, a popular dynamic neural network structure called Cellular Neural Network (CNN) is employed as a channel equalizer in digital communications. It is shown that, this nonlinear system is capable of suppressing the effect of intersymbol interference (ISI) and the noise at the channel. The architecture is a small-scaled, simple neural network containing only 25 neurons (cells) with a neighborhood of r = 2 , thus including only 51 weight coefficients. Furthermore, a special technique called repetitive codes in equalization process is also applied to the mentioned CNN based system to show that the two-dimensional structure of CNN is capable of processing such signals, where performance improvement is observed. Simulations are carried out to compare the proposed structures with minimum mean square error (MMSE) and multilayer perceptron (MLP) based equalizers.

A Wiener-Type Dynamic Neural Network Approach to the Modeling of Nonlinear Microwave Devices

A new Wiener-type dynamic neural network (DNN) approach for nonlinear device modeling is proposed in this paper. The proposed analytical formulation of Wiener-type DNN structure consists of a cascade of a simplified linear dynamic part and a nonlinear static part. The simplified linear dynamic equations are obtained by vector fitting enhancing the efficiency of the model. A new formulation and new sensitivity analysis technique is derived to train the proposed Wiener-type DNN with dc, small- and large-signal data. A new gradient-based training algorithm is also formulated to speed up the training of proposed Wiener-type DNN model. The proposed Wiener-type DNN model can be trained to be accurate relative to device data. Furthermore, the proposed Wiener-type DNN provides enhanced convergence properties over existing neural network approaches such as time delay neural network (TDNN) and TDNN with extrapolation. Application examples on modeling GaAs metal-semiconductor-field-effect transistor, GaAs high-electron mobility transistor (HEMT), and a real <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 \times 50$ </tex-math></inline-formula> gatewidth GaAs pseudomorphic HEMT are presented. The use of Wiener-type DNN model in harmonic balance simulations demonstrates that the Wiener-type DNN is a robust method for developing models for different types of microwave devices. These models could be implemented into circuit simulators conveniently.

Dynamic Neural Network-Based Output Feedback Tracking Control for Uncertain Nonlinear Systems

A dynamic neural network (DNN) observer-based output feedback controller for uncertain nonlinear systems with bounded disturbances is developed. The DNN-based observer works in conjunction with a dynamic filter for state estimation using only output measurements during online operation. A sliding mode term is included in the DNN structure to robustly account for exogenous disturbances and reconstruction errors. Weight update laws for the DNN, based on estimation errors, tracking errors, and the filter output are developed, which guarantee asymptotic regulation of the state estimation error. A combination of a DNN feedforward term, along with the estimated state feedback and sliding mode terms yield an asymptotic tracking result. The developed output feedback (OFB) method yields asymptotic tracking and asymptotic estimation of unmeasurable states for a class of uncertain nonlinear systems with bounded disturbances. A two-link robot manipulator is used to investigate the performance of the proposed control approach.

Кластеризация пользователей по голосу с помощью улучшенных самоорганизующихся растущих нейронных сетей

The paper describes the use of an unsupervised learning method based on self-organizing incremental neural networks for the problem of speaker clustering. I uses a set of mel-frequency cepstral coefficients as a user model. This set is obtained by applying a special filter to the sound signal frequency, which was transferred into the mel-frequency scale (mel is an abbreviation of “melody”). The main difference of this work is the consideration of the dynamics of mel-frequency cepstral coefficients changing, which also contains information about the user. The possibility of new unique users emergence in the system while operating makes it impossible to use the major-ity of neural network classes, because learning on a new data set will lead to malfunction, “forgetting” of prior learn-ing. Neural networks for on-line learning impose a limit on the maximum number of clusters, that is unknown for this problem, and, in general, they require a priori knowledge of the input data (to establish thresholds, etc.) that is difficult to achieve in practice. Self-organizing incremental neural networks allow lifetime learning, that means learning during the operation stage, and do not require any a priori knowledge about the users or their quantity. A dynamic neural net-work structure makes it possible to create an unlimited number of new clusters for new previously unregistered users. Thus, this method allows building a flexible speaker clustering system that adapts itself to the changing input data.

Dynamic neural networks with hybrid structures for nonlinear system identification

Dynamic neural networks (DNNs) have important properties that make them convenient to be used together with nonlinear control approaches based on state space models and differential geometry, such as feedback linearisation. However the mapping capability of DNNs are quite limited due to their fixed structure, that is, the number of layers and the number of hidden units. An example shown in this paper has demonstrated this limitation of DNNs. The development of novel DNN structures, which has good mapping capability, is a relevant challenge being addressed in this paper. Although the structure is changed minorly only, the mapping capability of the new designed DNN in this paper has been improved dramatically. Previous work [J. Deng et al., 2005. The dynamic neural network of a hybrid structure for nonlinear system identification. In: 16th IFAC World Congress, Prague.] presents a new dynamic neural network structure which is suitable for the identification of highly nonlinear systems, which needs the outputs from the real system for training and operation. This paper presents a hybrid dynamic neural network structure which presents a similar idea of serial–parallel hybrid structure, but it uses an output from another neural network for training and operation classified as a serial–parallel model. This type of DNNs does not require the output of the plant to be used as an input to the model. This neural network has the advantages of good mapping capabilities and flexibilities in training complicated systems, compared to the existed DNNs. A theoretical proof showing how this hybrid dynamic neural network can approximate finite trajectories of general nonlinear dynamic systems is given. To illustrate the capabilities of the new structure, neural networks are trained to identify a real nonlinear 3D crane system.

Adaptive dissolved oxygen control based on dynamic structure neural network

Activated sludge wastewater treatment processes (WWTPs) are difficult to control because of their complex nonlinear behavior. In this paper, an adaptive controller based on a dynamic structure neural network (ACDSNN) is proposed to control the dissolved oxygen (DO) concentration in a wastewater treatment process (WWTP). The proposed ACDSNN incorporates a structure variable feedforward neural network (FNN), where the FNN can determine its structure on-line automatically. The structure of the FNN is adapted to cope with changes in the operating characteristics, while the weight parameters are updated to improve the accuracy of the controller. A particularly strong feature of this method is that the control accuracy can be maintained during adaptation, and therefore the control performance will not be degraded when the character of the model changes. The performance of the proposed ACDSNN is illustrated with numerical simulations and is compared with the fixed structure fuzzy and FNN approaches; it provides an effective solution to the control of the DO concentration in a WWTP.

Intrusion Detection Based on Rule Extraction from Dynamic Cell Structure Neural Networks

Knowledge embedded within artificial neural networks (ANNs) is distributed over the connections and weights of neurons. So, the user considers ANN as a black box system. There are many researches investigating the area of rule extraction by ANNs. In this paper, a dynamic cell structure (DCS) neural network and a modified version of LERX algorithm are used for rule extraction. On the other hand, intrusion detection system (IDS) is known as a critical technology to secure computer networks. So, the proposed algorithm is used to develop IDS and classify the patterns of intrusion. To compare the performance of the proposed system with other machine learning algorithms, multi-layer perceptron (MLP) with output weight optimization-hidden weight optimization (OWO-HWO) training algorithm is employed with selected inputs based on the results of a feature relevance analysis. Empirical results show the superior performance of the IDS based on rule extraction from DCS, in recognizing hard-detectable attack categories, e.g. userto-root (U2R) and also offering competitive false alarm rate (FAR). Although, MLP with 25 selected input features, instead of 41 standard features introduced by knowledge discovery and data mining group (KDD), performs better in terms of detection rate (DR) and cost per example (CPE) when compared with some other machine learning methods, as well.

Dynamic structure adaptive neural fuzzy control for MIMO uncertain nonlinear systems

This paper proposes a novel dynamic structure neural fuzzy network (DSNFN) to address the adaptive tracking problems of multiple-input–multiple-output (MIMO) uncertain nonlinear systems. The proposed control scheme uses a four-layer neural fuzzy network (NFN) to estimate system uncertainties online. The main feature of this DSNFN is that it can either increase or decrease the number of fuzzy rules over time based on tracking errors. Projection-type adaptation laws for the network parameters are derived from the Lyapunov synthesis approach to ensure network convergence and stable control. A hybrid control scheme that combines the sliding-mode control and the adaptive bound estimation control with different weights improves system performance by suppressing the influence of external disturbances and approximation errors. As the employment of the DSNFN, high-quality tracking performance could be achieved in the system. Furthermore, the trained network avoids the problems of overfitting and underfitting. Simulations performed on a two-link robot manipulator demonstrate the effectiveness of the proposed control scheme.

Infrared thermometer sensor dynamic error compensation using Hammerstein neural network

A novel dynamic neural network structure based on Hammerstein model is proposed and applied to dynamic error compensation for infrared thermometer sensor in this paper. First, the devices of dynamic calibration for infrared thermometer sensor are designed and the calibration experiments with continuous excitation are carried out. Then, the non-linear inverse system of the sensor dynamic compensator is expressed by a non-linear static subunit followed by a linear dynamic subunit—Hammerstein model. A novel neural network structure is designed, the weights in which are corresponding with the parameters of Hammerstein model. Finally, The iterative algorithm is derived, through which the non-linear static and linear dynamic subunit in Hammerstein model can be optimised and the coefficients of the dynamic compensator are gotten. The dynamic calibration data of the uIRt/c sensor are used to test and the experiment results show that the stabilizing time of the sensor is reduced less than 6 ms from 26 ms and the dynamic characteristic is obviously improved after compensation.

Short-Term Prediction of Traffic Rate Interval Router Using Hybrid Training of Dynamic Synapse Neural Network Structure

Short-Term Prediction of Traffic Rate Interval Router Using Hybrid Training of Dynamic Synapse Neural Network Structure

Adaptive control of uncertain chaotic systems with time delays using dynamic structure neural network

A kind of control methods is surveyed to deal with a class of nonlinear systems with uncertain time delay. By combining the Lyapunov-Krasovskii function and Lyapunov function, the time-delay compensator with adjustable parameters is presented to make the control input independent of the multiple time delays. At the same time, the dynamic structure adaptive neural network is introduced to eliminate the uncertainties in the chaotic system, which approximates the function with better relationship between the calculation rate and structure complexity by increasing hidden units when the tracking error is beyond the allowable bound. The demonstration with a given Duffing chaotic system shows the presented control method is effective.

STABLE ADAPTIVE CONTROL UNDER UNMEASURABLE PLANT STATES

A dynamic neural network based algorithm for learning control of an unknown nonlinear continuous-time multiple-input-multiple-output plant with unmeasurable states is proposed in this paper. A new dynamic neural network structure is utilised to model the unknown plant dynamics through modelling the input-output mapping. A feedback linearization is applied to design controller for the neural model and the neural states are used as a source of precious information about current plant dynamics. A gradient based update of the weights is performed at discrete time instants over a moving measurement window in order to reduce the model output – real output mismatch. The learning controller is applied to a double link robot arm. Stability of the system is analysed through ultimate boundedness of all signals.

Robust incremental growing multi-experts network

Most supervised neural networks are trained by minimizing the mean square error (MSE) of the training set. In the presence of outliers, the resulting neural network model can differ significantly from the underlying model that generates the data. This paper outlines two robust learning methods for a dynamic structure neural network called incremental growing multi-experts network (IGMN). It is convincingly shown by simulation that by using a scaled robust objective function instead of the least squares function, the influence of the outliers in the training data can be completely eliminated. The network generates a much better approximation in the neighborhood of outliers. Thus, the two proposed robust learning methods namely robust least mean squares (RLMSs) and least mean log squares (LMLSs) are insensitive to the presence of outliers unlike the least mean squares (LMSs) cost function. Moreover, various types of supervised learning algorithms can easily adopt LMLS, which is a parameter-free method.

A NOVEL DYNAMIC NEURAL NETWORK STRUCTURE FOR NONLINEAR SYSTEM IDENTIFICATION

Dynamic neural networks are often used for nonlinear system identification. This paper presents a novel series-parallel dynamic neural network structure which is suitable for nonlinear system identification. A theoretical proof is given showing that this type of dynamic neural network is able to approximate finite trajectories of nonlinear dynamical systems. Also, this neural network is trained to identify a practical nonlinear 3D crane system.