Elman Recurrent Network Research Articles

Entropies based detection of epileptic seizures with artificial neural network classifiers

Computer assisted automated detection is highly inevitable for recognizing neurological disorders, as it involves continuous monitoring of Electroencephalogram (EEG) signal. Being a non-stationary signal, suitable analysis is essential for EEG to differentiate the normal EEG and epileptic seizures. This paper highlights the importance of entropy based features to recognize the normal EEGs, and ictal as well as interictal epileptic seizures. Three non-linear features, such as, wavelet entropy, sample entropy, and spectral entropy are used to extract quantitative entropy features from the given EEG time using two neural network models, namely, recurrent Elman network (REN) and radial basis network (RBN) are then incorporated for the purpose of classification. The stationary properties of the EEG are exploited by estimating entropies at various time frames and the performance of the proposed scheme is evaluated using specificity, sensitivity and classification accuracy. From the experimental results, it is found that among the different entropies applied, the wavelet entropy features with recurrent Elman networks yields 99.75% and 94.5% accuracy for detecting normal vs. epileptic seizures and interictal focal seizures respectively.

Elman Network in Kappa Number Prediction

AbstractVariations in the pulp quality are mainly due to variations in the quality of wood chips used for pulping. Moisture content of chips, chip size and chip composition vary, and the measurements to reflect these variations do not always exist. Kappa number is the usual way of describing the pulp quality and it measures the residual lignin content in the pulp. In batch digesting, the main difficulty in maintaining consistent Kappa number is the lack of on-line Kappa number measurement. The most common way is to use the calculated H-factor to define the cooking end-point. Kappa number cannot be measured on-line in batch digesters, but there are many approaches to model, or predict, it from the existing measurement, and use the predicted Kappa to complement the H-factor control. Both mechanistic and data-based model are in use, but due to the complexity of fundamental models, current industrial practices rely almost exclusively on simple empirical or semi-empirical models. Several intelligent methods (neural networks and fuzzy logic) together with advanced controls have found applications also on this arena. This paper concerns with using recurrent Elman network together with alkali measurement and H-factor in Kappa number modelling.

A transmission line fault locator based on Elman recurrent networks

In this paper, a transmission line fault location model which is based on an Elman recurrent network (ERN) has been presented for balanced and unbalanced short circuit faults. All fault situations with different inception times are implemented on a 380-kV prototype power system. Wavelet transform (WT) is used for selecting distinctive features about the faulty signals. The system has the advantages of utilizing single-end measurements, using both voltage and current signals. ERN is able to determine the fault location occurred on transmission line rapidly and correctly as an important alternative to standard feedforward back propagation networks (FFNs) and radial basis functions (RBFs) neural networks.

Modelling and Predictive Control of a Multivariable Process Using Recurrent Neural Networks

Industrial processes are naturally multivariable in nature, which exhibit non-linear behaviour and complex dynamic properties. The multivariable four-tank system has attracted recent attention as it exhibits elegantly complex dynamics, which includes interaction, transmission zero, and non-minimum phase characteristics that emerge from a simple cascade of tanks. In this paper, a neural model predictive controller (NMPC) is compared with a decentralized PI controller designed for a four-tank system. The approach adopts the approximation capabilities of a recurrent Elman network for modelling purposes. The learning process is performed with a modification of a back propagation through time (BPTT) algorithm. The algorithms are simulated using MATLAB and the performances of NMPC and decentralized PI control are compared based on integral square error (ISE) values for set point tracking and load changes. The performance comparison shows that NMPC is a suitable controller for four-tank system.

Therapeutic Drug Monitoring of Kidney Transplant Recipients Using Profiled Support Vector Machines

This paper proposes a twofold approach for therapeutic drug monitoring (TDM) of kidney recipients using support vector machines (SVMs), for both predicting and detecting Cyclosporine A (CyA) blood concentrations. The final goal is to build useful, robust, and ultimately understandable models for individualizing the dosage of CyA. We compare SVMs with several neural network models, such as the multilayer perceptron (MLP), the Elman recurrent network, finite/infinite impulse response networks, and neural network ARMAX approaches. In addition, we present a profile-dependent SVM (PD-SVM), which incorporates a priori knowledge in both tasks. Models are compared numerically, statistically, and in the presence of additive noise. Data from 57 renal allograft recipients were used to develop the models. Patients followed a standard triple therapy, and CyA trough concentration was the dependent variable. The best results for the CyA blood concentration prediction were obtained using the PD-SVM (mean error of 0.36 ng/mL and root-mean-square error of 52.01 ng/mL in the validation set) and appeared to be more robust in the presence of additive noise. The proposed PD-SVM improved results from the standard SVM and MLP, specially significant (both numerical and statistically) in the one-against-all scheme. Finally, some clinical conclusions were obtained from sensitivity rankings of the models and distribution of support vectors. We conclude that the PD-SVM approach produces more accurate and robust models than do neural networks. Finally, a software tool for aiding medical decision-making including the prediction models is presented

Resolution of overlapping signals in spectrometry using a wavelet packet transform and an Elman recurrent neural network

A novel method named a wavelet packet transform based Elman recurrent neural network (WPTERNN) was proposed for the simultaneous UV-visible spectrometric determination of Cu(II), Cd(II) and Zn(II). This method combined wavelet packet denoising with an Elman recurrent neural network. A wavelet packet transform was applied to perform data compression, to extract relevant information, and to eliminate noise and collinearity. An Elman recurrent network was applied for nonlinear multivariate calibration. In this case, using trials, the kind of wavelet function, the decomposition level, and the number of hidden nodes for the WPTERNN method were selected as Daubechies 14, 3, and 8, respectively. A program (PWPTERNN) was designed that could perform the simultaneous determination of Cu(II), Cd(II) and Zn(II). The relative standard errors of prediction (RSEP) obtained for all components using WPTERNN, a Elman recurrent neural network (ERNN), partial least squares (PLS), principal component regression (PCR), Fourier transform based PCR (FTPCR), and multivariate linear regression (MLR) were compared. Experimental results demonstrated that the WPTERRN method was successful even where there was severe overlap of spectra. The results obtained from an additional test case also demonstrated that the WPTERNN method performed very well.

Prediction of retail sales of footwear using feedforward and recurrent neural networks

Fluctuation of sales over time is one of the major problems faced by most of the industries. To alleviate this problem management tries to base their plans on forecast of sales pattern, which are mostly adhoc and rarely provides solid foundation for the plans. This study makes an attempt to solve this problem by taking a neural network approach, at the process of sales of footwear, and arriving at an optimum neural network model. The algorithms used for developing such model through neural network are both feedforward and recurrent Elman network. The data used in this work are the weekly sales of footwear and the information about the seasonality of sales process. While solving the problem, the focus is on forecasting of weekly retail sales as per the requirement of management. This work would reduce the uncertainty existing in the short-term/middle term planning of sales and distribution logistics of footwear over different time horizons across the entire supply chain of footwear business.

Identification and Control of Bioreactor using Recurrent Networks

A nonlinear model predictive control (NMPC) strategy based on recurrent neural networks (RNN) is proposed for a single‐input single‐output system (SISO) to control the uncertain nonlinear process. The automatic configuration and modeling of the networks is carried out using a recurrent Elman network using back propagation through time (BPTT) with MATLAB. Identification of the process is performed with a RNN based nonlinear autoregressive with exogenous input (NARX) model and the incorporation of the developed model in the formulation of NMPC is presented. Further, the results of the NMPC is compared with a well tuned IMC based PI controller, which shows a better performance based on the rise time and settling time of the proposed NMPC scheme for the control of an unstable bioreactor.

Resolve of overlapping voltammetric signals in using a wavelet packet transform based Elman recurrent neural network

A novel method named wavelet packet transform based Elman recurrent neural network (WPTERNN) was applied to differential pulse voltammetric techniques for simultaneous determination of Ni(II), Zn(II) and Co(II) by combining wavelet packet denoising with Elman recurrent neural network (ERNN). The performances of the WPT methods were compared with seven other filtering techniques in terms of root mean square deviations between reconstructed and original mean voltammogram. The visual inspection of filtering effects was supplied by figure. Wavelet packet representations of signals provided a local time–frequency description, thus in the wavelet domain, the quality of the noise removal can be improved. Elman recurrent network was applied for non-linear multivariate calibration and together with the whole voltammogram to improve predictive ability. In this case, by trials wavelet function, decomposition level and numbers of hidden nodes for WPTERNN method were selected as Daubechies 4, 6 and 8, respectively. A program PWPTERNN was designed to perform simultaneous determination of Ni(II), Zn(II) and Co(II). The relative standard errors of prediction (RSEP) for all components with WPTERNN, ERNN, PLS, PCR, TTFA and MLR were 9.53, 9.82, 12.3, 17.0, 16.7 and 1.46 × 10 5%, respectively. Experimental results demonstrated that the WPTERRN method had the best performance among the six methods and the two ANN methods had the clear superiority over the three factor-based method.

IDENTIFIABILITY OF RECURRENT NEURAL NETWORKS

We examine the identifiability of a nonlinear state space system under general assumptions. The discrete time evolution of the state is generated by a recurrent Elman network. For a large set of Elman networks we determine the class of observationally equivalent minimal systems, i.e., minimal systems that exhibit the same input-output behavior.The authors are grateful for discussions with M. Deistler and D. Bauer. A.A. Al-Falou was funded by the ERNSI network within the European Union program Training and Mobility of Researchers (TMR). D. Trummer was funded by the FWF (Austrian Science Fund). We thank an anonymous referee for helpful comments and suggestions.

Auroral electrojet predictions with dynamic neural networks

Neural networks with internal feedback from the hidden nodes to the input [Elman, 1990] are developed for prediction of the auroral electrojet index AE from solar wind data. Unlike linear and nonlinear autoregressive moving‐aver age (ARMA) models, such networks are free to develop their own internal representation of the recurrent state variables. Further, they do not incorporate an explicit memory for past states; the memory is implicitly given by the feedback structure of the networks. It is shown that an Elman recurrent network can predict around 70% of the observed AE variance using a single sample of solar wind density, velocity, and magnetic field as input. A neural network with identical solar wind input, but without a feedback mechanism, only predicts around 45% of the AE variance. It is also shown that four recurrent state variables are optimal: the use of more than four hidden nodes does not improve the predictions, but with less than that the prediction accuracy drops. This provides an indication that the global‐scale auroral electrojet dynamics can be characterized by a small number of degrees of freedom.

Implementation and laboratory test results of an Elman network-based transmission line directional relay

A transmission line fault direction identification module based on an Elman recurrent network has been implemented on a DSP board and its behavior is investigated on a physical power system model. Details of implementation and the experimental studies are given and analysed in the paper. Studies show that the proposed approach is able to identify the direction of a fault on a transmission line rapidly and correctly. It is suitable for realizing a fast transmission line directional comparison protection scheme.

High speed transmission system directional protection using an Elman network

Detection of the direction of a fault on a transmission line is essential to the proper performance of a power system. It would be desirable to develop a high speed and accurate approach to determine the fault direction for different power system conditions. To classify forward and backward faults on a given line, a neural network's abilities in pattern recognition and classification could be considered as a solution. To demonstrate the applicability of this solution, neural network technique is employed and a novel Elman recurrent network is designed and trained. Details of the design procedure and the results of performance studies with the proposed network are given and analysed in the paper. System simulation studies show that the proposed approach is able to detect the direction of a fault on a transmission line rapidly and correctly. It is suitable to realize a very fast transmission line directional comparison protection scheme.

K-complex detection using multi-layer perceptrons and recurrent networks

The feasibility of using a multi-layer perceptron and Elman's recurrent network for the detection of specific waveforms (K-complexes) in electroencephalograms (EEGs), regardless of their location in the signal segment, is explored. Experiments with simulated and actual EEG data were performed. In case of the perceptron, the input consisted of the magnitude and/or phase values obtained from 10-s signal intervals, whereas the recurrent net operated on the digitized data samples directly. It was found that both nets performed well on the simulated data, but not on the actual EEG data. The reasons for the failure of both nets are discussed.

Performance of Neural Networks in Managerial Forecasting

AbstractThis paper investigates the effectiveness of a multi‐layered neural network as a tool for forecasting in a managerial time‐series setting. To handle noisy data of limited length we adopted two different neural network approaches. First, the neural network is used as a pattern classifier to automate the ARMA model‐identification process. We tested the performance of multi‐layered neural networks with two statistical feature extractors: ACF/PACF and ESACF. We found that ESACF provides better performance, although the noise in ESACF patterns still caused the classification performance to deteriorate. Therefore we adopted the noise‐filtering network as a preprocessor to the pattern‐classification network, and were able to achieve an average of about 89% classification accuracy. Second, the neural network is used as a tool for function approximation and prediction. To alleviate the overfitting problem we adopted the structure of minimal networks and recurrent networks. The experiment with three real‐world time series showed that the prediction by Elman's recurrent network outperformed those by the ARMA model and other structures of multi‐layered neural networks, especially when the time series contained significant noise.