Elman Recurrent Neural Network Research Articles

ПІДХОДИ ДО СТВОРЕННЯ ПРОГРАМНОГО АГЕНТА НА ОСНОВІ МЕТАЕВРИСТИК І ШТУЧНИХ НЕЙРОННИХ МЕРЕЖ

In this article, has been developed a software agent based on meta-heuristics and artificial neural networks. The analysis of existing classes of agents and the selected reactive agent with internal state, which is well suited for partially observable, dynamic and non-episodic media, was carried out, and this agent has an internal state that preserves the state of the environment, obtained on the basis of the history of acts of perception, in the form of structured data. Were proposed approaches to create an agent based on meta-heuristics and an agent based on an artificial neural network. The development of reactive agents with internal state, based on the PSO (particle swarm optimization) metaheuristics, which are related to individual particles and to a whole swarm and interact by messages was proposed. Also, has been proposed an approach to the creation of a reactive agent with an internal state based on the Elman recurrent neural network. The agent-based approach allows combining different areas of artificial intelligence, digital signal processing, mathematical modeling, and game theory. The proposed agents were implemented using the JADE (Java Agent Development Framework) toolkit, which is one of the most popular tools for the creation of agent systems. A numerical study was made to determine the parameters of the swarm PSO metaheuristics and the Elman recurrent neural network. As a purpose function, the Rastrigin test function has been used. The number of visits to the website of DonNTU was used as an input sample for the Elman network. The minimum average square error forecast was the criterion for choosing the structure of a network model. 10 hiding neurons were used to predict the number of visits to the website page, since, with increasing of hidden neurons number, the change in the error value is small. To determine the number of particles in the swarm, a series of experiments was conducted, the results of which are presented by graphs. The proposed approaches can be used in intelligent computer systems.

Prediction of summer rainfall over the source region of the Blue Nile by using teleconnections based on sea surface temperatures

Summer rainfall (June–September) accounts for about 80% of the annual rainfall in the Lake Tana basin of Ethiopia, the source region of the Blue Nile River. Prediction of summer rainfall would be valuable for managing the region’s water resources and agricultural operations. This study investigated the influence of sea surface temperature (SST) as a predictor of summer rainfall in the basin by applying cross-correlation analysis between summer rainfall and climatic indices and SST in various oceanic regions from 1985 to 2015. Summer rainfall showed a strong negative correlation (− 0.619) with the Pacific Decadal Oscillation index at short time lags. Further analysis identified strong teleconnections (|r| ≥ 0.5) between SSTs in specific parts of the Pacific Ocean and summer rainfall in the Lake Tana basin, raising the possibility of predicting summer rainfall from Pacific SSTs with a lead time of 4 to 5 months. Average SSTs of an area near the Philippines and an area west of North America were positively correlated (0.629) and negatively correlated (− 0.538), respectively, with summer rainfall in the Lake Tana basin. Predictions of summer rainfall from these teleconnected SSTs by an Elman recurrent neural network model were encouraging, indicating a strong correlation (r > 0.77) between the observed and predicted summer rainfall. Our results suggest that neural network techniques may have sufficient skill to support seasonal rainfall predictions over the Lake Tana basin.

Integration of a Parsimonious Hydrological Model with Recurrent Neural Networks for Improved Streamflow Forecasting

This study applied a GR4J model in the Xiangjiang and Qujiang River basins for rainfall-runoff simulation. Four recurrent neural networks (RNNs)—the Elman recurrent neural network (ERNN), echo state network (ESN), nonlinear autoregressive exogenous inputs neural network (NARX), and long short-term memory (LSTM) network—were applied in predicting discharges. The performances of models were compared and assessed, and the best two RNNs were selected and integrated with the lumped hydrological model GR4J to forecast the discharges; meanwhile, uncertainties of the simulated discharges were estimated. The generalized likelihood uncertainty estimation method was applied to quantify the uncertainties. The results show that the LSTM and NARX better captured the time-series dynamics than the other RNNs. The hybrid models improved the prediction of high, median, and low flows, particularly in reducing the bias of underestimation of high flows in the Xiangjiang River basin. The hybrid models reduced the uncertainty intervals by more than 50% for median and low flows, and increased the cover ratios for observations. The integration of a hydrological model with a recurrent neural network considering long-term dependencies is recommended in discharge forecasting.

Application of Neural Network Based Data Security in to LFC of a Two Area Power System

The paper is concentrated with the study of design of a data security system based on neural networks. This data security method added Load-Frequency Control of reheat interconnected two area power systems problems with non-linearity. The neural network control is incorporated to load-frequency control in power systems. Elman Recurrent neural network is involves forecasting controller and system’s output. The system was simulated and the frequency variations in area 1 and area 2 and tie-line power variations for 1% step-load disturbance in area 1 were obtained. The comparison due to frequency variations and tie-line power deviations for the two area interconnected thermal power system. The result data of different keys were taken as test data, encrypted, decrypted and compared with the original data. The results have ensured better its advantages over conventional techniques.

Temporal prediction of carbon monoxide using the Elman Recurrent Neural Network

Peatland fires in Indonesia are considered as regional disasters which occur periodically. These negative impacts, especially on our health, continue to threaten the society across the region. The objective of this study was to create a temporal model for predicting the pollutant concentration from peatland fires using the Elman Recurrent Neural Network (ERNN) and training by gathering data from fires which have been occurred recently in Sumatera, Indonesia. The data describing the haze from the peatland fires were generated using the HYSPLIT model with the input of hotspot sequences and meteorological data from NOAA. The stages of the model development consisted of data pre-processing, pollutant concentrations generating using HYSPLIT, pollutant concentration analysis, network architecture formation, weight determination, model training, and the prediction of the model evaluation. Experimental results indicated that the calculation of the ISPU (standard air pollution index) using the GDAS data of 20.5 g / m3 obtained an ISPU value of 221. This value indicated that the air in the South Sumatera Province was very unhealthy. Similar to the calculation of ISPU using the WRF-Chem data of 26 g / m3 obtained an ISPU value of 253. This value indicated that air in the South Sumatera Province was very unhealthy.

Identification of the Dynamics of a Moving Object with the Use of Neural Networks

A method for identification of the dynamics of a quadrotor-type vehicle is proposed. The method is based on the Elman recurrent neural network, which corresponds to the canonical form of a dynamic system in the space of states and does not require structural correction. The results of a numerical experiment reveal the convergence of the network learning algorithm with the use of an extended Kalman filter.

On extreme learning machines in sequential and time series prediction: A non-iterative and approximate training algorithm for recurrent neural networks

Recurrent neural networks (RNN) are a type of artificial neural networks (ANN) that have been successfully applied to many problems in artificial intelligence. However, they are expensive to train since the number of learned weights grows exponentially with the number of hidden neurons. Non-iterative training algorithms have been proposed to reduce the training time, mainly on feedforward ANN. In this work, the application of non-iterative randomized training algorithms to various RNN architectures, including Elman RNN, fully connected RNN, and long short-term memory (LSTM), are investigated. The mathematical formulation and theoretical computational complexity of the proposed algorithms are presented. Finally, their performance is empirically compared to other iterative RNN training algorithms on time series prediction and sequential decision-making problems. Non-iteratively-trained RNN architectures showed promising results as significant training speedup of up to 99%, and improved repeatability were achieved compared to backpropagation-trained RNN. Although the decrease in prediction accuracy was found to be statistically significant based on Friedman and ANOVA testing, some applications like real-time embedded systems can tolerate and make use of that.

Determination of endometrial carcinoma with gene expression based on optimized Elman neural network

Endometrial carcinoma is a life-threatening disease that causes serious damage to the women’s health. This paper discusses classifications of 87 endometrial samples with gene expressions that are cancerous or cancer-free. Every sample has 5 indicators. For every indicator, the corresponding genes of the missing data are deleted and the signal noise ratios (SNRs) are calculated to filter the irrelevant genes. Then the obtained new samples use the principle component analysis to decrease the dimensions. Finally 10 random samples are selected to be the testing samples for classification. Thus the classification accuracy rate is given for every indicator. Based on cancer related to 5 indicators, the combination of the 5 indicators is used to classify to make new 87 endometrial samples as cancerous or cancer-free. We repeatedly process these new samples by deleting the missing data, filtering the irrelevant genes with SNRs, and decreasing the dimensions with PCA, an obtain the new data. The proposed method is that the particle swarm algorithm (PSO) and the grey wolf optimizer (GWO) is combined to optimize the parameters of Elman recurrent neural network (ERNN), written as PSOGWO-ERNN. The results show that PSOGWO-ERNN is superior to the single ERNN, ERNN optimized by PSO or GWO (PSO-ERNN or GWO-ERNN), and the classification accuracy rate of PSOGWO-ERNN reaches 88.8506%. The results also show that the neural networks optimized by some swarm intelligence algorithms are more useful for classification.

Comparison of ARCH / GARCH model and Elman Recurrent Neural Network on data return of closing price stock

Time series data can be modeled using the Autoregressive Intergrated Moving Average (ARIMA) model. ARIMA model requires the assumption of homoscedasticity. But on financial data, like stock data, has a high fluctuation so that the variant is heteroscedastic. To solve this problem, Autoregressive Conditional Heteroscedasticity (ARCH) or Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model is used. The ARCH process introduced in Engle allows the conditional variance to change over time as a function of past errors leaving the unconditional variance constant. The ARCH model was generalized to the Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model and introduced in Bollerslev allowing for a much more flexible lag structure.Another model that can be used is the Elman Recurrent Neural Network model (ERNN) which is a nonlinear model. The purpose of this research is to determine the best model between ARCH / GARCH model and ERNN model based on MSE value. The data used is the return of daily closing price of BRI stocks period of January 1st, 2015 to 17th February 2017. In this study, the best model obtained is GARCH (1,1) which has the smallest MSE value.

New Algorithm to Handle Routing with Load Balancing in Wireless Networks Using EERNN

In this paper, static wireless network load balance algorithm is proposed, that use only optimal paths from point-to-point to achieve good load balance. This algorithm based on Enhanced version of Elman Recurrent Neural Network (EERNN) to make load balance decision depended on two metrics (traffic load on node and probability of link failure). This algorithm make good work in terms of both metrics simultaneously. Also, this algorithm use only local information. The execution of the proposed algorithm is compared with ERNN based on traffic load on node and probability of link failure.
 
 © 2018 JASET, International Scholars and Researchers Association
 Author Biographies
 Lubna Najah Rasoul
 Mazaya University Collage
 Wijdan Rasheed Abd Al-Hussain
 University of Thi-Qar

A hybrid elman neural network predictor for time series prediction

Artificial Neural Networks have become popular in the world of prediction and forecasting due to their nonlinear nonparametric adaptive-learning property. They become an important tool in data analysis and data mining applications. Elman neural network due to its recurrent nature and dynamic processing capabilities can perform the prediction process with a good range of accuracy. In this paper an Elman recurrent Neural Network is hybridised with a time delay called a tap delay line for time series prediction process to improve its performance. The Elman neural network with the time delay inputs is trained tested and validated using the solar sun spot time series data that contains the monthly mean sunspot numbers for a 240 year period having 2899 data values. The results confirm that the proposed Elman network hybridised with time delay inputs can predict the time series with more accurately and effectively than the existing methods.

The Prediction and Analysis of the Power Energy Time Series by Using the Elman Recurrent Neural Network

The Prediction and Analysis of the Power Energy Time Series by Using the Elman Recurrent Neural Network

Application of the Hybrid Artificial Neural Network Coupled with Rolling Mechanism and Grey Model Algorithms for Streamflow Forecasting Over Multiple Time Horizons

Streamflow forecasting is paramount process in water and flood management, determination of river water flow potentials, environmental flow analysis, agricultural practices and hydro-power generation. However, the dynamicity, stochasticity and inherent complexities present in the temporal evolution of streamflow could hinder the accurate and reliable forecasting of this important hydrological parameter. In this study, the uncertainty and nonstationary characteristics of streamflow data has been treated using a set of coupled data pre-processing methods before being considered as input for an artificial neural network algorithm namely; rolling mechanism (RM) and grey models (GM). The rolling mechanism method is applied to smooth out the dataset based on the antecedent values of the model inputs before being applied to the GM algorithm. The optimization of the input datasets selection was performed using auto-correlation (ACF) and partial auto-correlation (PACF) functions. The pre-processed data was then integrated with two artificial neural network models, the back propagation (RMGM-BP) and Elman Recurrent Neural Network (RMGM-ERNN). The development, training, testing and evaluation of the proposed hybrid models were undertaken using streamflow data for two tropical hydrological basins (Johor and Kelantan Rivers). The hybrid RMGM-ERNN was found to provide better results than the hybrid RMGM-BP model. Relatively good performance of the proposed hybrid models with a data pre-processing approach provides a successful alternative to achieve better accuracy in streamflow forecasting compared to the traditional artificial neural network approach without a data pre-processing scheme.

An artificial neural network-based solution to locate the multilocation faults in double circuit series capacitor compensated transmission lines

During thunder storms, multilocation faults may occur at different locations in different phases of the 3 phase transmission lines at same or different time. This paper proposes an artificial neural network-based solution to locate the multilocation faults in double-circuit series capacitor compensated transmission lines (SCCTLs), unlike previous works that only locate the fault at single location. Although various fault location schemes have been proposed for normal shunt faults occurring at 1 location in SCCTL, nevertheless, finding the locations of multilocation faults in double circuit SCCTL has not been addressed so far. The proposed artificial neural network-based method determines the location of multilocation faults by using current and voltage signals of 1 end of line only, thus avoiding the need of communication link. The signals are preprocessed by using discrete wavelet transform. A comparative study of various neural networks such as feed-forward back-propagation network with Levenberg-Marquardt algorithm, Elman recurrent neural network with gradient descent algorithm, radial basis function neural network has been carried out. The proposed method is not affected by variation in different parameters viz. fault type, fault location, fault inception angle, fault resistance, degree of series compensation, and location of series capacitor. The key advantage of the method is that it correctly estimates the locations of multilocation faults as well as single-location fault, thus making it more reliable and accurate as compared to conventional fault location schemes.

Forecasting ozone concentration levels using Box-Jenkins ARIMA modelling and artificial neural networks: A comparative study

An accurate forecasting of tropospheric ozone (O3) concentration is beneficial for strategic planning of air quality. In this study, various forecasting techniques are used to forecast the daily maximum O3 concentration levels at a monitoring station in the Klang Valley, Malaysia. The Box-Jenkins autoregressive integrated moving-average (ARIMA) approach and three types of neural network models, namely, back-propagation neural network, Elman recurrent neural network and radial basis function neural network are considered. The daily maximum data, spanning from 1 January 2011 to 7 August 2011, was obtained from the Department of Environment, Malaysia. The performance of the four methods in forecasting future values of ozone concentrations is evaluated based on three criteria, which are root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE). The findings show that the Box-Jenkins approach outperformed the artificial neural network methods.

Deep Elman recurrent neural networks for statistical parametric speech synthesis

Owing to the success of deep learning techniques in automatic speech recognition, deep neural networks (DNNs) have been used as acoustic models for statistical parametric speech synthesis (SPSS). DNNs do not inherently model the temporal structure in speech and text, and hence are not well suited to be directly applied to the problem of SPSS. Recurrent neural networks (RNN) on the other hand have the capability to model time-series. RNNs with long short-term memory (LSTM) cells have been shown to outperform DNN based SPSS. However, LSTM cells and its variants like gated recurrent units (GRU), simplified LSTMs (SLSTM) have complicated structure and are computationally expensive compared to the simple recurrent architecture like Elman RNN. In this paper, we explore deep Elman RNNs for SPSS and compare their effectiveness against deep gated RNNs. Specifically, we perform experiments to show that (1) Deep Elman RNNs are better suited for acoustic modeling in SPSS when compared to DNNs and perform competitively to deep SLSTMs, GRUs and LSTMs, (2) Context representation learning using Elman RNNs improves neural network acoustic models for SPSS, and (3) Elman RNN based duration model is better than the DNN based counterpart. Experiments were performed on Blizzard Challenge 2015 dataset consisting of 3 Indian languages (Telugu, Hindi and Tamil). Through subjective and objective evaluations, we show that our proposed systems outperform the baseline systems across different speakers and languages.

A hybrid time series prediction model based on recurrent neural network and double joint linear–nonlinear extreme learning network for prediction of carbon efficiency in iron ore sintering process

Iron ore sintering process is the second-most energy-consuming procedure in the iron making industry. The main energy for it is the combustion of coke, which consists primary of carbon. In order to improve the carbon efficiency, it is necessary to predict it. A comprehensive carbon ratio (CCR) was used to be the metric for estimating the carbon efficiency. An iron ore sintering process has the characteristics of autocorrelation of time series of CCR, multiple variables, linearity and nonlinearity, and time delay. In this study, a hybrid time series prediction model was built to predict the CCR based on these characteristics. It consists of two parts: time series prediction based on Elman recurrent neural network (RNN) and Elman-residuals prediction based on double joint linear-nonlinear extreme learning network (JLNELN). The Elman RNN with a context layer has the ability to model the dynamical and nonlinear components in the time series, and the double JLNELN with the input neurons not only connected to the hidden neurons but also to the output neurons has the ability to model both the nonlinear and linear components in the prediction residuals. Actual run data was collected to verify the validity of the devised hybrid model. Experiment results have shown that the hybrid model achieved much higher regression precision than a single Elman RNN, which shows the necessity and validity of the double JLNELN model in the prediction of the Elman residuals. The experiment results of the double JLNELN method also show higher regression precision than both a double extreme learning machine method and a single JLNELN method, which verified the validity of the JLNELN method and the double structure of the prediction model.

Pronóstico de la curva de producción de huevos con redes neuronales

La comparación entre la curva de producción real del huevo y la gráfica propuesta por las pautas de gestión, tiene como objetivo la evaluación continua del rendimiento. Los objetivos de este estudio fueron comparar la capacidad de la curva de ajuste de la producción diaria de huevo de Lokjorst (LM), la red neuronal del perceptrón multicapa (MP) y las redes neuronales recurrantes de Jordania y Elman (RNNJ y RNNE, respectivamente) para la predicción del huevo diario producción en gallinas ponedoras comerciales. Los modelos se instalaron utilizando 4650 datos de 12 lotes seleccionados. Los modelos MP y LM dieron un buen ajuste a los datos, con valores de correlación superiores a 0,95 y que representan más del 95% de la variabilidad en la producción diaria de óvulos. Para el pronóstico de producción, MP fue una técnica con una precisión aceptable y menos variación. El modelo MP se recomienda como herramienta de ajuste y previsión de la curva diaria de producción de huevos en gallinas comerciales.

Modelling of lead removal from battery industrial wastewater treatment sludge leachate on cement kiln dust by using Elman's RNN

In this study, an Elman type recurrent neural network (RNN) was employed to develop a prediction model for lead removal from industrial sludge leachate using cement kiln dust. The leaching characteristics of industrial sludge were observed through the toxicity characteristics leaching procedure (TCLP). Dosage, contact time, and temperature were considered as independent experimental factors. A comparison between the model results and experimental data showed that the Elman's RNN model is able to predict lead removal from industrial sludge leachate. The outcomes of suggested Elman's RNN modelling were then compared to batch experimental studies. The results show that industrial sludge leachate using cement kiln dust is an efficient sorbent, and Elman's RNN is dynamic in nature and is able to model the batch experimental system.

Prediction by a Hybrid of Wavelet Transform and Long-Short-Term-Memory Neural Network

Data originating from some specific fields, for in-stance tourist arrivals, may exhibit a high degree of fluctuations as well as non-linear characteristics due to time varying behaviors. This paper proposes a new hybrid method to perform prediction for such data. The proposed hybrid model of wavelet transform and long-short-term memory (LSTM) recurrent neural network (RNN) is able to capture non-linear attributes in tourist arrival time series. Firstly, data is decomposed into constitutive series through wavelet transform. The decomposition is expressed as a function of a combination of wavelet coefficients, which have different levels of resolution. Then, LSTM neural network is used to train and simulate the value at each level to find the bias vectors and weighting coefficients for the prediction value. A sliding windows model is employed to capture the time series nature of the data. An evaluation is conducted to compare the proposed model with other RNN algorithms, i.e., Elman RNN and Jordan RNN, as well as the combination of wavelet transform with each of them. The result shows that the proposed model has better performance in terms of training time than the original LSTM RNN, while the accuracy is better than the hybrid of wavelet-Elman and the hybrid of wavelet-Jordan.