Recurrent Neural Network Method Research Articles

A complete ensemble empirical mode decomposition with adaptive noise deep autoregressive recurrent neural network method for the whole life remaining useful life prediction of lithium-ion batteries

A complete ensemble empirical mode decomposition with adaptive noise deep autoregressive recurrent neural network method for the whole life remaining useful life prediction of lithium-ion batteries

Low-Resource Language Information Processing using Dwarf Mongoose Optimization with Deep Learning Based Sentiment Classification

Asian and low-resource language information processing refers to the field of computational linguistics that aims to develop natural language processing (NLP) technologies for languages that have fewer available language resources or are less commonly spoken. This is an important field of study because many languages in Asia and other parts of the world are underrepresented in the field of NLP, which may limit access to information and technology for speakers of these languages. The growing volume of user-generated content on the web has made sentiment analysis (SA) a significant tool for extracting data regarding human emotional states. Twitter sentiment detectors provide a superior solution for assessing the quality of products and services compared to other conventional technologies. The detection performance and classifier accuracy of SA, which can be highly dependent on classifier methods and the quality of input features have been utilised. Deep learning (DL) methods use distinct techniques to extract data from raw data such as tweets or texts and represent them in different forms of models. Therefore, this article presents a Dwarf Mongoose Optimization with Deep Learning-Based Twitter Sentiment Classification (DMODL-TSC) technique to classify sentiments based on tweets. The presented DMODL-TSC technique leverages the concepts of natural language processing (NLP) and DL. Primarily, the raw tweets are preprocessed to transform them into a useful format. Next, the DMODL-TSC technique uses the advanced FastText word embedding technique. Moreover, the bidirectional recurrent neural network (BiRNN) method is utilized for the recognition of sentiments. Finally, the DMO technique is utilized for the optimal hyperparameter optimization of the BiRNN method, which leads to effective classification performance. The comprehensive result examination of the DMODL-TSC system was tested on three datasets, and the obtained outcomes illustrate the supremacy of the DMODL-TSC approach.

Localized estimation of electromagnetic sources underlying event-related fields using recurrent neural networks.

To use a recurrent neural network (RNN) to reconstruct neural activity responsible for generating noninvasively measured electromagnetic signals.
Approach: Output weights of an RNN were fixed as the lead field matrix from volumetric source space computed using the boundary element method with co-registered structural magnetic resonance images and magnetoencephalography (MEG). Initially, the network was trained to minimize mean-squared-error loss between its outputs and MEG signals, causing activations in the penultimate layer to converge towards putative neural source activations. Subsequently, L1 regularization was applied to the final hidden layer, and the model was fine-tuned, causing it to favour more focused activations. Estimated source signals were then obtained from the outputs of the last hidden layer. We developed and validated this approach with simulations before applying it to real MEG data, comparing performance with beamformers, minimum-norm estimate, and mixed-norm estimate source reconstruction methods.
Main results: The proposed RNN method had higher output signal-to-noise ratios and comparable correlation and error between estimated and simulated sources. Reconstructed MEG signals were also equal or superior to the other methods regarding their similarity to ground-truth. When applied to MEG data recorded during an auditory roving oddball experiment, source signals estimated with the RNN were generally biophysically plausible and consistent with expectations from the literature.
Significance: This work builds on recent developments of RNNs for modelling event-related neural responses by incorporating biophysical constraints from the forward model, thus taking a significant step towards greater biological realism and introducing the possibility of exploring how input manipulations may influence localized neural activity.

Implementation of recurrent neural network for the forecasting of USD buy rate against IDR

<span lang="EN-US">This study implements a recurrent neural network (RNN) by comparing two RNN network structures, namely Elman and Jordan using the backpropagation through time (BPTT) programming algorithm in the training and forecasting process in foreign exchange forecasting cases. The activation functions used are the linear transfer function, the tan-sigmoid transfer function (Tansig), and the log-sigmoid transfer function (Logsig), which are applied to the hidden and output layers. The application of the activation function results in the log-sigmoid transfer function being the most appropriate activation function for the hidden layer, while the linear transfer function is the most appropriate activation function for the output layer. Based on the results of training and forecasting the USD against IDR currency, the Elman BPTT method is better than the Jordan BPTT method, with the best iteration being the 4000<sup>th</sup> iteration for both. The lowest root mean square error (RMSE) values for training and forecasting produced by Elman BPTT were 0.073477 and 122.15 the following day, while the Jordan backpropagation RNN method yielded 0.130317 and 222.96 also the following day.</span><p> </p>

Human Age Estimation Through Audio Utilising MFCC and RNN

Age is one of human main attributes. Age is important factor to improve communication experience. Age estimation has been used in several applications to improve user experience. Therefore, an approach is needed to estimate the user age, one of which is through audio. In this study, Mel Frequency Cepstrum Coefficients (MFCC) and Recurrent Neural Network (RNN) will be used to estimate age through audio. MFCC is used to get features from audio data, while RNN is used to estimate age. Dataset used here was taken from corpus of user speech data on the Common Voice website. This study shows that MFCC and RNN methods are able to estimate human age through audio with highest accuracy obtained in SimpleRNN is 0.5647, and 0.7087 in LSTM.

EARLY PREDICTION OF UNEXPECTED LATENT SHOCK IN THE EMERGENCY DEPARTMENT USING VITAL SIGNS.

Objective/Introduction : Sequential vital-sign information and trends in vital signs are useful for predicting changes in patient state. This study aims to predict latent shock by observing sequential changes in patient vital signs. Methods : The dataset for this retrospective study contained a total of 93,194 emergency department (ED) visits from January 1, 2016, and December 31, 2020, and Medical Information Mart for Intensive Care (MIMIC)-IV-ED data. We further divided the data into training and validation datasets by random sampling without replacement at a 7:3 ratio. We carried out external validation with MIMIC-IV-ED. Our prediction model included logistic regression (LR), random forest (RF) classifier, a multilayer perceptron (MLP), and a recurrent neural network (RNN). To analyze the model performance, we used area under the receiver operating characteristic curve (AUROC). Results : Data of 89,250 visits of patients who met prespecified criteria were used to develop a latent-shock prediction model. Data of 142,250 patient visits from MIMIC-IV-ED satisfying the same inclusion criteria were used for external validation of the prediction model. The AUROC values of prediction for latent shock were 0.822, 0.841, 0.852, and 0.830 with RNN, MLP, RF, and LR methods, respectively, at 3 h before latent shock. This is higher than the shock index or adjusted shock index. Conclusion : We developed a latent shock prediction model based on 24 h of vital-sign sequence that changed with time and predicted the results by individual.

Earthquake Prediction for the Düzce Province in the Marmara Region Using Artificial Intelligence

By definition, an earthquake is a naturally occurring event. This natural event may be a disaster that causes significant damage, loss of life, and other economic effects. The possibility of predicting a natural event such as an earthquake will minimize the negative effects mentioned above. In this study, data collection, processing, and data evaluation regarding earthquakes were carried out. Earthquake forecasting was performed using the RNN (recurrent neural network) method. This study was carried out using seismic data with a magnitude of 3.0 and above of the Düzce Province between 1990 and 2022. In order to increase the learning potential of the method, the b and d values of earthquakes were calculated. The detection of earthquakes within a specific time interval in the Marmara region of Turkey, the classification of earthquake-related seismic data using artificial neural networks, and the generation of predictions for the future highlight the importance of this study. Our results demonstrated that the prediction performance could be significantly improved by incorporating the b and d coefficients of earthquakes, as well as the data regarding the distance between the Moon and the Earth, along with the use of recurrent neural networks (RNNs).

Time series analysis model for forecasting unsteady electric load in buildings

Accurate and reliable load forecasting is crucial for ensuring the security and stability of the power grid. This paper proposes a combined prediction method based on Empirical Wavelet Transform (EWT) and Autoformer time series prediction model for the non-stationary and non-linear time series of electric load. The original sequence is first decomposed by EWT to obtain a set of stable subsequences, and then the Autoformer time series prediction model is used to predict each subsequence. Finally, the prediction results of each subsequence are combined to obtain the final prediction results. The proposed EWT-Autoformer prediction model is applied to an electric load example, and the experimental results are compared with the Recurrent Neural Network (RNN) method, Long Short-Term Memory (LSTM) method, and Informer method under the same conditions. The experimental results indicate that compared to LSTM, the method proposed in the paper has an R2 improvement of 9–20 percentage points, an improvement of 6–8 percentage points compared to RNN, an improvement of 3–7 percentage points compared to Informer, and an improvement of 2–3 percentage points compared to Autoformer. In addition, the RMSE and MAE are also significantly lower than other models.

Implementation of CRNN Method for Lung Cancer Detection based on Microarray Data

Lung Cancer is one of the cancer types with the most significant mortality rate, mainly because of the disease's slow detection. Therefore, the early identification of this disease is crucial. However, the primary issue of microarray is the curse of dimensionality. This problem is related to the characteristic of microarray data, which has a small sample size yet many attributes. Moreover, this problem could lower the accuracy of cancer detection systems. Various machines and deep learning techniques have been researched to solve this problem. This paper implemented a deep learning method named Convolutional Recurrent Neural Network (CRNN) to build the Lung Cancer detection system. Convolutional neural networks (CNN) are used to extract features, and recurrent neural networks (RNN) are used to summarize the derived features. CNN and RNN methods are combined in CRNN to derive the advantages of each of the methods. Several previous research uses CRNN to build a Lung Cancer detection system using medical image biomarkers (MRI or CT scan). Thus, the researchers concluded that CRNN achieved higher accuracy than CNN and RNN independently. Moreover, CRNN was implemented in this research by using a microarray-based Lung Cancer dataset. Furthermore, different drop-out values are compared to determine the best drop-out value for the system. Thus, the result shows that CRNN gave a higher accuracy than CNN and RNN. The CRNN method achieved the highest accuracy of 91%, while the CNN and RNN methods achieved 83% and 71% accuracy, respectively.

Dynamic graph neural network with adaptive edge attributes for air quality prediction: A case study in China

Air quality prediction is a typical Spatiotemporal modeling problem, which always uses different components to handle spatial and temporal dependencies in complex systems separately. Previous models based on time series analysis and recurrent neural network (RNN) methods have only modeled time series while ignoring spatial information. Previous graph convolution neural networks (GCNs) based methods usually require providing spatial correlation graph structure of observation sites in advance. The correlations among these sites and their strengths are usually calculated using prior information. However, due to the limitations of human cognition, limited prior information cannot reflect the real station-related structure or bring more effective information for accurate prediction. To this end, we propose a novel Dynamic Graph Neural Network with Adaptive Edge Attributes (DGN-AEA) on the message passing network, which generates the adaptive bidirected dynamic graph by learning the edge attributes as model parameters. Unlike prior information to establish edges, our method can obtain adaptive edge information through end-to-end training without any prior information. Thus reducing the complexity of the problem. Besides, the hidden structural information between the stations can be obtained as model by-products, which can help make some subsequent decision-making analyses. Experimental results show that our model received state-of-the-art performance than other baselines.

The Application of Recurrent Neural Network Method with the Long Short Term Memory (LSTM) Approach to Forecast Hybe Corporation's Stock Price

Background: Indonesia has a large land and sea area that has the potential for salt production. Efforts to increase salt production are still not optimal because there are still several areas that produce salt products in small quantities even though they have a large land area and a large number of farmers, so for areas like this, it is necessary to carry out guidance so that they can increase the amount of salt production. So far, no system has been used to group salt-producing regions, so it is impossible to know which areas still have the potential to increase their production.Objective: This research aims to create an application system to classify salt-producing regions.Methods: The clustering method used is K-Means, where this method can group datasets into several predetermined clusters.Result: The results of application testing carried out using 10 salt-producing region datasets obtained 4 regions with small production volume clusters, 3 regions with sufficient production volume clusters, 3 regions with large production volume clusters.Conclusion: The application of grouping salt-producing areas using the K-Means method can help the process of grouping salt-producing regions easier, faster, and more objectively.

Detecting Hoax Content on Social Media Using Bi-LSTM and RNN

Online media, such as websites and applications, have become a communication tool available on the internet. Social media is a part of online media that can be used to spread news, opinions, or even hoaxes, such as through Twitter. Although hoaxes are difficult to eliminate, several systems have been built using deep learning approaches that can process text and images to detect the truthfulness of news. In this study, four systems were built using four deep learning methods, namely Bi-directional Long Short-Term Memory (Bi-LSTM), Recurrent Neural Network (RNN), hybrid RNN-Bi-LSTM, and hybrid Bi-LSTM-RNN. Feature extraction was performed using Term Frequency - Inverse Document Frequency (TF-IDF) and feature expansion was performed using Global Vectors (GloVe). The data used has been adjusted according to the keyword of fake news on mainstream news portals. This study attempted several scenarios to compare the various methods that have been built, with the aim of finding the best method that provides the highest accuracy. The results showed that the Bi-LSTM method had the highest accuracy of 96.48%, while the hybrid Bi-LSTM-RNN method ranked second with an accuracy of 96.36%, followed by the RNN method with an accuracy of 95.49%, and the hybrid RNN-Bi-LSTM method with an accuracy of 95.34%.

Improving Performance of RNN-Based Models With Genetic Algorithm Optimization For Time Series Data

Stock price data or similar time series data can be used to carry out forecasting processes using past data. The method that can be used is like a neural network, one type of neural network that is used is the Recurrent Neural Network. When using the Recurrent Neural Network (RNN) method, we need to determine the appropriate parameters in order to get the best forecasting results. It takes experience or . In this study, this problem can be solved using optimization algorithms, such as Genetic Algorithms. With genetic algorithms, neural networks can be trained to get the best objective function. So that after implementing the RNN which was optimized using the Genetic Algorithm on stock time series data, when the trial was carried out without optimization the Genetic Algorithm got an RMSE value of 0.108, after being combined using the genetic algorithm it got an RMSE value of 0.106.

Brain Tumor Detection Using RNN

This research work aims to utilize the developed and evaluated Magnetic Resonance Imaging(MRI) technique for the classification of brain tumor and seizures employing Recurrent Neural Network (RNN). The medical science in the image processing is an emergent area that has suggested many progressive methods in detecting as well as analyzing a specific disease. Brain tumors treatment is recently getting progressively more challenging owing to the intricate shape,structureandthetextureoftumor.So,viaprogressingintheimageprocessing,differentmethodologies have been suggested for identifying the tumors inside brain. The progression in such area made a need for searching more upon the methods and approaches evolved for the extraction of tumor. Therefore, an extraction system the tumor from thebrainissuggestedutilizingMRIimages.Suchmethodincludesvariousproceduresofimageprocessing,likefiltering,theremovalofnoise,segmentation,andmorphologicalprocesses. Brain tumor extraction can be successfully achieved via conducting such processes upon. The cross-correlation is calculated between the changeable vector of a target and the zone of tumor for determining in what way the values of people of the zone of tumor are narrowly, associated utilizing the image processing and the RNN method accomplishing 99.71%accuracy. Key Word: Brain RNN, Image processing, Image segmentation, Feature extraction, Image classification

Evaluation of Early Concrete Damage Caused by Chloride-Induced Steel Corrosion Using a Deep Learning Approach Based on RNN for Ultrasonic Pulse Waves.

The objective of this study is to explore the feasibility of using ultrasonic pulse wave measurements as an early detection method for corrosion-induced concrete damages. A series of experiments are conducted using concrete cube specimens, at a size of 200 mm, with a reinforcing steel bar (rebar) embedded in the center. The main variables include the water-to-cement ratio of the concrete (0.4, 0.5, and 0.6), the diameter of the rebar (10 mm, 13 mm, 19 mm, and 22 mm), and the corrosion level (ranging from 0% to 20% depending on rebar diameter). The impressed current technique is used to accelerate corrosion of rebars in concrete immersed in a 3% NaCl solution. Ultrasonic pulse waves are collected from the concrete specimens using a pair of 50 kHz P-wave transducers in the through-transmission configuration before and after the accelerated corrosion test. Deep learning techniques, specifically three recurrent neural network (RNN) models (long short-term memory, gated recurrent unit, and bidirectional long short-term memory), are utilized to develop a classification model for early detection of concrete damage due to rebar corrosion. The performance of the RNN models is compared to conventional ultrasonic testing parameters, namely ultrasonic pulse velocity and signal consistency. The results demonstrate that the RNN method outperforms the other two methods. Among the RNN methods, the bidirectional long short-term memory RNN model had the best performance, achieving an accuracy of 74% and a Cohen's kappa coefficient of 0.48. This study establishes the potentiality of utilizing deep learning of ultrasonic pulse waves with RNN models for early detection of concrete damage associated with steel corrosion.

DATA COLLECTION AND PERFORMANCE EVALUATION OF RUNNING TRAINING SPORT USING DIFFERENT NEURAL NETWORK TECHNIQUES

With the increasing engagement of human beings in the pursuit of healthcare, running as a sport has become a fashionable and healthcare first choice. This research uses artificial intelligence technology to carry out intelligent analysis when conducting running training. Artificial intelligence technology can accurately analyze and predict the application requirements of sports training postures. We proposed an analysis of sports posture and a prediction system, which uses running training data in the form of a heart rate, recorded on a GPS smart sports watch, as well as using the recurrent neural network (RNN), long and short-term memory (LSTM) and the gate recursive unit (GRU). These three types of neural network methods can predict which method is best suited for a road race and can confirm that it will be completed within the scheduled finish time; these models will also perform an intelligent analysis of physical fitness (heart rate, pace) and running technology (cadence, pace). The training and test data are collected from the running training records (running distance, time, heart rate, stride frequency, stride length, pace, calories, altitude and other characteristic values) as input parameters, to test and compare the running completion time trends of the RNN, LSTM and GRU neural network methods in the exercise table, so as to evaluate their predictive abilities. The results show that the GRU method has the best predictive accuracy, and the least accurate is the LSTM method. After the hidden layers are added to the three predictive methods, the RNN is slightly regressive, the LSTM indicates a trend of significant improvement and the GRU exhibits less obvious changes.

Developing an optimized recurrent neural network model for air quality prediction using K-means clustering and PCA dimension reduction

Prediction is a means of forecasting a future value by using and analyzing historical or current data. A popular neural network architecture used as a prediction model is the Recurrent Neural Network (RNN) because of its wide application and very high generalization performance. This study aims to improve the RNN prediction model’s accuracy using k-means grouping and PCA dimension reduction methods by comparing the five distance functions. Data were processed using Python software and the results obtained from the PCA calculation yielded three new variables or principal components out of the five examined. This study used an optimized RNN prediction model with k-means clustering by comparing the Euclidean, Manhattan, Canberra, Average, and Chebyshev distance functions as a measure of data grouping similarity to avoid being trapped in the local optimal solution. In addition, PCA dimension reduction was also used in facilitating multivariate data analysis. The k-means grouping showed that the most optimal distance is the average function producing a DBI value of 0.60855 and converging at the 9th iteration. The RNN prediction model results evaluated based on the number of RMSE errors which was 0.83, while that of MAPE was 8.62%. Therefore, it was concluded that the K-means and PCA methods generated a more optimal prediction model for the RNN method.

Recurrent Neural Network Based Joint Equalization and Decoding Method for Trellis Coded Modulated Optical Communication System

In this paper, a joint equalization and decoding method based on recurrent neural networks (RNNs) is proposed for trellis coded modulation (TCM) systems. For traditional methods based on concatenated equalizers and Viterbi decoder, the information may be lost between the equalization and decoding. However, our proposed joint method can obtain the information bits directly from the received symbols and the error also can be corrected in iterations. In two-dimensional eight-level pulse amplitude modulation (2D-PAM8) links, we experimentally demonstrate that, compared with the traditional method based on Volterra filter and Viterbi decoder, the proposed joint method based on RNNs with similar complexity can achieve bit-error rate (BER) performance improvements of 1 dB, 1 dB, and 2 dB in 87.5 Gb/s, 100 Gb/s, and 112.5 Gb/s transmissions at 7% hard-decision forward-error-correction (HD-FEC) limit, respectively. For the 10 km standard single-mode fiber (SSMF) link at 87.5 Gb/s, the traditional method cannot get the BER below the 7% HD-FEC limit whereas the joint RNNs method with similar complexity can get the BER below the 7% HD-FEC limit. As the nonlinearity increases, the proposed joint method can achieve greater improvement of BER performance.

Predicting Cryptocurrency Price Using RNN and LSTM Method

Cryptocurrency price prediction is a crucial task for financial investors as it helps determine appropriate investment strategies and mitigate risk. In recent years, deep learning methods have shown promise in predicting time-series data, making them a viable approach for cryptocurrency price prediction. In this study, we compare the effectiveness of two deep learning techniques, the Recurrent Neural Network (RNN) and Long-Short Term Memory (LSTM), in predicting the prices of Bitcoin and Ethereum. Results of this research show that the LSTM method outperformed the RNN method, obtaining lower Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE) values for predicting both cryptocurrencies. Bitcoin and Ethereum. Specifically, the LSTM model had a RMSE of 0.061 and MAPE of 5.66% for predicting Bitcoin, and a RMSE of 0.036 and MAPE of 4.58% for predicting Ethereum. In this research, we found that the LSTM model is a more effective method for predicting cryptocurrency prices than the RNN model.

Effective RNN-Based Forecasting Methodology Design for Improving Short-Term Power Load Forecasts: Application to Large-Scale Power-Grid Time Series

This article introduces a carefully-engineered forecasting methodology for day-ahead electric power load forecasts evaluated using the European Network of Transmission System Operators for Electricity (ENTSO-E). Two steps were employed to configure the desired forecasting methodology: First, a straightforward processing pipeline is proposed to enable systematic preprocessing of raw multivariate time-discrete power data extracted from the ENTSO-E repository, including a stride-based sliding window approach to generate time series-based batches ready for the supervised learning procedure. Second, the lightweight type of recurrent neural network method, namely gated recurrent units (GRU), is selected and carefully calibrated to yield accurate multi-step forecasts, which was trained using the preprocessed multivariate time series data to render day-ahead power load forecasts. The forecasting estimates generated by the proposed GRU model are evaluated using a set of regression-based metrics to assess the models’ precisions. The empirical results show that the proposed forecasting methodology yields outstanding day-ahead power load forecasting performance regarding the enterprise-class measured data compared to a statistical model, namely autoregressive integrated moving average with exogenous variables (ARIMAX), as well as the actual day-ahead forecasts generated by the ENTSO-E platform.