Recurrent Neural Network Method Research Articles

A novel hybrid deep learning approach with GWO–WOA optimization technique for human activity recognition

The effectiveness of Human Activity Recognition (HAR) models can be largely attributed to the components derived from domain expertise. The classification system swiftly and effectively categorizes human physical activity by utilizing a comprehensive collection of variables. To construct HAR models and categorize different activities, deep learning algorithms have recently seen increased application in academic research for autonomously extracting features from raw sensory data. The primary goal of this study is to develop a strong and accurate activity classification model by first extracting features from an input dataset and then using deep learning models. In this research, we look at how well the model does in the field of HAR. Analyzing time-series data from smartphones and wearable sensors, algorithms based on deep learning (DL) have been used to forecast various human actions. Applying DL-based methods to time-series data presents a variety of challenges, notwithstanding their usage for activity identification. If the suggested approach is put into practice, the concerns mentioned before may be alleviated. To construct HAR classification techniques using data collected from wearable sensors, this study presents two Hybrid Learning Algorithms (HLA). This research employs Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods to teach the model sequential and spatial patterns. The process of selecting characteristics has been enhanced through the use of optimization techniques. Together, the Whale Optimization Algorithm and the Grey Wolf Optimizer form a Hybrid Optimization approach, which is introduced in this study. By combining and capitalizing the benefits of both approaches, the hybrid technique enhances the optimization process as a whole. This research may also assist individuals who have undergone amputations or experienced strokes in selecting an appropriate model and developing a personalized gait reference trajectory.

A Novel Swarm Exploring Varying Parameter Recurrent Neural Network for Solving Non-Convex Nonlinear Programming.

Aiming at solving non-convex nonlinear programming efficiently and accurately, a swarm exploring varying parameter recurrent neural network (SE-VPRNN) method is proposed in this article. First, the local optimal solutions are searched accurately by the proposed varying parameter recurrent neural network. After each network converges to the local optimal solutions, information is exchanged through a particle swarm optimization (PSO) framework to update the velocities and positions. The neural network searches for the local optimal solutions again from the updated position until all the neural networks are searched to the same local optimal solution. For improving the global searching ability, wavelet mutation is applied to increase the diversity of particles. Computer simulations show that the proposed method can solve the non-convex nonlinear programming effectively. Compared with three existing algorithms, the proposed method has advantages in accuracy and convergence time.

Adjusted SpikeProp algorithm for recurrent spiking neural networks with LIF neurons

A problem related to the development of a supervised learning method for recurrent spiking neural networks is addressed in the paper. The widely used Leaky-Integrate-and-Fire model has been adopted as a spike neuron model. The proposed method is based on a known SpikeProp algorithm. In detail, the developed method enables gradient descent learning of recurrent or multi-layer feedforward spiking neural networks. The research included an extended verification study for the classical XOR classification problem. In addition, the developed learning method has been used to provide a spiking neural black-box model of fast processes occurring in a pressurised water nuclear reactor. The obtained simulation results demonstrate satisfactory effectiveness of the proposed approach.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Fatigue driving has emerged as the predominant causative factor for road traffic safety accidents. The fatigue driving detection method, derived from laboratory simulation data, faces challenges related to imbalanced data distribution and limited recognition accuracy in practical scenarios. In this study, we introduce a novel approach utilizing a gated recurrent neural network method, employing whale optimization algorithm for fatigue driving identification. Additionally, we incorporate an attention mechanism to enhance identification accuracy. Initially, this study focuses on the driver's operational behavior under authentic vehicular conditions. Subsequently, it employs wavelet energy entropy, scale entropy, and singular entropy analysis to extract the fatigue-related features from the driver's operational behavior. Subsequently, this study adopts the cross-validation recursive feature elimination method to derive the optimal fatigue feature index about operational behavior. To effectively capture long-range dependence relationships, this study employs the gated recurrent unit neural network method. Lastly, an attention mechanism is incorporated in this study to concentrate on pivotal features within the data sequence of driving behavior. It assigns greater weight to crucial information, mitigating information loss caused by the extended temporal sequence. Experimental results obtained from real vehicle data demonstrate that the proposed method achieves an accuracy of 89.84% in third-level fatigue driving detection, with an omission rate of 10.99%. These findings affirm the feasibility of the approach presented in this study.

Comparison of the Accuracy of The Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android

Communication is a process of exchanging information that aims to establish relationships between humans. Communication difficulties are an obstacle for people with deaf disabilities or often called Deaf Friends, where they find it difficult to interact with friends around them. Sign language is the main medium of communication used worldwide by people with disabilities i.e. deaf and speech impaired. Communication between deaf people and those around them is often an obstacle because most people do not understand sign language which is often used as a medium of communication by deaf people. In dealing with this problem, researchers want to analyze the accuracy level of the Android-Based Bahasa Isyarat Indonesia Detection System (BISINDO) using the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods in order to determine which methods can be implemented to an Android device. This study shows that Convolutional Neural Network (CNN) has a greater and more stable accuracy rate compared to the Recurrent Neural Network (RNN) model where the CNN model produces an accuracy rate of 89%, and indicates that the ability to recognize images based on the division of Bahasa Isyarat Indonesia (BISINDO) alphabetic classes is good..

Exploring Different Dynamics of Recurrent Neural Network Methods for Stock Market Prediction - A Comparative Study

ABSTRACT The intricate and unpredictable nature of stock markets underscores the importance of precise forecasting for timely detection of downturns and subsequent rebounds. Various factors, including news, rumors surrounding events or companies, market sentiments, and governmental policies, can significantly impact stock prices. Nevertheless, the precision of current methods remained insufficient until the adoption of artificial neural network architectures like long short-term memory (LSTM). The aim of this study is to create a precise AI-driven platform tailored for both the Indian and international stock markets. This platform is designed to assist retail investors in navigating digital environments by employing various LSTM algorithms. Its primary goals include predicting stock price fluctuations, pinpointing potential investment prospects, and refining trading strategies. The application aims to leverage advanced LSTM algorithms to analyze historical market data, recognize patterns, and provide real-time insights. It will take past price and process it through LSTM algorithms to take a logical decision. In the quest to broaden retail participation in the capital markets, the effort is to develop an application for novice investors who either have no time in research or are the victims of financial mis-selling and enable them to leverage the technology to their advantage.

Forecasting Hourly Energy Fluctuations Using Recurrent Neural Network (RNN)

Energy forecasting is currently essential due to its various benefits. Energy data analysis for forecasting requires a functional method due to the complexity of the observed data. This forecasting study used the Recurrent Neural Networks (RNN) method. Parameters used include batch size, epoch, hidden layers, loss function, and optimizer obtained from hyperparameter tuning grid search. A comparison of different normalization methods, namely min-max, and z-score, was conducted. Using min-max normalization yielded the best performance with MAPE of 3.9398%, RMSE of 0.0630, and R2 of 0.8996. In testing with z-score normalization, it showed a performance of MAPE of 10.6120%, RMSE of 0.7648, and R2 of 0.4142.

InfusedHeart: A Novel Knowledge-Infused Learning Framework for Diagnosis of Cardiovascular Events

In the undertaken study, we have used a customized dataset termed ``Cardiac-200'' and the benchmark dataset ``PhysioNet.'' which contains 1500 heartbeat acoustic event samples (without augmentation) and 1950 samples (with augmentation) heartbeat acoustic events such as normal, murmur, extrasystole, artifact, and other unlabeled heartbeat acoustic events. The primary reason for designing a customized dataset, ``cardiac-200,'' is to balance the total number of samples into categories such as normal and abnormal heartbeat acoustic events. The average duration of the recorded heartbeat acoustic events is 10-12 s. In the undertaken study, we have analyzed and evaluated various heartbeat acoustic events using audio processing libraries such as Chromagram, Chroma-cq, Chroma-short-time Fourier transform (STFT), Chroma-cqt, and Chroma-cens to extract more information from the recorded heartbeat sound signals. The noise removal process has been carried out using local binary pattern (LBP) methodology. The noise-robust heartbeat acoustic images are classified using long short-term memory (LSTM)-convolutional neural network (CNN), recurrent neural network (RNN), LSTM, Bi-LSTM, CNN, K-means Clustering, and support vector machine (SVM) methods. The obtained results have shown that the proposed InfusedHeart Framework had outclassed all the other customized machine learning and deep learning approaches such as RNN, LSTM, Bi-LSTM, CNN, K-means Clustering, and SVM-based classification methodologies. The proposed Knowledge-infused Learning Framework has achieved an accuracy of 89.36% (without augmentation), 93.38% (with augmentation), and a standard deviation of 10.64 (without augmentation), and 6.62 (with augmentation). Furthermore, the proposed framework has been tested for various signal-to-noise ratio conditions such as SignaltoNoiseRatio0, SignaltoNoiseRatio3, SignaltoNoiseRatio6, SignaltoNoiseRatio9, SignaltoNoiseRatio12, SignaltoNoiseRatio15, and SignaltoNoiseRatio18. In the end, we have shown a detailed comparison of texture and without texture approaches and have discussed future enhancements and prospective ways for future directions.

Molecular Generation, QSAR, and Molecular Dynamic Simulation Studies for Virtual Screening of DNA Polymerase Theta Inhibitors.

The machine learning-based QSAR modeling procedure, molecular generations, and molecular dynamic simulations were applied to virtually screen the DNA polymerase theta inhibitors. The DNA polymerase theta (Polθ or POLQ) is an attractive target for treatments of homologous recombination deficient (such as BRCA deficient) cancers. There are no approved drugs for targeting POLQ, and only one inhibitor is in Phase Ⅱclinical trials; thus, it is necessary to develop novel POLQ inhibitors. To build machine learning models that predict the bioactivities of POLQ inhibitors. To build molecular generation models that generate diverse molecules. To virtually screen the generated molecules by the machine learning models. To analyze the binding modes of the screening results by molecular dynamic simulations. In the present work, 325 inhibitors with POLQ polymerase domain bioactivities were Collected. Two machine learning methods, random forest and deep neural network, were used for building the ligand- and structure-based quantitative structure-activity relationship (QSAR) models. The substructure replacement-based method and transfer learning-based deep recurrent neural network method were used for molecular generations. Molecular docking and consensus QSAR models were carried out for virtual screening. The molecular dynamic simulations and MM/GBSA binding free energy calculation and decomposition were used to further analyze the screening results. The MCC values of the best ligand- and structure-based consensus QSAR models reached 0.651 and 0.361 for the test set, respectively. The machine learning-based docking scores had better-predicted ability to distinguish the highly and weakly active poses than the original docking scores. The 96490 molecules were generated by both molecular generation methods, and 10 molecules were retained by virtual screening. Four favorable interactions were concluded by molecular dynamic simulations. We hope that the screening results and the binding modes are helpful for designing the highly active POLQ polymerase inhibitors and the models of the molecular design workflow can be used as reliable tools for drug design.

Research on land subsidence-rebound affected by dualistic water cycle driven by climate change and human activities in Dezhou City, China

Over-exploitation of groundwater is the principal anthropogenic trigger for land subsidence in the North China Plain (NCP). With the promulgation of policies on groundwater extraction restrictions and bans, seasonal fluctuations and uplift have begun to occur in some typical subsidence areas in the NCP. In order to explore the land subsidence-rebound drivers, this study takes Dezhou City, a typical subsidence area in the NCP, as the study area and selects the main drivers in climate change: precipitation and evapotranspiration and the main driver in human activities: groundwater extraction, which together forms the background of the regional dualistic water cycle. In this background, this study selects the corresponding datasets and uses the improved Water Balance constraints Recurrent Neural Network (WB-RNN) method to establish the link between precipitation data, evapotranspiration data, groundwater extraction and groundwater level. Then uses the Gate Recurrent Unit-Convolutional Neural Network (GRU-CNN) method to establish the link between groundwater level and Interferometric Synthetic Aperture Radar (InSAR) deformation, and then obtains the link between the drivers of the dualistic water cycle and land subsidence. Comparative analysis and changing model inputs found that 1) Seasonal fluctuations in land subsidence-rebound in Dezhou City are mainly controlled by the groundwater levels in Aquifer Groups III and IV, with amplitudes up to 15 mm per year. 2) Groundwater level changes in different layers lag precipitation at different times. Excluding the influence of human extraction factors, the maximum ground surface deformation fluctuation caused by monthly 260 mm precipitation can be up to 21.44 mm. 3) The impact of short-term heavy precipitation caused by Typhoon Lekima and Doksuri in 2019 and 2023 can cause deformation fluctuations to a certain extent.

Particle Swarm Optimization-Based Model Abstraction and Explanation Generation for a Recurrent Neural Network

In text classifier models, the complexity of recurrent neural networks (RNNs) is very high because of the vast state space and uncertainty of transitions, which makes the RNN classifier’s explainability insufficient. It is almost impossible to explain the large-scale RNN directly. A feasible method is to generalize the rules undermining it, that is, model abstraction. To deal with the low efficiency and excessive information loss in existing model abstraction for RNNs, this work proposes a PSO (Particle Swarm Optimization)-based model abstraction and explanation generation method for RNNs. Firstly, the k-means clustering is applied to preliminarily partition the RNN decision process state. Secondly, a frequency prefix tree is constructed based on the traces, and a PSO algorithm is designed to implement state merging to address the problem of vast state space. Then, a PFA (probabilistic finite automata) is constructed to explain the RNN structure with preserving the origin RNN information as much as possible. Finally, the quantitative keywords are labeled as an explanation for classification results, which are automatically generated with the abstract model PFA. We demonstrate the feasibility and effectiveness of the proposed method in some cases.

Enhanced supercapacitive performance of lead vanadate hybrid architect on nickel foam with machine learning-driven capacitance prediction

The structural and chemical architecture of the active electrode material must provide an optimum environment for charge storage as well as stability to provide crucial supercapacitance properties. In this study, we developed a binder-free protocol to obtain an optimized polymorph of lead vanadate (PbV) with a unique hybrid architecture on nickel foam for hybrid supercapacitor applications. Furthermore, a detailed capacitance power prediction model is proposed based on several machine learning and recurrent neural network methods to improve the reliability of the fabricated hybrid supercapacitor. Specifically, recursive feature elimination is implemented to alleviate the problems of overfitting and the curse of dimensionality by identifying and eliminating extraneous characteristics while determining the appropriate features of the capacitance power. The experimental observations are validated by DFT analysis concerning the role of oxygen vacancies for the improvement of electrical characteristics. Ultimately, the fabricated energy storage device exhibits impressive energy and power densities of 26.18 Whkg−1 and 6−1, respectively. Importantly, the predicted results confirm that the long-short-term memory network provides excellent results for capacitance prediction of hybrid supercapacitors with R2 = 0.83 in the test set and synthesized sample prediction with a mean absolute error of 20.37.

An adaptive Marine Predator Optimization Algorithm (MPOA) integrated Gated Recurrent Neural Network (GRNN) classifier model for arrhythmia detection

Cardiovascular disorders are typically diagnosed using an Electrocardiogram (ECG). It is a painless method that mimics the cyclical contraction and relaxation of the heart's muscles. By monitoring the electrical activity of the heart, an ECG can be used to identify irregular heartbeats, strokes, cardiac illnesses, or enlarged hearts. Numerous studies and analyses of ECG signals to identify heart problems have been conducted during the past few years. The disadvantages of the current and hybrid approaches include low accuracy, prolonged time, and a dearth of high-quality datasets. The proposed work aims to design a new detection tool for the diagnosis of Arrhythmia from ECG signals. In this study, the wavelet transformation technique is used for preprocessing after collecting the input ECG signal in order to create the quality-improved and normalized signals. The most pertinent characteristics are then selected from the preprocessed output using the cutting-edge Marine Predator Optimization Algorithm (MPOA), which accelerates classifier learning. In addition, the hyper tuned Gated Recurrent Neural Network (Hyp-GRNN) method is used to learn the most effective features and properly classify the normal and arrhythmia-affected signals. The Ebola Optimization Algorithm (EOSA) is used to execute the hyper parameter tweaking in order to enhance the accuracy and training time of the classifier. At the time of evaluation, performance is assessed and compared using a variety of metrics, including accuracy, precision, recall, F-measure and etc.

Effect of Epoch Value on the Performance of the RNN-LSTM Algorithm in Classifying Lazada App Review Sentiments

In today's development, the process of buying and selling transactions between sellers and buyers is so developed. not only done directly but can also be done online or can be called e-commerce. Which is where the development of technology is so fast that it indirectly encourages entrepreneurs to develop through e-commerce. Lazada is one of the online stores in Indonesia that has many users and Lazada makes it easy to shop without the need to come to the place or directly. However, purchasing goods using e-commerce has problems regarding the quality of the goods you want to buy, therefore purchasing goods can be seen through reviews of each one you want to buy. Sentiment analysis is carried out using the Recurrent Neural Network (RNN) method with Long Short Term Memory (LSTM). And using the Epoch value as a parameter in processing validation data and test data to produce the best accuracy value

Cascade transformers with dynamic attention for video question answering

Visual question answering (VQA) has become a hot study topic with challenging motivation of correctly answering the videos or images questions in recent years. However, the existing VQA model mostly aimed at answering questions about images and performed poorly in the video question answering (VideoQA) domain. VideoQA needs to simultaneously consider the correlations between video frames and the dynamic information of multiple objects in video. Therefore, we propose a novel Cascade Transformers with Dynamic Attention for Video Question Answering (CTDA-QA), which aims to simultaneously solve the above considerations. Specifically, the proposed CTDA-QA model utilizes multiple transformers structure to encode videos for reasoning complex spatial and temporal information, which is different from the previous recurrent neural network methods. Besides, in order to effectively capture the dynamic information from various scenarios in videos, a flexible attention module has been proposed to explore the essential relations between objects in a dynamic timeline. Finally, to avoid spurious answers and fully explore the cross-modal relationships, a mixed-supervised learning strategy is designed for optimizing the reasoning tasks. The experiments on several benchmark video question–answer datasets clearly verify the performance and effectiveness of CTDA-QA, which contains the results in contrast to the state-of-the-art methods. Besides, the provided ablation study and visualization results further reveal the potential of CTDA-QA.

Deep Learning-Based Prediction Model for Gait Recovery after a Spinal Cord Injury.

Predicting gait recovery after a spinal cord injury (SCI) during an acute rehabilitation phase is important for planning rehabilitation strategies. However, few studies have been conducted on this topic to date. In this study, we developed a deep learning-based prediction model for gait recovery after SCI upon discharge from an acute rehabilitation facility. Data were collected from 405 patients with acute SCI admitted to the acute rehabilitation facility of Korea University Anam Hospital between June 2008 and December 2022. The dependent variable was Functional Ambulation Category at the time of discharge (FAC-DC). Seventy-one independent variables were selected from the existing literature: basic information, International Standards for Neurological Classification of SCI scores, neurogenic bladders, initial FAC, and somatosensory-evoked potentials of the lower extremity. Recurrent neural network (RNN), linear regression (LR), Ridge, and Lasso methods were compared for FAC-DC prediction in terms of the root-mean-squared error (RMSE). RNN variable importance, which is the RMSE gap between a complete RNN model and an RNN model excluding a certain variable, was used to evaluate the contribution of this variable. Based on the results of this study, the performance of the RNN was far better than that of LR, Ridge, and Lasso. The respective RMSEs were 0.3738, 2.2831, 1.3161, and 1.0246 for all the participants; 0.3727, 1.7176, 1.3914, and 1.3524 for those with trauma; and 0.3728, 1.7516, 1.1012, and 0.8889 for those without trauma. In terms of RNN variable importance, lower-extremity motor strength (right and left ankle dorsiflexors, right knee extensors, and left long toe extensors) and the neurological level of injury were ranked among the top five across the boards. Therefore, initial FAC was the seventh, third, and ninth most important predictor for all participants, those with trauma, and those without trauma, respectively. In conclusion, this study developed a deep learning-based prediction model with excellent performance for gait recovery after SCI at the time of discharge from an acute rehabilitation facility. This study also demonstrated the strength of deep learning as an explainable artificial intelligence method for identifying the most important predictors.

ANALISIS SENTIMEN VAKSIN COVID-19 PADA TWITTER MENGGUNAKAN RECURRENT NEURAL NETWORK (RNN) DENGAN ALGORITMA LONG SHORT-TERM MEMORY (LSTM)

The Coronavirus, also known as the Covid-19 pandemic, has reached every country worldwide, including Indonesia. Covid-19 is still prevalent and has killed many people in Indonesia. This makes it impossible to stop Covid-19 from spreading. The government's attempt to stop the Covid-19 pandemic is acquiring the vaccine. The administration of the Covid-19 vaccine has generated much discussion on social media, particularly Twitter. Tweets displaying public opinion on Twitter can be used for sentiment analysis and categorizing public opinion on the Covid-19 vaccine. 20,000 tweets were collected by Twitter crawling between January 10 and January 15, 2022. 3.290 tweets were left after pre-processing and meaningless tweets were eliminated. The data were processed using the Recurrent Neural Network method with the Long Short-Term Memory algorithm to determine its accuracy and identify topics often discussed by the public on Twitter. The LSTM method is capable of storing old information/data. A model with 70% training data, a learning rate of 0.01, 100 LSTM units, 32 batch sizes, 100 epochs, a cross-entropy loss function, and Adam optimizers was used to build the classification in this study. The accuracy value obtained from the performance evaluation of the Long Short-Term Memory model research was 80.34%.

Fault Diagnosis of Three-Phase Induction Motor (IM) Using a Hybrid ELSE-RNN Technique

This manuscript proposes a hybrid technique for fault detection and classification in a three-phase induction motor (IM). The proposed hybrid technique combines enhanced ladder spherical evaluation (ELSE) and recurrent neural networks (RNN); hence, it is called the ELSE–RNN method. The main goal of the manuscript is to detect and categorize the faults that occur in the IM. The key objective of the proposed method is to enhance the performance of supercapacitor (SC) storage technology and fuzzy-tuned proportional–integral (PI) supervision over conventional control. The main contribution of this paper is developing an effective fault diagnosis method for three-phase IMs. The presence of stator, rotor, winding, and bearing faults is employed using signal processing techniques, such as the Hilbert transform and SIFT. The proposed ELSE–RNN technique is utilized with the end goal of detecting and classifying faults. Here, the proposed ELSE–RNN technique recognizes motors’ healthy or unhealthy conditions in many situations to distinguish faults for protection. The proposed ELSE–RNN technique reduces the complexity of detecting and classifying faults with a validated system and increases the system's accuracy. The ELSE–RNN technique is implemented in MATLAB, and its performance is compared to existing techniques.

Diabetic Retinopathy Blood Vessel Detection Using CNN and RNN Techniques

This research aims to detect diabetic retinopathy using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The main objective is to compare these two methods in detecting the condition. Based on the study’s result after training 10 times on each method, the accuracy results were 92% for the CNN method and 50% for the RNN method. These results show, this study with the dataset used, the CNN method is much more effective in detecting diabetic retinopathy than the RNN method. The CNN method is better due to its ability to extract spatial features from images, which is important in image classification tasks.

Multivariat Predict Sales Data Using the Recurrent Neural Network (RNN) Method

Multivariat Predict Sales Data Using the Recurrent Neural Network (RNN) Method