Deep Learning Recurrent Neural Network Research Articles

Fuzzy Deep Learning Recurrent Neural Network Algorithm to Detect Corn Leaf Disease

Corn is a major commodity after rice in supporting food self-sufficiency in Indonesia. However, due to leaf disease, the quality and quantity of corn plants are greatly reduced. The problem with detecting corn leaf diseases is that the detection method is still manual, making it inefficient and ineffective. Therefore, in this study, disease detection on corn leaves was performed using the Fuzzy C-Means (FCM) and Long Short-Term Memory (LSTM) methods. First, oversampling was carried out to ensure an equal amount of data in all classes, then the corn leaf images were pre-processed before being input into the LSTM algorithm. After completing clustering process in the [Formula: see text] algorithm, the next step involved extracting texture features using the Gray Level Co-occurrence Matrix (GLCM) technique, followed by classification using LSTM. To assess their performance, both algorithms underwent evaluation using the k-fold cross-validation method, and their accuracy and speed were compared. The results of the k-fold cross-validation demonstrated that the [Formula: see text] algorithm achieved an accuracy of 63.53%, whereas the LSTM algorithm achieved an accuracy of 80.24%. In terms of the time required for training and prediction, the LSTM algorithm took 13[Formula: see text]min and 18[Formula: see text]s for training on corn leaf disease images, while the prediction process only took 1.59[Formula: see text]s. The training and prediction time required for the [Formula: see text] algorithm were 65[Formula: see text]min and 24[Formula: see text]s and 5[Formula: see text]min and 44[Formula: see text]s, respectively. The conclusion of this study is that the LSTM algorithm has better accuracy and time compared to [Formula: see text] on the dataset used in this study in terms of corn leaf disease detection.

Diabetic Drug ontology mapping for individual Diabetic Person and predict Insulin Dosage on Daily Basis

Managing diabetes requires a personalized treatment plan based on the patient's health records and drug usage effectively. In this paper, we have proposed accurate prediction of insulin dosage for diabetic patients daily by using ontology mapping with deep learning recurrent neural network models such as long short-term memory (LSTM), Gated recurrent unit( GRU), and Bidirectional long short-term memory(BiLSTM). The patient's information is arranged in a structured format. The structured ontology data provides a meaningful connection between patients' personal information, physical activity, insulin intake, carbohydrate intake and glucose level. The structured data is beneficial to take decisions effectively to recommend the insulin dosage level daily to avoid a sudden decrease in sugar level. The proposed LSTM model outperforms well on insulin prediction compared to GRU and BiLSTM models. The proposed model is very effective for diabetic personal care and improves patient outcomes and quality of life.

The Analysis of Deep Learning Recurrent Neural Network in English Grading Under the Internet of Things

The Analysis of Deep Learning Recurrent Neural Network in English Grading Under the Internet of Things

Motion prediction using brain waves based on artificial intelligence deep learning recurrent neural network.

Electroencephalogram (EEG) research has gained widespread use in various research domains due to its valuable insights into human body movements. In this study, we investigated the optimization of motion discrimination prediction by employing an artificial intelligence deep learning recurrent neural network (gated recurrent unit, GRU) on unique EEG data generated from specific movement types among EEG signals. The experiment involved participants categorized into five difficulty levels of postural control, targeting gymnasts in their twenties and college students majoring in physical education (n=10). Machine learning techniques were applied to extract brain-motor patterns from the collected EEG data, which consisted of 32 channels. The EEG data underwent spectrum analysis using fast Fourier transform conversion, and the GRU model network was utilized for machine learning on each EEG frequency domain, thereby improving the performance index of the learning operation process. Through the development of the GRU network algorithm, the performance index achieved up to a 15.92% improvement compared to the accuracy of existing models, resulting in motion recognition accuracy ranging from a minimum of 94.67% to a maximum of 99.15% between actual and predicted values. These optimization outcomes are attributed to the enhanced accuracy and cost function of the GRU network algorithm's hidden layers. By implementing motion identification optimization based on artificial intelligence machine learning results from EEG signals, this study contributes to the emerging field of exercise rehabilitation, presenting an innovative paradigm that reveals the interconnectedness between the brain and the science of exercise.

Memory Unlocks the Future of Biomolecular Dynamics: Transformative Tools to Uncover Physical Insights Accurately and Efficiently.

Conformational changes underpin function and encode complex biomolecular mechanisms. Gaining atomic-level detail of how such changes occur has the potential to reveal these mechanisms and is of critical importance in identifying drug targets, facilitating rational drug design, and enabling bioengineering applications. While the past two decades have brought Markov state model techniques to the point where practitioners can regularly use them to glimpse the long-time dynamics of slow conformations in complex systems, many systems are still beyond their reach. In this Perspective, we discuss how including memory (i.e., non-Markovian effects) can reduce the computational cost to predict the long-time dynamics in these complex systems by orders of magnitude and with greater accuracy and resolution than state-of-the-art Markov state models. We illustrate how memory lies at the heart of successful and promising techniques, ranging from the Fokker-Planck and generalized Langevin equations to deep-learning recurrent neural networks and generalized master equations. We delineate how these techniques work, identify insights that they can offer in biomolecular systems, and discuss their advantages and disadvantages in practical settings. We show how generalized master equations can enable the investigation of, for example, the gate-opening process in RNA polymerase II and demonstrate how our recent advances tame the deleterious influence of statistical underconvergence of the molecular dynamics simulations used to parameterize these techniques. This represents a significant leap forward that will enable our memory-based techniques to interrogate systems that are currently beyond the reach of even the best Markov state models. We conclude by discussing some current challenges and future prospects for how exploiting memory will open the door to many exciting opportunities.

LSTM and GRU based Accurate Water Quality Prediction for Smart Aquaculture

The fast and precise water quality prediction in smart aquaculture can assist the farmers in taking corrective actions before the ecological environment deteriorates. However, the empirical methods take high computation time and provide low accuracy in water quality prediction due to the non-linear and dynamic nature of water quality parameters. This research work proposes long short-term memory (LSTM) and gated recurrent unit (GRU) deep learning recurrent neural network (DL-RNN) models for aquaculture water quality prediction (A-WQP). This work also presents an extensive study about the impact of hyper-parameters (hp ) on the performance of the proposed DL-RNN models using two different water quality datasets. The experimental evaluations show that for the optimal sets of hyper-parameters, the proposed DL models offer superior prediction accuracy and computation efficiency.

Fog-cloud based intrusion detection system using Recurrent Neural Networks and feature selection for IoT networks

Deep learning (DL) techniques are being widely researched for their effectiveness in detecting cyber intrusions against the Internet of Things (IoT). Time sensitive Critical Infrastructures (CIs) that rely on IoT require rapid detection of cyber intrusions close to the constrained devices in order to prevent service delays. Deep learning techniques perform better in detecting attacks compared to shallow machine learning algorithms and can be used for intrusion detection. However, communication overheads due to large volume of IoT data and computation requirements for deep learning models prevents effective application of deep learning models closer to the constrained devices. Existing IDS techniques are either based on shallow learning algorithms or not trained on relevant IoT datasets and furthermore not designed for distributed fog-cloud deployment. To counter these issues, we propose a novel fog-cloud based IoT intrusion detection framework which incorporates a distributed processing by splitting the dataset according to attack class and a feature selection step on time-series IoT data. This is followed by a deep learning Recurrent Neural Network (SimpleRNN and Bi-directional Long Short-Term Memory (LSTM)) for attack detection. The effectiveness of the proposed approach was evaluated using the high-dimensional BoT-IoT dataset which contains large volumes of realistic IoT attack traffic. Results show that feature selection methods significantly reduced the dataset size by 90% under the computation requirements without compromising on the attack detection ability. The models built on reduced dataset achieved higher recall rate compared to models trained on full feature set without loosing class differentiation ability. The SimpleRNN and Bi-LSTM models also did not suffer any underfitting or overfitting with the reduced feature space. The proposed deep learning based IoT intrusion detection framework is suitable for fog-cloud based deployment and can scale well even with large volumes of IoT data.

Trapezoidal pile-up nuclear pulse parameter identification method based on deep learning transformer model

Pile-up between adjacent nuclear pulses is unavoidable in the actual detection process. Some scholars have tried to apply deep learning techniques to identify pile-up nuclear pulse parameters. However, traditional deep learning recurrent neural networks (RNNs) suffer from inefficient pulse recognition and poor recognition of pile-up nuclear pulses with short intervals between adjacent pulses. In this paper, a Transformer model with an attention mechanism as the core to recognize pile-up nuclear pulses is innovatively applied, aiming to provide a more accurate and efficient method for pile-up nuclear pulse recognition. Thus, it gives a better help for the spectrum correction with a high count rate.

Denial-of-Sleep Attack Detection in NB-IoT Using Deep Learning

With increasing Internet-of-Things (IoT) protocols and connectivity, a growing number of attacks are emerging in the associated networks. This work presents approaches using deep learning (DL) to detect attacks in an IoT environment, particularly in narrowband Internet-of-Things (NB-IoT). By virtue of its low cost, low complexity and limited energy, an NB-IoT device will not likely permit cutting-edge security mechanisms, leaving it vulnerable to, for example, denial-of-sleep (DoSl) attacks. For performance analysis, a NB-IoT network was simulated, using ns-3, to generate a novel dataset to represent an implementation of DoSl attacks. After preprocessing, the dataset was presented to a collection of machine learning (ML) models to evaluate their performance. The considered DL recurrent neural network (RNN) models have proven capable of reliably classifying traffic, with very high accuracy, into either a DoSl attack or a normal record. The performance of a long short-term memory (LSTM) classifier has provided accuracies up to 98.99%, with a detection time of 2.54 x 10-5 second/record, surpassing performance of a gated recurrent unit (GRU). RNN DL models have superior performance in terms of accuracy of detecting DoSl attacks in NB-IoT networks, when compared with other ML algorithms, including support vector machine, Gaussian naïve-Bayes, and logistic regression.

Study on Intelligent Online Piano Teaching System Based on Deep Learning Recurrent Neural Network Model

This study has been conducted to solve the problem of repetitive piano lessons and to bring a personalized experience for each piano learner. The application of deep learning (DL) technology for children’s piano teaching has a positive effect on their interest in the subject and improves the teaching quality. Music instruments were identified in the system using an instrument recognition model that was developed using deep learning techniques. It was also utilized to help children learn to play the piano by giving them direction and boosting their excitement for it. The proposed model’s ability to recognize and acquire features has been improved. The recurrent neural network (RNN) demonstrated instrument recognition accuracy of 96.4%, and the model’s recognition error rate decreased and stabilized as the number of iterations increased. The proposed RNN for musical instruments recognizes instruments by using DL to accurately identify musical properties.

Deep Learning for Multi-Aspect Sentiment Analysis of TikTok App using the RNN-LSTM Method

Applications built expressly for consumers to communicate online are known as social media apps. Social media applications are utilized for enjoyment as well as for interacting. For Android users, applications may be found in the Google Play Store, while for iOS users, they can be found in the Apple App Store. The site offers a collection that is a big resource-rich in thoughts, opinions, and feelings, notably on Google Playstore. Each user's review has an aspect value. Due to a large number of reviews, sentiment analysis is tough. The author proposes to do an Aspect-Based Sentiment Analysis (ABSA) utilizing TikTok app reviews on the Google Play Store in this paper. Currently, there are 65.2 million active users of the Tik Tok program, including 8.5 million users from Indonesia, there are still a few studies that use the TikTok application dataset. In this study, sentiment classification is carried out on each aspect that has been determined, namely, aspects of features, business, and content, the method used is deep learning Recurrent Neural Network with the Long Short-Term Memory (RNN – LSTM) model and the addition of word embedding BERT. The results showed that the classification of sentiment in the business aspect showed the highest score, namely 0.94, the sentiment classification in the aspect received an accuracy of 0.91 while the feature aspect got the lowest accuracy, which was 0.85.

Deep Learning Neural Network Prediction System Enhanced with Best Window Size in Sliding Window Algorithm for Predicting Domestic Power Consumption in a Residential Building.

Buildings are considered to be one of the world's largest consumers of energy. The productive utilization of energy will spare the accessible energy assets for the following ages. In this paper, we analyze and predict the domestic electric power consumption of a single residential building, implementing deep learning approach (LSTM and CNN). In these models, a novel feature is proposed, the “best N window size” that will focus on identifying the reliable time period in the past data, which yields an optimal prediction model for domestic energy consumption known as deep learning recurrent neural network prediction system with improved sliding window algorithm. The proposed prediction system is tuned to achieve high accuracy based on various hyperparameters. This work performs a comparative study of different variations of the deep learning model and records the best Root Mean Square Error value compared to other learning models for the benchmark energy consumption dataset.

Multi-step-ahead prediction of river flow using NARX neural networks and deep learning LSTM

Abstract Kelantan river (Sungai Kelantan in Malaysia) basin is one of the essential catchments as it has a history of flood events. Numerous studies have been conducted in river basin modelling for the prediction of flow and mitigation of flooding events as well as water resource management. Therefore, having multi-step-ahead forecasting for river flow (RF) is of important research interest in this regard. This study presents four different approaches for multi-step-ahead forecasting for the Kelantan RF, using NARX (nonlinear autoregressive with exogenous inputs) neural networks and deep learning recurrent neural networks called LSTM (long short-term memory). The dataset used was obtained in monthly record for 29 years between January 1988 and December 2016. The results show that two recursive methods using NARX and LSTM are able to do multi-step-ahead forecasting on 52 series of test datasets with NSE (Nash–Sutcliffe efficiency coefficient) values of 0.44 and 0.59 for NARX and LSTM, respectively. For few-step-ahead forecasting, LSTM with direct sequence-to-sequence produces promising results with a good NSE value of 0.75 (in case of two-step-ahead forecasting). However, it needs a larger data size to have better performance in longer-step-ahead forecasting. Compared with other studies, the data used in this study is much smaller.

Abd-Rnn: Attention Based Bi-Directional Deep Learning Recurrent Neural Networks for Aquaculture Water Quality Prediction

Abd-Rnn: Attention Based Bi-Directional Deep Learning Recurrent Neural Networks for Aquaculture Water Quality Prediction

Evaluation of deep learning techniques for identification of sarcoma-causing carcinogenic mutations.

The abnormal growth of human healthy cells is called cancer. One of the major types of cancer is sarcoma, mostly found in human bones and soft tissue cells. It commonly occurs in children. According to a survey of the United States of America, there are more than 17,000 sarcoma patients registered each year which is 15% of all cancer cases. Recognition of cancer at its early stage saves many lives. The proposed study developed a framework for the early detection of human sarcoma cancer using deep learning Recurrent Neural Network (RNN) algorithms. The DNA of a human cell is made up of 25,000 to 30,000 genes. Each gene is represented by sequences of nucleotides. The nucleotides in a sequence of a driver gene can change which is termed as mutations. Some mutations can cause cancer. There are seven types of a gene whose mutation causes sarcoma cancer. The study uses the dataset which has been taken from more than 134 samples and includes 141 mutations in 8 driver genes. On these gene sequences RNN algorithms Long and Short-Term Memory (LSTM), Gated Recurrent Units and Bi-directional LSTM (Bi-LSTM) are used for training. Rigorous testing techniques such as Self-consistency testing, independent set testing, 10-fold cross-validation test are applied for the validation of results. These validation techniques yield several metrics such as Area Under the Curve (AUC), sensitivity, specificity, Mathew's correlation coefficient, loss, and accuracy. The proposed algorithm exhibits an accuracy of 99.6% with an AUC value of 1.00.

Ultrasound Elastography under Deep Learning Algorithm to Analyze the Therapeutic Effect of Clustered Regularly Interspaced Short Palindromic Repeats Short Hairpin Ribonucleic Acid Nanoparticles on Cervical Cancer.

This study aimed to analyze the effect of the deep learning algorithm on ultrasound elastography on the treatment of cervical cancer with clustered regularly interspaced short palindromic repeats (CRISPR) short hairpin ribonucleic acid (shRNA) nanoparticles, aiming to provide a reference for the clinical application of deep learning to analyze the therapeutic effect of the disease. In this study, CRISPR and shRNA plasmid nanoparticle drugs were used to treat 55 patients with cervical cancer in the experimental group, and normal saline was injected to another 53 patients in the control group, so compare the effect of nanoparticles in the treatment of cervical cancer. Professional doctors and the recurrent neural network (RNN) intelligent algorithm were used to score cervical cancer based on the ultrasound elastograph images by taking blue, green, and red (BGR) as diagnosis criteria. As a result, the experimental group had a total of 217 points before drug administration and a total of 224 points after drug administration. Each patient had an average increase of 0.13 points. The control group had a total of 200 points before drug administration and a total of 223 points after drug administration, and each patient had an average increase of 0.43 points. The experimental group was obviously different from the control group (P < 0.05). Each tissue image output by the RNN was clearer than the original image, and the score given by intelligent calculation was faster than that of professional doctors. The monitoring effect of the deep learning RNN intelligent algorithm on the therapeutic effect of nanomedicine was analyzed. It was found that the average accuracy of the experimental group and the control group was 98.95% and 90.34%, respectively; and the experimental group was greatly different from the control group (P < 0.05). In short, nano-CRISPR and shRNA drugs had remarkable effects on the treatment of cervical cancer, and the scores given by the deep learning intelligent algorithm were faster and more accurate, which provided theoretical guidance for the clinical application of deep learning algorithms to analyze the treatment effects of diseases.

Deep Learning Recurrent Neural Network for Concussion Classification in Adolescents Using Raw Electroencephalography Signals: Toward a Minimal Number of Sensors.

Artificial neural networks (ANNs) are showing increasing promise as decision support tools in medicine and particularly in neuroscience and neuroimaging. Recently, there has been increasing work on using neural networks to classify individuals with concussion using electroencephalography (EEG) data. However, to date the need for research grade equipment has limited the applications to clinical environments. We recently developed a deep learning long short-term memory (LSTM) based recurrent neural network to classify concussion using raw, resting state data using 64 EEG channels and achieved high accuracy in classifying concussion. Here, we report on our efforts to develop a clinically practical system using a minimal subset of EEG sensors. EEG data from 23 athletes who had suffered a sport-related concussion and 35 non-concussed, control athletes were used for this study. We tested and ranked each of the original 64 channels based on its contribution toward the concussion classification performed by the original LSTM network. The top scoring channels were used to train and test a network with the same architecture as the previously trained network. We found that with only six of the top scoring channels the classifier identified concussions with an accuracy of 94%. These results show that it is possible to classify concussion using raw, resting state data from a small number of EEG sensors, constituting a first step toward developing portable, easy to use EEG systems that can be used in a clinical setting.

Online Machine Learning of Available Capacity for Vehicle-to-Grid Services during the Coronavirus Pandemic

Vehicle-to-grid services make use of the aggregated capacity available from a fleet of vehicles to participate in energy markets, help integrate renewable energy in the grid and balance energy use. In this paper, the critical components of such a service are described in the context of a commercial service that is currently under development. Key among these components is the prediction of available capacity at a future time. In this paper, we extend a previous work that used a deep learning recurrent neural network for this task to include online machine learning, which enables the network to continually refine its predictions based on observed behaviour. The coronavirus pandemic that was declared in 2020 resulted in closures of the university and substantial changes to the behaviour of the university fleet. In this work, the impact of this change in vehicles usage was used to test the predictions of a network initially trained using vehicle trip data from 2019 with and without online machine learning. It is shown that prediction error is significantly reduced using online machine learning, and it is concluded that a similar capability will be of critical importance for a commercial service such as the one described in this paper.

Multivariate CDS risk premium prediction with SOTA RNNs on MI[N]T countries

In this study, CDS risk premiums of Mexico, Indonesia and Turkey were predicted by applying state-of-the-art forecasters in deep learning recurrent neural networks architectures which are the most recent ground-breaking predictors in the time series setting. The predictive power of each sota forecaster is compared, and the results are differentiated by country and type of sota predictors. While the long short-term memory model is better to predict Mexico’s CDS risk premiums, the nonlinear autoregressive network with exogenous inputs model is found to be more suitable for Indonesia and Turkey. The results of Turkey model reached the highest forecast accuracy.

Land Use Change Ontology and Traffic Prediction through Recurrent Neural Networks: A Case Study in Calgary, Canada

Land use and transportation planning have a significant impact on the performance of cities’ traffic conditions and the quality of people’s lives. The changing characteristics of land use will affect and challenge how a city is able to manage, organize, and plan for new developments and transportation. These challenges can be better addressed with effective methods of monitoring and predicting, which can enable optimal efficiency in how a growing city like Calgary, Canada, can perform. Using ontology in land use planning is a new initiative currently being researched and explored. In this regard, ontology incorporates relationships between the various entities of land use. The aim of this study is to present Land Use Change Ontology (LUCO) with a deep neural network for traffic prediction. We present a Land Use Change Ontology (LUCO) approach, using expressions of how the semantics of land use changes relate to the integration of temporal land use information. This study examines the City of Calgary’s land use data from the years 2001, 2010, and 2015. In applying the LUCO approach to test data, experimental outcomes indicated that from 2001 to 2015 residential land use increased by 30% and open space decreased by 40%. Forecasting traffic is increasingly essential for successful traffic modelling, operations, and management. However, traditional means for predicting traffic flow have largely assumed restrictive model architectures that have not controlled for the amounts of land use change. Inspired by deep learning methods and effective data mining computing capabilities, this paper introduces the deep learning Recurrent Neural Network (RNN) to predict traffic while considering the impact of land use change. The RNN was successful in learning the features of traffic flow under various land use change situations. Experimental results indicated that, with the consideration of LUCO, the deep learning predictors had better accuracy when compared with other existing models. Success of our modeling approach indicates that cities could apply this modeling approach to make land use transportation planning more efficient.