Deep Gated Recurrent Unit Research Articles

Daily Runoff Prediction with a Seasonal Decomposition-Based Deep GRU Method

Accurately predicting hydrological runoff is crucial for water resource allocation and power station scheduling. However, there is no perfect model that can accurately predict future runoff. In this paper, a daily runoff prediction method with a seasonal decomposition-based-deep gated-recurrent-unit (GRU) method (SD-GRU) is proposed. The raw data is preprocessed and then decomposed into trend, seasonal, and residual components using the seasonal decomposition algorithm. The deep GRU model is then used to predict each subcomponent, which is then integrated into the final prediction results. In particular, the hyperparameter optimization algorithm of tree-structured parzen estimators (TPE) is used to optimize the model. Moreover, this paper introduces the single machine learning model (including multiple linear regression (MLR), back propagation (BP), long short-term memory neural network (LSTM) and gate recurrent unit (GRU)) and a combination model (including seasonal decomposition–back propagation (SD-BP), seasonal decomposition–multiple linear regression (SD-MLR), along with seasonal decomposition–long-and-short-term-memory neural network (SD-LSTM), which are used as comparison models to verify the excellent prediction performance of the proposed model. Finally, a case study of the Qingjiang Shuibuya test set, which considers the period 1 January 2019 to 31 December 2019, is conducted. Case studies of the Qingjiang River show the proposed model outperformed the other models in prediction performance. The model achieved a mean absolute error (MAE) index of 38.5, a Nash-Sutcliffe efficiency (NSE) index of 0.93, and a coefficient of determination (R2) index of 0.7. In addition, compared to the comparison model, the NSE index of the proposed model increased by 19.2%, 19.2%, 16.3%, 16.3%, 2.2%, 2.2%, and 1.1%, when compared to BP, MLR, LSTM, GRU, SD-BP, SD-MLR, SD-LSTM, and SD-GRU, respectively. This research can provide an essential reference for the study of daily runoff prediction models.

Single and multi-crop species disease detection using ITSO based gated recurrent multi-attention neural network

<p>Diseases of crop plants pose a serious danger to agricultural output and progress. Predicting the onset of a disease outbreak in advance can help public health officials better manage the pandemic. Precision agriculture (PA) applications rely heavily on current information and communication technologies (ICTs) for their contribution to long-term sustainability. Preventative measures against plant diseases require accurate early disease prediction in order to be effective. The current computer vision-based illness detection technology can only detect the disease after it has already manifested. This research intends to provide a deep learning (DL) method for early disease attack prediction using Internet of Things (IoT) directly sensed environmental factors from crop fields. There is a robust relationship between environmental factors and the life cycles of plant diseases. Disease incidence in plants can be forecast based on environmental variables in the crop field. In order to solve these issues, the research presented here suggests using a gated recurrent multi-attention neural network (GRMA-Net). The study uses multilevel modules to zero down on informative areas in order to extract additional discriminative features, as informative characteristics tend to appear at various levels in a network. In order to capture long-range dependence and contextual interaction, these characteristics are first organised as spatial sequences and then input into a deep-gated recurrent unit (GRU). Finally, an enhanced version of the Tunicate swarm optimisation model (ITSO) is used to pick the best values for the model’s hyper-parameters. Four public datasets representing a wide range of crop types are used to assess the model’s efficacy. Some of these databases cover numerous crop species, like PlantVillage (38 categories), while others focus on a single crop, such as Apple (4), Maize (4), or Rice (5). The experimental findings show that the system achieves 99.16% accuracy in identifying agricultural diseases, which is higher than the accuracy of other current deep-learning approaches.</p>

Evolving deep gated recurrent unit using improved marine predator algorithm for profit prediction based on financial accounting information system

This research proposes a hybrid improved marine predator algorithm (IMPA) and deep gated recurrent unit (DGRU) model for profit prediction in financial accounting information systems (FAIS). The study addresses the challenge of real-time processing performance caused by the increasing complexity of hybrid networks due to the growing size of datasets. To enable effective comparison, a new dataset is created using 15 input parameters from the original Chinese stock market Kaggle dataset. Additionally, five DGRU-based models are developed, including chaotic MPA (CMPA) and the nonlinear MPA (NMPA), as well as the best Levy-based variants, such as the dynamic Levy flight chimp optimization algorithm (DLFCHOA) and the Levy-base gray wolf optimization algorithm (LGWO). The results indicate that the most accurate model for profit forecasting among the tested algorithms is DGRU-IMPA, followed by DGRU-NMPA, DGRU-LGWO, DGRU-DLFCHOA, DGRU-CMPA, and traditional DGRU. The findings highlight the potential of the proposed hybrid model to improve profit prediction accuracy in FAIS, leading to enhanced decision-making and financial management.

A Deep Stacked Bidirectional LSTM (SBiLSTM) Model for Petroleum Production Forecasting

Deep learning models have demonstrated outstanding forecasting effects and are extensively used in forecasting problems in numerous scenarios. The Long short-term memory neural network (LSTM), a deep learning variant, has huge potential in forecasting the forthcoming values of given patterns using historical time series data. Lately, the forecasting task has captivated the focus of researchers in deep learning areas to handle the constraints of traditional statistical approaches. Owing to the dire need for accurate forecasting and also due to technological advancements, the collection of time series data has become more accessible, and this has paved the way for deep forecasting models. In this paper, we propose the Stacked Bi-LSTM (SBiLSTM) architecture, an adaptation of the traditional deep long-short term memory (TDLM). The approach is evaluated using two oilfield production time series. The performance of the proposed SBiLSTM model is compared with the recurrent neural network-RNNs, multi-layer RNNs, deep gated recurrent unit (DGRU), and deep long-short term memory (DLSTM). Using different measurement criteria, the empirical results show that the proposed DLSTM model outperforms other standard approaches. It is observed that the proposed SBiLSTM model acquires long and short-range interdependent features of univariate time series data without the large memory requirement.

Prognostics and health management for piezoresistive pressure sensor based on improved gated recurrent unit networks

Application fields for piezoresistive pressure sensors have become increasingly extensive in recent years. So a high reliability of the sensor is also required. However, considering that some sensors operate in hostile environments,the need to ensure continuous operation accuracy, prognostics, and health management (PHM) for piezoresistive pressure sensors should not be ignored. To solve this problem, a fault diagnosis and prognostic method that combines a support vector machine (SVM) and deep gated recurrent unit (DGRU) network optimized by hunter–prey optimization (HPO) is proposed in this paper. First, three sensor fault types are defined. Second, SVM is adopted to realize the fault diagnosis. Third, two layers of DGRU are employed to predict the health index, which is defined to represent the health state of the sensor. Meanwhile, the optimal parameters of the DGRU are optimized by HPO algorithms. Finally, the remaining useful life can be estimated by the predicted health index and failure threshold. The method proposed in this paper is proved to be effective and accurate. The fault diagnosis accuracy is 100% in the three fault types defined by this paper. The minimum mean absolute error is 6 × 10−5. It proves the proposed method of PHM in this paper is viable in a real application.

An Enhanced Gated Recurrent Unit-Based Adaptive Fault Diagnosis of Rotating Machinery

As the most basic component of rotating machinery, rolling bearing frequently works in harsh environments and complex working conditions, and its health status affects seriously the working efficiency. The health statuses of rolling bearing can not only reduce equipment maintenance costs but also contribute to reducing major accidents. Based on this, an adaptive diagnosis method that combines deep gated recurrent unit (DGRU) with wavelet packet decomposition (WPD) and extreme learning machine (ELM) is proposed for rolling bearing. Firstly, WPD is utilized to eliminate the noise of data. Secondly, DGRU is designed to extract the representative features of denoised data. Finally, ELM is utilized to output the diagnosis results. Massive results prove that the superiority and robustness of our approach outperform existing popular methods. Additionally, the proposed method can also achieve powerful antinoise ability.

Resnet based Deep Gated Recurrent Unit for Image Captioning on Smartphone

Image captioning aims at generating grammatically and semantically acceptable natural language sentences for visual contents. Gated recurrent units (GRU) based approaches have recently attracted much attention due to their performance in caption generation. Challenges with GRU are to deal with vanishing gradient problems and modulation of the most relevant information flow in deep networks. In this paper, we propose a resnet-based deep GRU approach to overcome the vanishing gradient problem with residual connections while the most relevant information is ensured to flow using multiple layers of GRU. Residual connections are employed between consecutive layers of deep GRU, which improves the gradient flow from lower to upper layers. Experimental investigations on the publicly available MSCOCO dataset prove that the proposed approach achieves comparable performance with some state-of-the-art approaches. Moreover, the approach is embedded into our custom-designed Android application, CaptionEye, which offers the ability to generate captions without an internet connection under a voice user interface.

Gated Recurrent Multiattention Network for VHR Remote Sensing Image Classification

With the advances of deep learning, many recent CNN-based methods have yielded promising results for image classification. In very high-resolution (VHR) remote sensing images, the contributions of different regions to image classification can vary significantly, because informative areas are generally limited and scattered throughout the whole image. Therefore, how to pay more attention to these informative areas and better incorporate them over long distances are two main challenges to be addressed. In this article, we propose a gated recurrent multiattention neural network (GRMA-Net) to address these problems. Because informative features generally occur at multiple stages in a network (i.e., local texture features at shallow layers and global profile features at deep layers), we use multilevel attention modules to focus on informative regions to extract more discriminative features. Then, these features are arranged as spatial sequences and fed into a deep-gated recurrent unit (GRU) to capture long-range dependency and contextual relationship. We evaluate our method on the UC Merced (UCM), Aerial Image dataset (AID), NWPU-RESISC (NWPU), and Optimal-31 (Optimal) datasets. Experimental results have demonstrated the superior performance of our method as compared to other state-of-the-art methods.

A Hybrid Mechanism- and Data-Driven Soft Sensor Based on the Generative Adversarial Network and Gated Recurrent Unit

As an effective means of sensing difficult-to-measure process variables in real time, soft sensors are widely used but have a few significant limitations. Modeling errors between the mechanism model and real system can occur, which affect the accuracy of the mechanism-driven sensing result. Furthermore, deep learning-based data-driven soft sensors are complex black boxes, resulting in a lack of interpretability in terms of the established model and a lack of reliability regarding the sensing results. To solve these problems, this paper introduces the generative adversarial network (GAN) for soft sensor modeling and establishes an innovative GAN-based hybrid mechanism- and data-driven soft sensor framework. Meanwhile, considering the dynamic characteristics of the industrial process, a deep gated recurrent unit (DGRU) was adopted to compensate for the modeling errors in the mechanism model. This deep learning-based data-driven model not only captures the timing relationship between sensor data but also uses numerous unlabeled data. The generator to be identified in the GAN consists of the DGRU and mechanism model. In an industrial case for predicting the rotor thermal deformation of an air preheater in a power station boiler, the effectiveness and superiority of the hybrid-driven dynamic soft sensor model were verified, and the post hoc interpretability of the model was explained by manipulating the latent variables.

Deep-gated recurrent unit and diet network-based genome-wide association analysis for detecting the biomarkers of Alzheimer's disease

Genome-wide association analysis (GWAS) is a commonly used method to detect the potential biomarkers of Alzheimer's disease (AD). Most existing GWAS methods entail a high computational cost, disregard correlations among imaging data and correlations among genetic data, and ignore various associations between longitudinal imaging and genetic data. A novel GWAS method was proposed to identify potential AD biomarkers and address these problems. A network based on a gated recurrent unit was applied without imputing incomplete longitudinal imaging data to integrate the longitudinal data of variable lengths and extract an image representation. In this study, a modified diet network that can considerably reduce the number of parameters in the genetic network was proposed to perform GWAS between image representation and genetic data. Genetic representation can be extracted in this way. A link between genetic representation and AD was established to detect potential AD biomarkers. The proposed method was tested on a set of simulated data and a real AD dataset. Results of the simulated data showed that the proposed method can accurately detect relevant biomarkers. Moreover, the results of real AD dataset showed that the proposed method can detect some new risk-related genes of AD. Based on previous reports, no research has incorporated a deep-learning model into a GWAS framework to investigate the potential information on super-high-dimensional genetic data and longitudinal imaging data and create a link between imaging genetics and AD for detecting potential AD biomarkers. Therefore, the proposed method may provide new insights into the underlying pathological mechanism of AD.

A novel wide & deep transfer learning stacked GRU framework for network intrusion detection

With the increasing frequency, severity and complexity of recent cyber attacks around the world, network intrusion detection has become mandatory and highly sophisticated task. Achieving high performance in network intrusion detection by applying benchmark machine learning classifiers (including deep learning techniques) has become a major challenge in recent times. One of the biggest challenges is improving the memorization capacity and generalization ability of NIDS (Network Intrusion Detection Systems). In this paper, we propose a highly scalable novel wide & deep transfer learning (TL) based stacked GRU (Gated Recurrent Unit) model to deal with multi-dimensional point data and multi-variate time series regression and classification problems in network intrusion detection. The proposed model has the memorization capacity of linear regression model and the generalization ability of deep GRU model. The deep component consists of a transfer learning framework that pretrains a source model and then fine-tunes the whole source model on the same dataset multiple times until it gives peak performance. This method gives a multi-class evaluation accuracy of 99.92% on KDDCup 99(10%) dataset and 94.22% on UNSW-NB15 dataset respectively. Extensive experimentations and evaluations have been carried out by comparing it with other machine learning (including deep learning) network intrusion detection techniques. The proposed method outperforms most of the existing intrusion detection approaches.

Generative adversarial networks based remaining useful life estimation for IIoT

Artificial intelligence (AI) and Predictive Maintenance (PdM) become productive using IIoT-data with zero-downtime for maintenance in industries by estimating the remaining useful life (RUL). Most reported works consider training data availability with an equal number of normal and fault samples concerning different machine health conditions. However, practical scenarios have to deal with fault-data unavailability, resulting in an imbalanced training dataset. This problem can lead to inaccuracies with missed fault-prediction in RUL estimation approaches. This paper proposes a novel prognostics framework based on conditional generative adversarial network (CGAN) and deep gated recurrent unit (DGRU) network. The framework can generate multi-variate fault instances, solve data imbalance, and predict the RUL of complex systems with the least latency. We observed that the learning of fault samples using underlying noise distribution, data augmentation, and training DGRU improves the RUL prediction accuracy by at least 15% compared to reported imbalanced work on the C-MAPSS dataset.

A Deep Neural Network Model for Speaker Identification

Speaker identification is a classification task which aims to identify a subject from a given time-series sequential data. Since the speech signal is a continuous one-dimensional time series, most of the current research methods are based on convolutional neural network (CNN) or recurrent neural network (RNN). Indeed, these methods perform well in many tasks, but there is no attempt to combine these two network models to study the speaker identification task. Due to the spectrogram that a speech signal contains, the spatial features of voiceprint (which corresponds to the voice spectrum) and CNN are effective for spatial feature extraction (which corresponds to modeling spectral correlations in acoustic features). At the same time, the speech signal is in a time series, and deep RNN can better represent long utterances than shallow networks. Considering the advantage of gated recurrent unit (GRU) (compared with traditional RNN) in the segmentation of sequence data, we decide to use stacked GRU layers in our model for frame-level feature extraction. In this paper, we propose a deep neural network (DNN) model based on a two-dimensional convolutional neural network (2-D CNN) and gated recurrent unit (GRU) for speaker identification. In the network model design, the convolutional layer is used for voiceprint feature extraction and reduces dimensionality in both the time and frequency domains, allowing for faster GRU layer computation. In addition, the stacked GRU recurrent network layers can learn a speaker’s acoustic features. During this research, we tried to use various neural network structures, including 2-D CNN, deep RNN, and deep LSTM. The above network models were evaluated on the Aishell-1 speech dataset. The experimental results showed that our proposed DNN model, which we call deep GRU, achieved a high recognition accuracy of 98.96%. At the same time, the results also demonstrate the effectiveness of the proposed deep GRU network model versus other models for speaker identification. Through further optimization, this method could be applied to other research similar to the study of speaker identification.

FallDeF5: A Fall Detection Framework Using 5G-Based Deep Gated Recurrent Unit Networks

Fall prevalence is high among elderly people, which is challenging due to the severe consequences of falling. This is why rapid assistance is a critical task. Ambient assisted living (AAL) uses recent technologies such as 5G networks and the internet of medical things (IoMT) to address this research area. Edge computing can reduce the cost of cloud communication, including high latency and bandwidth use, by moving conventional healthcare services and applications closer to end-users. Artificial intelligence (AI) techniques such as deep learning (DL) have been used recently for automatic fall detection, as well as supporting healthcare services. However, DL requires a vast amount of data and substantial processing power to improve its performance for the IoMT linked to the traditional edge computing environment. This research proposes an effective fall detection framework based on DL algorithms and mobile edge computing (MEC) within 5G wireless networks, the aim being to empower IoMT-based healthcare applications. We also propose the use of a deep gated recurrent unit (DGRU) neural network to improve the accuracy of existing DL-based fall detection methods. DGRU has the advantage of dealing with time-series IoMT data, and it can reduce the number of parameters and avoid the vanishing gradient problem. The experimental results on two public datasets show that the DGRU model of the proposed framework achieves higher accuracy rates compared to the current related works on the same datasets.

DGRU based human activity recognition using channel state information

In this work, we have proposed a Deep Gated Recurrent Unit (DGRU) model for non-obtrusive human activity recognition using Channel State Information (CSI). Empirical model decomposition is used for de-noising, whereas discrete wavelet transforms and linear discriminant analysis are used for feature extraction and dimensionality reduction, respectively. For extensive experimental evaluation and comparative analysis, a Software Defined Radio (SDR) platform is used by implementing IEEE 802.11a on National Instruments’ Universal Software Radio Peripheral (USRP). The physical layer CSI is collected in an indoor environment to evaluate the performance for seven activities. 30 volunteers including both genders and of different age groups were involved in the data collection process. As demonstrated through experiments, the proposed scheme achieves promising results with an accuracy of 95–99% for all activities, outperforming the traditional benchmark approaches in the literature that use random forest and more advanced deep learning techniques, such as Long-Short Term Memory (LSTM).

LPG-model: A novel model for throughput prediction in stream processing, using a light gradient boosting machine, incremental principal component analysis, and deep gated recurrent unit network

In recent years, the volume and velocity of streaming data have been increasing rapidly. Thus, real-time processing scenarios for streaming data have continued to increase. Stream processing tasks face huge challenges in areas such as load optimization, task scheduling, and resource management. Throughput prediction for stream processing tasks is a key technology in these areas. To predict the throughput of stream processing tasks accurately and efficiently, we propose a novel model named the LPG-model. It includes three main components: a light gradient boosting machine (LightGBM), incremental principal component analysis (IPCA), and an evolving deep gated recurrent unit (GRU) network. Unlike existing state-of-the-art models, the LPG-model not only offers a network structure adaptation mechanism (hidden layer adaptation mechanism), but also provides feature processing mechanisms for streaming data. Data preprocessing provides an interpolation method for missing values through an incremental interpolation mechanism and two normalization methods for features through incremental normalization mechanisms. An efficient dimensionality reduction mechanism provided by the LightGBM and IPCA is used to improve the prediction efficiency of the LPG-model. The hidden layer growing mechanism of the evolving deep GRU network is capable of learning new knowledge and maintaining previous knowledge from data streams. Moreover, it also has the ability to capture the temporal aspects of the data streams. The experimental results from four open-source benchmarks illustrate that the LPG-model is more accurate and efficient than state-of-the-art algorithms or networks, under the prequential test-then-train protocol. This proves the effectiveness of the LPG-model in throughput prediction scenarios for stream processing tasks. Furthermore, the numerical results from standard benchmark problems of data streams indicate that the LPG-model has potential to reduce the execution time of high-dimensional data streams with a high classification accuracy.

A Deep Gated Recurrent Unit based model for wireless intrusion detection system

Abstract With the advances and growth of various wireless technologies, it is imperative to implement robust Intrusion Detection Systems (IDS). This paper proposes the implementation of Deep Gated Recurrent Unit (DGRU) Based classifier as well as a wrapper-based feature extraction algorithm for Wireless IDS. We assess the performance of the DRGU IDS using the NSL-KDD benchmark dataset. Furthermore, we compare our framework to several popular algorithms including Artificial Neural Networks, Deep Long–Short Term Memory, Random Forest, Naive Bayes and Feed Forward Deep Neural Networks. The experimental outcomes demonstrate that the DGRU IDS displays a significant increase in performance over existing methods.

Attention-Based Pose Sequence Machine for 3D Hand Pose Estimation

Most of the existing methods for 3D hand pose estimation are performed from a single depth map. In that case, the depth missing challenges from input frames caused by hand self-occlusions and imaging quality lead to multi-valued mapping phenomenon and sub-optimal model. In this paper, we proposed a novel recurrent architecture named Attention-based Pose Sequence Machine (APSM) to alleviate challenges by introducing temporal consistency. As for recurrent unit (RU), we extend traditional Gated Recurrent Unit (GRU) with 3D convolutional neural networks (CNNs) to handle voxelized inputs and features, and a novel RU named Deep Gated Recurrent Unit (DGRU) was proposed by rebuilding deeper gates based on GRU. To improve the model performance, a novel spatial attention mechanism denoted as Attention Model (AM) was proposed. Ablation experiments are designed to validate each contribution of our work, and experiments on two publicly available dataset show that our work outperforms state-of-the-art on hand pose estimation.

Deep GRU Neural-Network Prediction and Feedforward Compensation for Precision Multi-Axis Motion Control Systems

This article proposes a gated recurrent unit (GRU) neural network prediction and compensation (NNC) strategy for precision multiaxis motion control systems with contouring performance orientation. First, some characteristic contouring tasks are carried out on a multiaxis linear-motor-driven motion system, and the true contouring error values obtained by the Newton numerical calculation method are used as the training data of a developed artificial GRU neural network. Essentially, the proposed GRU neural network structure can be viewed as a data-based black-box error model, which can capture the dynamic characteristics of contouring motion rather accurately. The well-trained GRU network can predict the contouring error precisely even under the tasks those have not been conducted during the training session. Moreover, the predicted contouring error is compensated into the reference contour as feedforward compensation to improve the final contouring performance. Comparison between the predicted contouring error and the actual contouring error practically proves the effective prediction ability of the proposed GRU neural network. Furthermore, comparative experiments among proportional–integral–differential, iterative learning control (ILC), and the proposed NNC controller are conducted. The results consistently validate that NNC can basically achieve excellent contouring motion performance as ILC, significantly without need of motion repetition and iteration. Due to the implementation convenience and excellent prediction/compensation ability, the proposed NNC would have good potential in industrial mechatronic applications.