Bidirectional Gated Recurrent Unit Research Articles

A hybrid ensemble optimized BiGRU method for short-term photovoltaic generation forecasting

In the context of prominent energy crisis, photovoltaic power (PV) generation has received increasing attention, then accurate PV generation forecasting is crucial for ensuring the smooth operation of power stations. However, existing research is insufficient in comprehensively analyzing the impact of PV generation volatility. To fill the gaps and enhance the prediction accuracy, this paper proposes a new hybrid forecasting method. We first introduce the Locally Weighted Scatterplot Smoothing (LOWESS) method to process the data and enhance the data stability, and use Pearson correlation coefficient (PCC) and Random Forests (RF) for feature selection to improve the quality of input data. Then we use Attention mechanism and Convolutional Neural Network (CNN) layer to optimize Bi-directional Gate Recurrent Unit (BiGRU) model and form a new hybrid model. Finally, based on the Bagging algorithm, we use ensemble learning to further optimize the hybrid BiGRU model to enhance the depth and performance. The proposed method is validated through case analysis results from two different locations, Xuhui District in Shanghai, China and the DKASC area in Alice Springs, Australia. The results demonstrate that, compared with other models, the developed method exhibits exceptional prediction performance and effectively enhances the accuracy of PV generation forecasting. Keywords: photovoltaic power generation; Locally Weighted Scatterplot Smoothing; feature selection; ensemble learning; Bi-directional Gate Recurrent Unit.

Hyperspectral crop image classification via ensemble of classification model with optimal training

Agriculture is a significant source of income, and categorizing the crop has turned into vital factor that aids more in the crop production sector. Traditionally, crop development stage determination is done manually by eye inspection. However, producing high-quality crop type maps using modern approaches remains difficult. In this paper, the hyperspectral crop image classification model is proposed that includes four stages, they are (a) preprocessing, (b) segmentation, (c) feature extraction and (d) classification. In the preprocessing step, the hyperspectral image is provided as input, where the filtering process will carried out using median filtering. The filtered image is then used as the segmentation’s input. The image is segmented in the segmentation step using the enhanced entropy-based fuzzy c-means technique. Subsequently, spectral spatial features and vegetation index-based features are derived from segmented images. The final step is the classification, where the ensemble of classification model will be used that includes models like Convolutional Neural Networks (CNN), Deep Maxout (DMO), Recurrent Neural Networks (RNN), and Bidirectional Gated Recurrent Unit (Bi-GRU), respectively. The proposed Self Improved Tasmanian devil Optimization (SI-TDO) approach has optimally adjusted the Bi-GRU model’s training weights to enhance ensemble classification performance. Finally, the effectiveness of the proposed SI-TDO method compared to the traditional algorithm is examined for several metrics. The SI-TDO obtained the greatest accuracy of 94.68% in training rate 80, while other existing models have the lowest ratings.

Improved deep learning intrusion detection model based on GAN data enhancement

With the development of information technology, the number and methods of cyber attacks continue to increase, making network security issues increasingly important. Intrusion detection has become a vital means of dealing with cyber threats. Current intrusion detection methods predominantly rely on machine learning. However, machine learning suffers from limitations in detection capability and the requirement for extensive feature engineering. Additionally, current intrusion detection datasets face the challenge of data imbalance. To address these challenges, this paper proposes a novel solution leveraging Generative Adversarial Networks (GANs) to balance the dataset and introduces an attention mechanism into the generator to efficiently extract key feature information, the mechanism can effectively sort the key information of the data and quickly capture important features. Subsequently, a combination of 1D Convolutional Neural Networks (1DCNN) and Bidirectional Gated Recurrent Units (BiGRU) is employed to construct a classification model capable of extracting both spatial and temporal features. Furthermore, Particle Swarm Optimization (PSO) is utilized to optimize the input weights and hidden biases of the model, so as to further improve the accuracy and robustness of the model. Finally, the model is trained and implemented for network intrusion detection. To demonstrate the applicability of the model, experiments were conducted using the NSL-KDD dataset and the UNSW-NB15 dataset. The final results showed that the proposed model outperformed other models, achieving accuracies of 99.15% and 97.33% on the respective datasets. This indicates that the model improves the efficiency of network intrusion detection and better ensures the effectiveness of network security.

LNBi-GRU model for coal price prediction and pattern recognition analysis

Accurately predicting coal prices and identifying related factors are of great significance for energy market. However, limited attention has been devoted to the precision of predicting coal price trends and the comprehensive analysis of influencing factors. In this paper, we propose LNBi-GRU (Layer Normalization and Bidirectional GRU), which integrates Layer Normalization (LN) and Bidirectional network (Bi) to form the LNBi layer, thereby advancing coal price forecasting. Meanwhile, this study also achieves coal price pattern recognition through a combination of prediction, evaluation, and explanation. The results show that LNBi-GRU outperforms the selected baseline models in terms of both prediction accuracy and stability, and can predict mutation points more accurately. Ablation experiments prove the effectiveness of the added modules including LN and Bi. Moreover, market price emerges as a critical factor affecting coal prices. From a cyclical perspective, the dominant factor in the up cycle shifts from cost push to demand pull, while the dominant factor in the down cycle shifts from demand pull to cost push.

Cross-Project Defect Prediction Based on Domain Adaptation and LSTM Optimization

Cross-project defect prediction (CPDP) aims to predict software defects in a target project domain by leveraging information from different source project domains, allowing testers to identify defective modules quickly. However, CPDP models often underperform due to different data distributions between source and target domains, class imbalances, and the presence of noisy and irrelevant instances in both source and target projects. Additionally, standard features often fail to capture sufficient semantic and contextual information from the source project, leading to poor prediction performance in the target project. To address these challenges, this research proposes Smote Correlation and Attention Gated recurrent unit based Long Short-Term Memory optimization (SCAG-LSTM), which first employs a novel hybrid technique that extends the synthetic minority over-sampling technique (SMOTE) with edited nearest neighbors (ENN) to rebalance class distributions and mitigate the issues caused by noisy and irrelevant instances in both source and target domains. Furthermore, correlation-based feature selection (CFS) with best-first search (BFS) is utilized to identify and select the most important features, aiming to reduce the differences in data distribution among projects. Additionally, SCAG-LSTM integrates bidirectional gated recurrent unit (Bi-GRU) and bidirectional long short-term memory (Bi-LSTM) networks to enhance the effectiveness of the long short-term memory (LSTM) model. These components efficiently capture semantic and contextual information as well as dependencies within the data, leading to more accurate predictions. Moreover, an attention mechanism is incorporated into the model to focus on key features, further improving prediction performance. Experiments are conducted on apache_lucene, equinox, eclipse_jdt_core, eclipse_pde_ui, and mylyn (AEEEM) and predictor models in software engineering (PROMISE) datasets and compared with active learning-based method (ALTRA), multi-source-based cross-project defect prediction method (MSCPDP), the two-phase feature importance amplification method (TFIA) on AEEEM and the two-phase transfer learning method (TPTL), domain adaptive kernel twin support vector machines method (DA-KTSVMO), and generative adversarial long-short term memory neural networks method (GB-CPDP) on PROMISE datasets. The results demonstrate that the proposed SCAG-LSTM model enhances the baseline models by 33.03%, 29.15% and 1.48% in terms of F1-measure and by 16.32%, 34.41% and 3.59% in terms of Area Under the Curve (AUC) on the AEEEM dataset, while on the PROMISE dataset it enhances the baseline models’ F1-measure by 42.60%, 32.00% and 25.10% and AUC by 34.90%, 27.80% and 12.96%. These findings suggest that the proposed model exhibits strong predictive performance.

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition.

Teaching gesture recognition is a technique used to recognize the hand movements of teachers in classroom teaching scenarios. This technology is widely used in education, including for classroom teaching evaluation, enhancing online teaching, and assisting special education. However, current research on gesture recognition in teaching mainly focuses on detecting the static gestures of individual students and analyzing their classroom behavior. To analyze the teacher's gestures and mitigate the difficulty of single-target dynamic gesture recognition in multi-person teaching scenarios, this paper proposes skeleton-based teaching gesture recognition (ST-TGR), which learns through spatio-temporal representation. This method mainly uses the human pose estimation technique RTMPose to extract the coordinates of the keypoints of the teacher's skeleton and then inputs the recognized sequence of the teacher's skeleton into the MoGRU action recognition network for classifying gesture actions. The MoGRU action recognition module mainly learns the spatio-temporal representation of target actions by stacking a multi-scale bidirectional gated recurrent unit (BiGRU) and using improved attention mechanism modules. To validate the generalization of the action recognition network model, we conducted comparative experiments on datasets including NTU RGB+D 60, UT-Kinect Action3D, SBU Kinect Interaction, and Florence 3D. The results indicate that, compared with most existing baseline models, the model proposed in this article exhibits better performance in recognition accuracy and speed.

Fault transfer diagnosis of rolling bearings across different devices via multi-domain information fusion and multi-kernel maximum mean discrepancy

The current deep learning-based intelligent diagnosis algorithms depend on large amounts of well-labeled data, but they may not perform well in engineering practice where the fault data available from a single device is limited. The significant difference in data distribution between different devices makes it more difficult to transfer diagnosis across devices. Therefore, aiming at the problem that the great difference in the data distribution of source and target domains in cross-device transfer diagnosis leads to the low diagnosis accuracy, this paper investigates a deep transfer network model based on multi-domain information fusion and multi-kernel maximum mean discrepancy (MK-MMD) for fault diagnosis of rolling bearings across different devices. Firstly, to address the problem that single-domain information is difficult to adequately characterize rolling bearing health status information of different devices, a multi-domain information feature extractor based on multi-attention mechanism is constructed to capture the important features of vibration signals in different transform domains. Secondly, the extracted multi-domain features are input into the bidirectional gated recurrent unit (BiGRU) network for feature fusion, and MK-MMD is used to narrow the distribution distance between the source domain and target domain to obtain domain invariant features, and then achieve the transfer diagnosis of rolling bearing faults between different devices. Finally, the method investigated is verified by rolling bearing fault transfer diagnosis tests between different devices, and the results show that the suggested method can adapt to the feature distribution between different domains, and improve the transfer diagnosis accuracy between different devices.

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data.

Activity recognition is one of the significant technologies accompanying the development of the Internet of Things (IoT). It can help in recording daily life activities or reporting emergencies, thus improving the user's quality of life and safety, and even easing the workload of caregivers. This study proposes a human activity recognition (HAR) system based on activity data obtained via the micro-Doppler effect, combining a two-stream one-dimensional convolutional neural network (1D-CNN) with a bidirectional gated recurrent unit (BiGRU). Initially, radar sensor data are used to generate information related to time and frequency responses using short-time Fourier transform (STFT). Subsequently, the magnitudes and phase values are calculated and fed into the 1D-CNN and Bi-GRU models to extract spatial and temporal features for subsequent model training and activity recognition. Additionally, we propose a simple cross-channel operation (CCO) to facilitate the exchange of magnitude and phase features between parallel convolutional layers. An open dataset collected through radar, named Rad-HAR, is employed for model training and performance evaluation. Experimental results demonstrate that the proposed 1D-CNN+CCO-BiGRU model demonstrated superior performance, achieving an impressive accuracy rate of 98.2%. This outperformance of existing systems with the radar sensor underscores the proposed model's potential applicability in real-world scenarios, marking a significant advancement in the field of HAR within the IoT framework.

A fault diagnosis method for flexible converter valve equipment based on DSC-BIGRU-MA

Introduction: Precise fault diagnosis is crucial for enhancing the reliability and lifespan of the flexible converter valve equipment. To address this issue, depthwise separable convolution, bidirectional gate recurrent unit, and multi-head attention module (DSC-BiGRU-MAM) based fault diagnosis approach is proposed.Methods: By DSC and BiGRU operation, the model can capture the correlation between local features and temporal information when processing sequence data, thereby enhancing the representation ability and predictive performance of the model for complex sequential data. In addition, by incorporating a multi-head attention module, the proposed method dynamically learns important information from different time intervals and channels. The proposed MAM continuously stimulates fault features in both time and channel dimensions during training, while suppressing fault independent expressions. As a result, it has made an important contribution to improving the performance of the fault diagnosis model.Results and Discussion: Experimental results demonstrate that the proposed method achieves higher accuracy compared to existing methods, with an average accuracy of 95.45%, average precision of 88.67%, and average recall of 89.03%. Additionally, the proposed method has a moderate number of model parameters (17,626) and training time (935 s). Results indicate that the proposed method accurately diagnoses faults in flexible converter valve equipment, especially in real-world situations with noise overlapping signals.

Revealing the degradation patterns of lithium-ion batteries from impedance spectroscopy using variational auto-encoders

The aging life estimation of lithium-ion batteries (LIBs) is of great significance to the use, maintenance and economic analysis of energy storage systems. The estimation method of aging life based on electrochemical impedance spectroscopy (EIS) has received more attention due to its high accuracy. Due to the high dimensionality of impedance data, it cannot directly reflect the decline trend, this paper proposes a method of combining variational auto-encoders (VAE) and bidirectional gated recurrent unit (BiGRU) for automatic feature extraction from EIS data and battery aging life estimation. Reducing the dimensionality of impedance data through VAE can greatly reduce computational complexity, and the high-quality features extracted are mapped to battery capacity using BiGRU, which can improve estimation accuracy. The validation results show that the method can adapt to various working conditions, and has good estimation accuracy and robustness. MAE and RMSE are less than 1.27 mAh and 1.43 mAh, respectively. We demonstrate that the latent variables extracted from EIS data using unsupervised learning method reliably represent the degradation patterns of LIBs.

The marine diesel engine exhaust gas temperature baseline model based on particle swarm optimised generalised regression neural network

ABSTRACT This study addresses limitations in traditional condition monitoring for marine diesel engine (MDE) reliability and stability by proposing a hybrid machine learning and deep learning (DL) model called Particle Swarm Optimization-Generalized Regression Neural Network (PSO-GRNN) for real-time exhaust gas temperature (EGT) monitoring. The model integrates Mahalanobis distance (MD) calibration, wavelet packet denoising, and Pearson correlation analysis for comprehensive data preprocessing. Validation is conducted using historical '6L34DF' diesel engine data. Particle Swarm Optimization (PSO) optimizes the spread value (σ) of the Generalized Regression Neural Network (GRNN) to establish an optimal baseline model for EGT prediction. Experimental results show that PSO-GRNN outperforms Backpropagation Neural Network (BPNN), GRNN, Bidirectional Gated Recurrent Unit (BiGRU), Genetic Algorithm- Generalized Regression Neural Network (GA-GRNN), and Deep Belief Network-Support Vector Regression (DBN-SVR) in terms of training time and accuracy, demonstrating its suitability for MDE baseline modeling.

MFTrans: A multi-feature transformer network for protein secondary structure prediction

In the rapidly evolving field of computational biology, accurate prediction of protein secondary structures is crucial for understanding protein functions, facilitating drug discovery, and advancing disease diagnostics. In this paper, we propose MFTrans, a deep learning-based multi-feature fusion network aimed at enhancing the precision and efficiency of Protein Secondary Structure Prediction (PSSP). This model employs a Multiple Sequence Alignment (MSA) Transformer in combination with a multi-view deep learning architecture to effectively capture both global and local features of protein sequences. MFTrans integrates diverse features generated by protein sequences, including MSA, sequence information, evolutionary information, and hidden state information, using a multi-feature fusion strategy. The MSA Transformer is utilized to interleave row and column attention across the input MSA, while a Transformer encoder and decoder are introduced to enhance the extracted high-level features. A hybrid network architecture, combining a convolutional neural network with a bidirectional Gated Recurrent Unit (BiGRU) network, is used to further extract high-level features after feature fusion. In independent tests, our experimental results show that MFTrans has superior generalization ability, outperforming other state-of-the-art PSSP models by 3 % on average on public benchmarks including CASP12, CASP13, CASP14, TEST2016, TEST2018, and CB513. Case studies further highlight its advanced performance in predicting mutation sites. MFTrans contributes significantly to the protein science field, opening new avenues for drug discovery, disease diagnosis, and protein.

Prediction of small-scale leak flow rate in LOCA situations using bidirectional GRU

It is difficult to detect a small-scale leakage in a nuclear power plant (NPP) quickly and take appropriate action. Delaying these procedures can have adverse effects on NPPs. In this paper, we propose leak flow rate prediction using the bidirectional gated recurrent unit (Bi-GRU) method to detect leakage quickly and accurately in small-scale leakage situations because large-scale leak rates are known to be predicted accurately. The data were acquired by simulating small loss-of-coolant accidents (LOCA) or small-scale leakage situations using the modular accident analysis program (MAAP) code. In addition, to improve prediction performance, data were collected by distinguishing the break sizes in more detail. In addition, the prediction accuracy was improved by performing both LOCA diagnosis and leak flow rate prediction in small LOCA situations. The prediction model developed using the Bi-GRU showed a superior prediction performance compared with other artificial intelligence methods. Accordingly, the accurate and effective prediction model for small-scale leakage situations proposed herein is expected to support operators in decision-making and taking actions.

Motion feature estimation using bi-directional GRU for skeleton-based dynamic hand gesture recognition

Motion feature estimation using bi-directional GRU for skeleton-based dynamic hand gesture recognition

Correlation-based feature selection and parallel spatiotemporal networks for efficient passenger flow forecasting in metro systems

This paper presents a novel framework for predicting metro passenger flow that is both interpretable and computationally efficient. The proposed method first uses a correlation-based spatiotemporal feature selection strategy (Cor-STFS) to identify the optimal input scheme for the prediction model, effectively reducing unnecessary interference. The framework then introduces a new multivariate passenger flow prediction architecture called STA-PTCN-BiGRU, which combines a spatiotemporal attention (STA) mechanism, parallel temporal convolutional networks (PTCN), and bidirectional gated recurrent units (BiGRU) to capture the dynamic internal patterns of passenger flow. By utilising parallel computing, this architecture significantly reduces resource consumption. The effectiveness of the proposed approach is evaluated using four datasets from the Shanghai Metro. Experimental results show that the new method outperforms baseline approaches in terms of root mean square error (RMSE), mean absolute error (MAE), and symmetric mean absolute percentage error (SMAPE), achieving average reductions of 9.98%, 8.08%, and 13.29% in these metrics, respectively.

Empowering hate speech detection: leveraging linguistic richness and deep learning

Social media has become a vital part of most modern human personal life. Twitter is one of the social media that was formed from the development of communication technology. A lot of social media gives users the freedom to express themselves. This facility is misused by users, so hate speech is spread. Designing a system to detect hate speech intelligently is needed. This study uses the hybrid deep learning (HDL) and solo deep learning (SDL) approach with the convolutional neural networks (CNN) and bidirectional gated recurrent unit (Bi-GRU) algorithm. There are 4 models built, namely CNN, Bi-GRU, CNN+Bi-GRU, and Bi-GRU+CNN. Term frequency-inverse document frequency (TF-IDF) is used for feature extraction, which is to get linguistic features to be analyzed and studied. FastText is used to perform feature expansion to minimize mismatched vocabulary. Four scenarios are run. CNN with an accuracy of 87.63%, Bi-GRU produces an accuracy of 87.46%, CNN+Bi-GRU provides an accuracy of 87.47% and Bi-GRU+CNN provides an accuracy of 87.34%. The ability of this approach to understand the context is qualified. HDL outperforms SDL in terms of n-gram type, where HDL can understand sentences broken down by hybrid n-gram types, namely Unigram-Bigram-Trigram which is a complex n-gram hybrid.

Hidden Markov guided Deep Learning models for forecasting highly volatile agricultural commodity prices

Predicting agricultural commodity prices accurately is of utmost importance due to various factors such as perishability, seasonality, production uncertainty etc. Moreover, the substantial volatility that may be exhibited in time series further adds to the complexity and constitutes a significant challenge. In this paper, a Hidden Markov (HM) guided Deep Learning (DL) models has been developed on nonlinear and nonstationary price data of agricultural commodities for forecasting by considering technical indicators viz., Moving Average (MA), Bollinger Bands (BB), Moving Average Convergence Divergence (MACD), Exponential MA (EMA) and Fast Fourier Transformation (FFT). HM Models (HMMs) can effectively handle the sequential dependencies and hidden states, while DL approach can learn complex patterns and relationships within the price series and thus the drawback of lack of generalization capability in the DL model has been overcome by HMM. In this study, the Potato price data of the Champadanga district of West Bengal, India has been utilized to assess the performance of the proposed technique. HMM has been combined with six baseline DL models viz., Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), Bidirectional LSTM (BiLSTM) and Bidirectional GRU (BiGRU) for forecast modeling. Performance evaluation metrics viz., Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE) and the insightful Diebold–Mariano (DM) test revealed that Hidden Markov hybridized with DL models surpassed baseline DL models in forecasting accuracy for 1-week, 4-week, 8-week and 12-week ahead DL predictions. The proposed approach holds significant promise for enhancing the precision of agricultural commodity price forecasting with far-reaching implications for various stakeholders such as farmers and planners.

Enhancing Arabic offensive language detection with BERT-BiGRU model

With the advent of Web 2.0, various platforms and tools have been developed to allow internet users to express their opinions and thoughts on diverse topics and occurrences. Nevertheless, certain users misuse these platforms by sharing hateful and offensive speeches, which has a negative impact on the mental health of internet society. Thus, the detection of offensive language has become an active area of research in the field of natural language processing. Rapidly detecting offensive language on the internet and preventing it from spreading is of great practical significance in reducing cyberbullying and self-harm behaviors. Despite the crucial importance of this task, limited work has been done in this field for nonEnglish languages such as Arabic. Therefore, in this paper, we aim to improve the results of Arabic offensive language detection without the need for laborious preprocessing or feature engineering work. To achieve this, we combine the bidirectional encoder representations from transformers (BERT) model model with a bidirectional gated recurrent unit (BiGRU) layer to further enhance the extracted context and semantic features. The experiments were conducted on the Arabic dataset provided by the SemEval 2020 Task 12. The evaluation results show the effectiveness of our model compared to the baseline and related work models by achieving a macro F1- score of 93.16%.

Network Attack Prediction With Hybrid Temporal Convolutional Network and Bidirectional GRU

Network Attack Prediction With Hybrid Temporal Convolutional Network and Bidirectional GRU

Prediction of impurity spectrum function by deep learning algorithm

By using the numerical renormalization group (NRG) method, we construct a large dataset with about one million spectral functions of the Anderson quantum impurity model. The dataset contains the density of states (DOS) of the host material, the strength of Coulomb interaction between on-site electrons (U), and the hybridization between the host material and the impurity site (Γ). The continued DOS and spectral functions are stored with Chebyshev coefficients and wavelet functions, respectively. From this dataset, we build seven different machine learning networks to predict the spectral function from the input data, DOS, U, and Γ. Three different evaluation indexes, mean absolute error (MAE), relative error (RE) and root mean square error (RMSE), are used to analyze the prediction abilities of different network models. Detailed analysis shows that, for the two kinds of widely used recurrent neural networks (RNNs), gate recurrent unit (GRU) has better performance than the long short term memory (LSTM) network. A combination of bidirectional GRU (BiGRU) and GRU has the best performance among GRU, BiGRU, LSTM, and BiLSTM. The MAE peak of BiGRU+GRU reaches 0.00037. We have also tested a one-dimensional convolutional neural network (1DCNN) with 20 hidden layers and a residual neural network (ResNet), we find that the 1DCNN has almost the same performance of the BiGRU+GRU network for the original dataset, while the robustness testing seems to be a little weak than BiGRU+GRU when we test all these models on two other independent datasets. The ResNet has the worst performance among all the seven network models. The datasets presented in this paper, including the large data set of the spectral function of Anderson quantum impurity model, are openly available at https://doi.org/10.57760/sciencedb.j00113.00192.