Bidirectional Gated Recurrent Unit Research Articles

Chinese Mathematical Knowledge Entity Recognition Based on Linguistically Motivated Bidirectional Encoder Representation from Transformers

We assessed whether constructing a mathematical knowledge graph for a knowledge question-answering system or a course recommendation system, Named Entity Recognition (NER), is indispensable. The accuracy of its recognition directly affects the actual performance of these subsequent tasks. In order to improve the accuracy of mathematical knowledge entity recognition and provide effective support for subsequent functionalities, this paper adopts the latest pre-trained language model, LERT, combined with a Bidirectional Gated Recurrent Unit (BiGRU), Iterated Dilated Convolutional Neural Networks (IDCNNs), and Conditional Random Fields (CRFs), to construct the LERT-BiGRU-IDCNN-CRF model. First, LERT provides context-related word vectors, and then the BiGRU captures both long-distance and short-distance information, the IDCNN retrieves local information, and finally the CRF is decoded to output the corresponding labels. Experimental results show that the accuracy of this model when recognizing mathematical concepts and theorem entities is 97.22%, the recall score is 97.47%, and the F1 score is 97.34%. This model can accurately recognize the required entities, and, through comparison, this method outperforms the current state-of-the-art entity recognition models.

Automated rice mapping using multitemporal Sentinel-1 SAR imagery using dynamic threshold and slope-based index methods

Rice cultivation plays a crucial role in food security and economic development, particularly in regions like India, due to its vast population and position as the top rice producer globally. This work introduces a novel framework, the Rice Mapping Method (RMM), which leverages Multitemporal Sentinel-1 Synthetic Aperture Radar (SAR) imagery for automated rice mapping. Contrary to the traditional approaches, RMM combines the Dynamic Threshold Method (DTM) for robust rice field identification and a slope-based index for classifying single and double cropping practices. By analyzing VH backscatter patterns and employing specific thresholds, DTM separates rice pixels from the other background pixels. The DTM, which relies on VH backscatter values during the growing season, has been tested across various rice cultivation landscapes, demonstrating high accuracy upto 0.95. DTM is also tested on different rice-growing areas such as the hilly Kodagu district, with an F1 Score of 0.96, and in the flooded delta region of Kuttanad, achieving an F1 Score of 0.93. The Slope-based Index I(r,c) is introduced to differentiate the single and double cropping pixels by calculating the index for the second season of cropping and gives F1 Score of 0.81. The DTM’s effectiveness in rice field identification is evaluated by comparing it to the classification of the Bi-directional Gated Recurrent Unit (Bi-GRU) network. Similarly, the Slope-based Index is compared with other established automated rice mapping methods to assess its accuracy in distinguishing cropping patterns. RMM was successfully applied in mapping rice-growing areas in the Udupi district for 2021, estimating Kharif and Rabi season areas, the estimated rice area is compared to official statistics by the Directorate of Economics and Statistics, Karnataka State. The proposed RMM approach offers a robust solution for mapping rice fields, particularly in regions with complex cropping landscapes, and enhances agricultural monitoring and decision-making processes contributing to sustainable rice production and food security initiatives.

BiGRU-CNN-AT: classifiying emotion on social media

PurposeThis research aims to develop a robust deep-learning approach for classifying emotion in social media.Design/methodology/approachThis study integrates three deep learning techniques: Bidirectional Gated Recurrent Units (BiGRU), convolutional neural networks (CNN) and an attention mechanism, resulting in the Bidirectional Gated Recurrent Units Convolution Attention (BiGRU-CNN-AT) model. The BiGRU captures potential semantic features, the CNN extracts local features and the attention mechanism identifies keywords critical for classification.FindingsThe BiGRU-CNN-AT model outperformed several state-of-the-art emotion classification algorithms. The model was compared against various baselines across multiple emotion datasets, with deep learning methods consistently surpassing traditional approaches. BiGRU and Bi-LSTM networks demonstrated superior performance, particularly when combined with attention mechanisms. Additionally, analysis of execution times indicated that the BiGRU model processed data more efficiently. They were configuring hyperparameters and integrating GloVe word embeddings, which significantly enhanced model performance, with the adam optimizer proving effective for optimization.Originality/valueThis paper contributes to the development of a novel framework, BiGRU-CNN-AT, which integrates bidirectional GRU, CNN and attention mechanisms for text-based emotion classification. By leveraging the strengths of each component, this framework significantly enhances accuracy in emotion classification tasks. Furthermore, the study offers comprehensive experimental analyses across multiple emotion datasets.

Nature-Inspired Optimization with Deep Learning-Enabled Sustainable Predictive Model for Intelligent Systems

Solar energy can be considered as an alternate solution to conventional energy sources. Short-term Photovoltaic (PV) Power Generation (PVPG) prediction methods are essential stabilize power integration among PV and smart grids. The PVPG generation process is highly dependent on climatic conditions and therefore high intermittent. Highly accurate PVPG prediction of PVPG acts based on the generation, transmission and dispersion of electricity, confirming the stability and dependability of power system. The current progress of Machine Learning (ML) and Deep Learning (DL) approaches enables for designing of accurate PVPG prediction models. In this view, this paper develops a new Badger Optimization with Deep Learning Enabled PV Power Generation Predictive (HBODL-PVPGP) model. The presented HBODL-PVPGP model enables to forecast of the PVPG process. To accomplish this, the HBODL-PVPGP model initially investigates the features depending upon intrinsic characteristics earlier in the learning process. In addition, the Bidirectional Gated Recurrent Unit (BiGRU) model is implemented for forecasting process. The performance of the BiGRU model can be improvised by the design of HBO-based hyperparameter tuning procedure. For ensuring the enhanced performance of the HBODL-PVPGP model, an extensive range of experimental study was effectuated and the results were investigated under various factors. The result highlighted the precipitated performance of the HBODL-PVPGP procedure on the current algorithms.

BiGRU-Att: Blood Glucose Prediction with Bidirectional Recurrent Neural Networks and Attention Mechanisms

This paper presents an advanced deep learning model that combines a bidirectional gated recurrent unit (BiGRU) and an attention mechanism for predicting blood glucose levels. The innovation of the model lies in its unique structural design, in which the BiGRU is able to capture the backward and forward dependencies in time series data, while the attention mechanism further enhances the sensitivity of the model to critical time steps, thus improving the accuracy of the prediction. This combination not only exploits the power of BiGRU in processing serial data, but also enables adaptive weighting of important features in the input data through the attention mechanism. We validate the effectiveness of the model through experiments on 10 in silicodatasets generated by the UVA/Padova T1D simulator. On these datasets, the model demonstrated excellent performance with a root mean square error (RMSE) of only 0.0719, a metric that is significantly lower than that of existing techniques, proving the superiority and innovativeness of our model for blood glucose prediction tasks.

Remaining useful life prediction of pressure regulating shutoff valve based on feature fusion and bidirectional gated recurrent unit

Abstract In the bleed air system of the ARJ21 aircraft, pressure regulating shutoff valve (PRSOV) failures are common, and their failures can lead to disasters and economic losses. Accordingly, prediction of the degradation performance of PRSOV is crucial. This work proposes a life prediction method based on principal component analysis (PCA) and bidirectional gated recurrent units (BiGRU) to achieve accurate prediction. After obtaining pressure data throughout the entire life of PRSOV, considering that the pressure required for PRSOV during the takeoff and climb phases is the most critical, data from this phase are selected for focused monitoring. Classical statistical feature extraction methods are used to extract features from the raw pressure data during the takeoff and climb phases. An empirical feature extraction method with low-pressure weighting is also proposed based on engineering practical experience. Feature fusion is performed using PCA based on these two types of features. Finally, BiGRU is utilized to model the fused degradation feature indicators and estimate the remaining service life of PRSOV. The results of the analysis of the full life data of PRSOV in ARJ21 aircraft indicated that the proposed method can effectively predict its remaining service life. The proposed method demonstrated higher prediction accuracy compared with the related prediction methods.

Bidirectional Deep Learning Model based on Attention Mechanism in English Cloze Tests

English cloze tests, as a common form of language assessment, aim to evaluate learners' comprehensive understanding of context, vocabulary, and grammar. However, the complex contextual relationships and long-distance dependencies within the questions pose significant challenges for machine learning models. Traditional rule-based or statistical methods struggle to effectively capture the intricate contextual information present in sentences. This study aims to develop a deep learning-based English cloze test answering system to enhance students' ability to tackle such questions. To address the limitations of traditional methods in handling complex contexts and long-distance dependencies, a model that combines bidirectional gated recurrent units (BiGRU) and attention mechanisms is proposed. This model is better equipped to capture the surrounding context of sentences and dynamically adjust attention to accurately predict the missing words. Additionally, integrating the embedding layer with BiGRU and attention mechanisms further improves model performance. Testing results based on the Children’s Book Test dataset are highly promising. Our model excels in key metrics such as accuracy, recall, F1 score, and Cohen's Kappa, achieving scores of 77.5%, 77.5%, 0.758, and 0.7649, respectively. Compared to traditional models, our approach demonstrates clear advantages in handling complex contexts and long sentences. This research provides new technical support for developing more intelligent English learning systems.

Ensemble learning based on bi-directional gated recurrent unit and convolutional neural network with word embedding module for bioactive peptide prediction.

Bioactive peptides, as small protein fragments, are essential mediators of diverse physiological activities, such as antimicrobial, anti-inflammatory, anticancer, antioxidant, and immunomodulatory functions. Despite their substantial potential in pharmaceuticals and the food industry, conventional methods for peptide classification and activity prediction are limited by high costs, time-intensive procedures, and extensive data processing requirements. Here, we present BioPepPred-DLEmb, a novel computational model integrating Convolutional Neural Networks (CNNs) and Bidirectional Gated Recurrent Units (BiGRUs), augmented with natural language processing to encode amino acids into information-dense vectors. Evaluated across nine bioactive peptide datasets, BioPepPred-DLEmb demonstrates superior predictive accuracy (0.909) and sensitivity (0.911) compared to traditional methods. Through UMAP visualization and Kplogo analysis, the model effectively differentiates peptide activity states and identifies key biomarkers. The predicted antimicrobial peptides (Pred-AMPs) exhibit potent efficacy in vitro, achieving low micromolar inhibitory concentrations (2-16μmol/L) against pathogens such as Escherichia coli and Acinetobacter baumannii. These findings establish a robust foundation for bioactive peptide development, with implications for advancements in precision medicine, personalized therapies, and functional food innovations.

Wind and Photovoltaic Power Generation Forecasting for Virtual Power Plants Based on the Fusion of Improved K-Means Cluster Analysis and Deep Learning

Virtual power plants (VPPs) have emerged as an innovative solution for modern power systems, particularly for integrating renewable energy sources. This study proposes a novel prediction approach combining improved K-means clustering with Time Convolutional Networks (TCNs), a Bi-directional Gated Recurrent Unit (BiGRU), and an attention mechanism to enhance the forecasting accuracy of wind and photovoltaic power generation in VPPs. The proposed TCN-BiGRU-Attention model demonstrates superior predictive performance compared to traditional models, achieving high accuracy and robustness. These results provide a reliable basis for optimizing VPP operations and integrating renewable energy sources effectively.

Streamflow simulation at different temporal scales under rating curve uncertainty conditions using machine learning models

ABSTRACT Reducing uncertainty in streamflow simulation is vital for effective water resource management. The impact of uncertainty in model calibration data (discharge), commonly derived from the rating curve, is often overlooked. This study applies the Monte Carlo simulation technique (MCST) to assess uncertainty in the rating curve. Advanced machine learning (ML) models, bidirectional long short-term memory (BiLSTM), and bidirectional gated recurrent units (BiGRUs) were used comparatively to evaluate the propagation of this uncertainty onto streamflow simulation on both daily and monthly temporal scales. Different sets of streamflow data, derived from the fitted curve and its lower and upper uncertainty bands, were utilized to train ML models independently. The results show the substantial impact of rating curve uncertainty in streamflow simulations, with the BiGRU model surpassing the BiLSTM model on both scales. As a result, the uncertainty in the rating curve results in an uncertainty of the streamflow of up to 30 and 25% on daily and monthly simulations, respectively. These findings underscore the importance of considering rating curve uncertainty in streamflow simulation to ensure accurate and reliable results. Therefore, streamflow should be treated as an uncertain variable and managed by incorporating rating curve uncertainty in decision-making.

Sentiment Analysis for E-commerce Product Reviews by Deep Learning Model of Bert- BiGRU-Softmax

The sentiment analysis has the key problem for the e-commerce product quality management, which customers know their people’s attitudes on interested products by e-commerce reviews. Meanwhile, manufacturers are able to learn the public sentiment on their products being sold in E-commerce platforms. This paper proposed the deep learning models of Bert-BiGRU-Softmax, which uses the input layer of Sentiment Bert model to extract multi-dimensional of E-commerce reviews, the hidden layer of Bidirectional GRU model to obtain semantic codes and calculate representing weights of reviews, the Softmax with attention mechanism as the output layer to classify the sentiment tendency. We conduct experiments on a large-scale dataset involving over 500 thousand reviews compared with different learning models. The experimental results show that the proposed models reach the high accuracy 95.5% on the E-commerce reviews, and the outperforms of RNN, BiGRU, Bert-BiLSTM in terms of accuracy and loss.

Enhancing Artillery Barrel Wear Detection via CRNN-Based Acoustic Analysis and Domain Adaptation

Abstract The wear of artillery barrels directly impacts the accuracy and firepower of weapon systems, and precise detection of barrel wear is paramount for enhancing military capabilities. This study proposes a sound-based method for detecting artillery barrel wear, aimed at addressing the challenges of invasiveness and inefficiency in traditional methods, as well as the high cost and resource demands of modern approaches. We introduce AcouDAR, an end-to-end adaptive framework designed to elevate information extraction capabilities through the integration of time convolutional networks (TCN) and bidirectional gated recurrent units (Bi-GRU). Additionally, a domain adaptation module comprising domain alignment and domain relevance elimination modules is incorporated to mitigate the model’s reliance on specific historical sound data of artillery barrels and to bolster its adaptability in real-world scenarios. Extensive experimentation validates the superior performance of the AcouDAR model in detecting artillery barrel wear.

Vessel trajectory prediction in harbors: A deep learning approach with maritime-based data preprocessing and Berthing Side Integration

This study proposes a novel deep learning approach for vessel trajectory prediction in harbor environments, emphasizing the integration of berthing side information. Using Automatic Identification System (AIS) data from Busan Port, we compare Bidirectional Long Short-Term Memory (Bi-LSTM), Bidirectional Gated Recurrent Unit (Bi-GRU), and Transformer models. We identify the berthing side (port or starboard) as a crucial factor that influences vessel trajectories during berth approach, revealing significant differences in final approach paths. Port-side berthing vessels typically take the shortest course from the fairway endpoint, while starboard-side vessels approach parallel to the berth before final berthing. We employed advanced data preprocessing techniques, including DBSCAN clustering, cubic spline interpolation, and Gaussian noise-based data augmentation. Models were trained and evaluated on datasets with and without berthing side information. The Bi-LSTM model outperformed others in most scenarios, improving prediction accuracy by up to 23% when using berthing side-specific datasets. Paired t-tests confirmed statistically significant performance improvements for the Bi-LSTM model. This research enhances maritime safety and efficiency by providing more accurate trajectory predictions, enabling Vessel Traffic Services Operators (VTSOs) to respond to potential hazards swiftly and assisting pilots in developing safer berthing plans.

Next Basket Recommendation Paradigm Multi-Layer Stacked Sequence to Sequence Bidirectional GRU Model with Multiplicative Attention

The Recommendation System is one of the latest application tools developed through Deep Learning research activities under the Machine Learning technical domain. The application might have recognized an idea for establishing a user-item relationship on consumables with guidance from the user’s historical purchasing transaction data to achieve future demand predictions. Researchers had been able to bring out memory-based concepts at the beginning supplemented with model-based tools for further refinements with time. Incidentally, the significance of RS was well accepted by business entities, hence personalized services had been initiated to reduce user agony to search out the desired item. In recent years marked improvement was observed with the development of hybrid RS ornamented with the addition of Transformers in the model. The success of the introduction of Next Basket Recommendation relied on the capacity to analyze the user sequential change pattern in purchasing behavior.

A hybrid deep learning framework for short-term load forecasting with improved data cleansing and preprocessing techniques

The dynamic variations in connected loads and the intermittent nature of renewable energy sources significantly impact smart grid reliability. Accurate load and generation forecasting stand pivotal for enhancing grid reliability and efficient operations. In this study, a comprehensive four-step approach is introduced for short-term load forecasting (STLF) aimed at precisely estimating power demand and generation. The process unfolds with data collection, followed by rigorous standardization, preprocessing, and cleansing of demand and generation data. Subsequently, a hybrid deep learning model, comprising bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent unit (BiGRU), and a fully connected layer is trained using the clean data. This model harnesses the temporal dependencies within the data for accurate predictions. The model's performance is then evaluated using mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), providing forecasted values for both generation and demand on minute, hourly, daily, and weekly intervals. Notably, the proposed approach achieves a remarkable MSE of 0.0058 for load forecasting and 0.0033 for generation forecasting. Comparative analysis with state-of-the-art (SOTA) techniques in terms of accuracy and computational cost underscores the superior accuracy of the proposed framework in forecasting both generation and demand. Importantly, the proposed approach bridges the gap in reliability enhancement for smart grids operating, a facet lacking in many existing methodologies. This signifies the potential of the proposed approach to bolster smart grid reliability, ensuring more reliable operations.

Forecasting Shifts in Europe's Renewable and Fossil Fuel Markets Using Deep Learning Methods

ABSTRACTAccurate forecasts of renewable and nonrenewable energy output are essential for meeting global energy needs and resolving environmental issues. Energy sources like the sun and wind are variable, making forecasting difficult. Changes in weather, demand, and energy policy exacerbate this unpredictability. These challenges will be addressed by the bidirectional gated recurrent unit (Bi‐GRU) model, which forecasts power‐generating outcomes more efficiently. The investigation is done over a health data set from 2000 to 2023, including the energy states of the United Kingdom, Finland, Germany, and Switzerland. The comparison of our model (Bi‐GRU) performance with other popular models, including bidirectional long short‐term memory (Bi‐LSTM), ensemble techniques combining convolutional neural networks (CNN) and Bi‐LSTM, and CNNs, make the study more interesting. The performance remains better with a mean absolute percentage error (MAPE) of 2.75%, root mean square error (RMSE) of 0.0414, mean squared error (MSE) of 0.0017, and authentify that Bi‐GRU performs much better than others. This model's superior prediction accuracy significantly enhances our ability to forecast renewable and nonrenewable energy outputs in European states, contributing to more effective energy management strategies.

PM2.5 Time Series Imputation with Moving Averages, Smoothing, and Linear Interpolation

In this work, a novel model for hourly PM2.5 time series imputation is proposed for the estimation of missing values in different gap sizes, including 1, 3, 6, 12, and 24 h. The proposed model is based on statistical techniques such as moving averages, linear interpolation smoothing, and linear interpolation. For the experimentation stage, two datasets were selected in Ilo City in southern Peru. Also, five benchmark models were implemented to compare the proposed model results; the benchmark models include exponential weighted moving average (EWMA), autoregressive integrated moving average (ARIMA), long short-term memory (LSTM), gated recurrent unit (GRU), and bidirectional GRU (BiGRU). The results show that, in terms of average MAPEs, the proposed model outperforms the best deep learning model (GRU) between 26.61% and 90.69%, and the best statistical model (ARIMA) between 2.33% and 6.67%. So, the proposed model is a good alternative for the estimation of missing values in PM2.5 time series.

A real-time predicting online tool for detection of people’s emotions from Arabic tweets based on big data platforms

Emotion prediction is a subset of sentiment analysis that aims to extract emotions from text, speech, or images. The researchers posit that emotions determine human behavior, making the development of a method to recognize emotions automatically crucial for use during global crises, such as the COVID-19 pandemic. In this paper, a real-time system is developed that identifies and predicts emotions conveyed by users in Arabic tweets regarding COVID-19 into standard six emotions based on the big data platform, Apache Spark. The system consists of two main stages: (1) Developing an offline model and (2) Online emotion prediction pipeline. For the first stage, two different approaches: The deep Learning (DL) approach and the Transfer Learning-based (TL) approach to find the optimal classifier for identifying and predicting emotion. For DL, three classifiers are applied: Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and Bidirectional GRU (BiGRU). For TL, five models are applied: AraBERT, ArabicBERT, ARBERT, MARBERT, and QARiB. For the second stage, create a Transmission Control Protocol (TCP) socket between Twitter’s API and Spark used to receive streaming tweets and Apache Spark to predict the label of tweets in real-time. The experimental results show that the QARiB model achieved the highest Jaccard accuracy (65.73%), multi-accuracy (78.71%), precision-micro (78.71%), recall-micro (78.71%), f-micro (78.71%), and f-macro (78.55%). The system is available as a web-based application that aims to provide a real-time visualization of people’s emotions during a crisis.

Integrated machine learning models for enhancing tropical rainfall prediction using NASA POWER meteorological data

ABSTRACT This research evaluates the performance of deep learning (DL) models in predicting rainfall in George Town, Penang, utilizing the open-source NASA POWER meteorological data, which includes variables such as rainfall, dew point, solar radiation, wind speed, relative humidity, and temperature. This study introduces a newly developed hybrid DL based on the integration of a 2D convolutional neural network (CNN2D) with a bidirectional recurrent neural network (BRNN) and a bidirectional gated recurrent unit (BGRU). The proposed models, CNN2D–BGRU and BRNN–BGRU, were compared against standalone models CNN2D, BRNN, and BGRU. The results indicate that the BRNN–BGRU model is the most effective, with a root mean square error (RMSE) value of 2.59, a mean absolute error (MAE) value of 1.97, a Pearson correlation coefficient (PCC) value of 0.79, and a Willmott index (WI) value of 0.88. In a 3-day prediction, the BRNN–BGRU model also performed the best, with a test WI value of 0.83, a PCC value of 0.69, a RMSE value of 3.02, and MAE value of 2.34. The hybrid BRNN–BGRU model consistently excels in predicting multi-step rainfall in tropical regions using the NASA POWER dataset. These findings can contribute to the development of advanced rainfall-predicting systems for more effective management of water resources and flooding in urban areas.

Automatic Modulation Recognition Based on Multimodal Information Processing: A New Approach and Application

Automatic modulation recognition (AMR) has wide applications in the fields of wireless communications, radar systems, and intelligent sensor networks. The existing deep learning-based modulation recognition models often focus on temporal features while overlooking the interrelations and spatio-temporal relationships among different types of signals. To overcome these limitations, a hybrid neural network based on a multimodal parallel structure, called the multimodal parallel hybrid neural network (MPHNN), is proposed to improve the recognition accuracy. The algorithm first preprocesses the data by parallelly processing the multimodal forms of the modulated signals before inputting them into the network. Subsequently, by combining Convolutional Neural Networks (CNN) and Bidirectional Gated Recurrent Unit (Bi-GRU) models, the CNN is used to extract spatial features of the received signals, while the Bi-GRU transmits previous state information of the time series to the current state to capture temporal features. Finally, the Convolutional Block Attention Module (CBAM) and Multi-Head Self-Attention (MHSA) are introduced as two attention mechanisms to handle the temporal and spatial correlations of the signals through an attention fusion mechanism, achieving the calibration of the signal feature maps. The effectiveness of this method is validated using various datasets, with the experimental results demonstrating that the proposed approach can fully utilize the information of multimodal signals. The experimental results show that the recognition accuracy of MPHNN on multiple datasets reaches 93.1%, and it has lower computational complexity and fewer parameters than other models.