Bidirectional Gated Recurrent Unit Research Articles

ADAPT: Attack detection and prevention via deep learning techniques

Cloud computing is a new technology that provides services to customers anywhere, anytime, under varying conditions and managed by a third-party cloud provider. Even though cloud computing has progressed a lot, some attacks still happen. The recent anomalous and signature attacks use clever strategies such as low-rate attacks and attacking as an authenticated user. In this paper, a novel Attack Detection and Prevention (ADAPT) method is proposed to overcome this issue. The proposed system consists of three stages. An Intrusion Detection System is initially used to check whether there is an attack or not by comparing the IP address in the Blacklist IP Database. If an attack occurs, the IP address will be added to the Blacklist IP database and blocked. The second stage uses Bi-directional LSTM and Bi-directional GRU to check the anomalous and signature attack. In the third stage, classified output is sent to reinforcement learning, if any attack occurs the IP address is added to the blacklist IP database otherwise the packets are forwarded to the user. The proposed ADAPT technique achieves a higher accuracy range than existing techniques.

Hybrid attention-based improved temporal convolutional BiGRU approach for short-term load forecasting

Accurate load forecasting can ensure the safe and reliable operation of power systems, reduce generation costs, and improve economic efficiency. To improve the accuracy and performance of short-term load forecasting, this paper proposes a hybrid short-term load forecasting method composed of an improved temporal convolutional network (TCNPlus) with an attention mechanism and a bidirectional gated recurrent unit (BiGRU). Firstly, the collected pre-processed training data is reconstructed using a fixed-length sliding window. Secondly, using the self-attention mechanism (SA) in the improved TCN to further enhance the weight of key features, and introducing residual connections can allow the input to propagate forward faster and improve the representation ability and efficiency of error backpropagation of the network, to eliminate the impact of interference signals. Finally, BiGRU is used to learn the forward and backward dependencies of the load sequence in both directions and predict the true load value. Based on the real load data of a national power grid in South China, through experimental comparison of multiple models, the results show that this model still has higher short-term load forecasting accuracy with fewer input features.

Multimodal Abstractive Summarization using bidirectional encoder representations from transformers with attention mechanism

In recent decades, abstractive text summarization using multimodal input has attracted many researchers due to the capability of gathering information from various sources to create a concise summary. However, the existing methodologies based on multimodal summarization provide only a summary for the short videos and poor results for the lengthy videos. To address the aforementioned issues, this research presented the Multimodal Abstractive Summarization using Bidirectional Encoder Representations from Transformers (MAS-BERT) with an attention mechanism. The purpose of the video summarization is to increase the speed of searching for a large collection of videos so that the users can quickly decide whether the video is relevant or not by reading the summary. Initially, the data is obtained from the publicly available How2 dataset and is encoded using the Bidirectional Gated Recurrent Unit (Bi-GRU) encoder and the Long Short Term Memory (LSTM) encoder. The textual data which is embedded in the embedding layer is encoded using a bidirectional GRU encoder and the features with audio and video data are encoded with LSTM encoder. After this, BERT based attention mechanism is used to combine the modalities and finally, the BI-GRU based decoder is used for summarizing the multimodalities. The results obtained through the experiments that show the proposed MAS-BERT has achieved a better result of 60.2 for Rouge-1 whereas, the existing Decoder-only Multimodal Transformer (D-MmT) and the Factorized Multimodal Transformer based Decoder Only Language model (FLORAL) has achieved 49.58 and 56.89 respectively. Our work facilitates users by providing better contextual information and user experience and would help video-sharing platforms for customer retention by allowing users to search for relevant videos by looking at its summary.

ICNCS: Internal Cascaded Neuromorphic Computing System for Fast Electric Vehicle State-of-Charge Estimation

Accuracy and speed of Lithium-ion battery state of charge (SOC) estimation determine the reliability and stability of electric vehicle (EV), as well as promoting the development of smart home energy management system. Existing SOC estimation approaches embedded in commercial EVs still suffer from limitations of low precision, effectiveness, and relatively low robustness. To address these issues, we propose an internal cascaded neuromorphic computing system (ICNCS) via memristor circuits for EV SOC estimation. Specifically, we use three circuit modules to facilitate the design of the proposed ICNCS. Firstly, a bidirectional gated recurrent unit (Bi-GRU) circuit module is designed, enabling adequate feature extraction from the time-contextual battery information. Secondly, an attention circuit module is proposed to distinguish the useful and unimportant information related to SOC estimation. Thirdly, the Kalman filter module is constructed to eliminate the random noise caused by data transmission and processing. Finally, a series of experiments and analysis demonstrate that the proposed ICNCS has good performances in terms of accuracy (the lowest achievable RMSE and MAE are 0.853 and 0.711, respectively), time efficiency (approximately 16 20 times), and robustness (anti-noise capacity), indicating an advancement in consumer electronics applications.

An Explainable and Resilient Intrusion Detection System for Industry 5.0

Industry 5.0 is a futuristic transformative model that aims to develop a hyperconnected, automated, and data-driven industrial ecosystem. This digital transformation will boost productivity and efficiency throughout the production process but will be more prone to new sophisticated cyber-attacks. Deep learning-based Intrusion Detection Systems (IDS) have the potential to recognize intrusions with high accuracy. However, these models are complex and are treated as a black box by developers and security analysts due to the inability to interpret the decisions made by these models. Motivated by the challenges, this paper presents an explainable and resilient IDS for Industry 5.0. The proposed IDS is designed by combining bidirectional long short-term memory networks (BiLSTM), a bidirectional-gated recurrent unit (Bi-GRU), fully connected layers and a softmax classifier to enhance the intrusion detection process in Industry 5.0. Then, we employ the SHapley Additive exPlanations (SHAP) mechanism to interpret and understand the features that contributed the most in the decision of the proposed cyber-resilient IDS. The evaluation of the proposed model using the explainability can ensure that the model is working as expected. The experimental results based on the CICIDDoS2019 dataset confirms the superiority of the proposed IDS over some recent approaches.

Unmanned aerial vehicle fault diagnosis based on ensemble deep learning model

To address the problems of external interference during unmanned aerial vehicle (UAV) flight and the low accuracy and weak generalization ability of the current single fault diagnosis model, this work proposes a weighted ensemble deep learning UAV fault diagnosis method. First, considering the differences in training methods and fault feature recognition principles of deep networks with different structures, three hybrid fault diagnosis models are constructed. These models are constructed by combining convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM) and bidirectional gate recurrent unit (BiGRU). CNN is used to extract the features of the UAV flight data and the obtained feature information is fed into BiLSTM and BiGRU to explore the fault information inherent in the time series data. Then, the three hybrid fault diagnosis models are used as the individual model of the ensemble learning algorithm, and the weights of the three individual models are optimized using a random grid search algorithm to construct a UAV fault diagnosis model based on hybrid deep learning weighted ensemble, which further improves the fault diagnosis performance. Finally, it is demonstrated experimentally that the proposed hybrid deep learning weighted ensemble fault diagnosis model can effectively identify the fault of UAV with an accuracy of 99.22% and 99.62% on binary and multivariate classification, respectively, and reflects better generalization performance in the metrics of precision, recall, and F1 score.

Tool wear prediction through AI-assisted digital shadow using industrial edge device

Flank wear of drilling tools in manufacturing is among the main factors affecting product quality and productivity. In this study, an AI-assisted digital shadow was created for the instant prediction of the drilling flank tool wear. The drilling data were collected simultaneously using an industrial edge device and a rotary dynamometer. Feature engineering was conducted on the collected data from devices in time and frequency domains. A recurrent neural network (RNN) based on bidirectional long short-term memory (Bi-LSTM) and bidirectional gated recurrent unit (Bi-GRU) architectures was implemented on specified tool wear regions dataset. The digital shadow was created using the industrial edge device and the predictive AI model to minimize costs by reducing the need for expensive multi-sensors, manufacturing downtime, and tool underuse or overuse in a smart manufacturing system. The proposed model predicts with high accuracy and computational time efficiency and can be integrated into digital twin systems.

Convolutional bidirectional GRU for dynamic functional connectivity classification in brain diseases diagnosis

Dynamic functional connectivity (DFC) classification is helpful for computer-aided diagnosis of brain diseases. In recent years, DFC classification based on deep learning has drawed increasing attention. However, how to effectively extract the deep spatio-temporal features of DFC to improve classification performance is still a very challenging research topic. To this end, this paper proposes a DFC classification method based on convolutional bidirectional gated recurrent unit, called DFC-CBGRU, which mainly includes three key operations: multi-scale topological features extraction, bidirectional spatio-temporal feature extraction, and feature fusion. Firstly, the proposed method uses convolutional neural network (CNN) to extract the multi-scale topological features composed of node-level, module-level, and graph-level features from functional connectivity network at each time point. Then, it employs bidirectional gated recurrent unit (GRU) to extract the bidirectionally dependent spatio-temporal features from the obtained time series of multi-scale topological features. Finally, it utilizes the one-dimensional CNN to fuse forward and backward spatio-temporal features to obtain the joint spatio-temporal features for classification. Experimental results on multiple brain diseases datasets show that the proposed method has a superior classification performance over other methods and is promising for extracting the discriminative FCs related to brain diseases accurately.

Forecasting model for short-term wind speed using robust local mean decomposition, deep neural networks, intelligent algorithm, and error correction

Wind power generation has aroused widespread concern worldwide. Accurate prediction of wind speed is very important for the safe and economic operation of the power grid. This paper presents a short-term wind speed prediction model which includes data decomposition, deep learning, intelligent algorithm optimization, and error correction modules. First, the robust local mean decomposition (RLMD) is applied to the original wind speed data to reduce the non-stationarity of the data. Then, the salp swarm algorithm (SSA) is used to determine the optimal parameter combination of the bidirectional gated recurrent unit (BiGRU) to ensure prediction quality. In order to eliminate the predictable components of the error further, a correction module based on the improved salp swarm algorithm (ISSA) and deep extreme learning machine (DELM) is constructed. The exploration and exploitation capability of the original SSA is enhanced by introducing a crazy operator and dynamic learning strategy, and the input weights and thresholds in the DELM are optimized by the ISSA to improve the generalization ability of the model. The actual data of wind farms are used to verify the advancement of the proposed model. Compared with other models, the results show that the proposed model has the best prediction performance. As a powerful tool, the developed forecasting system is expected to be further used in the energy system.

Fault Detection and Diagnosis of a Photovoltaic System Based on Deep Learning Using the Combination of a Convolutional Neural Network (CNN) and Bidirectional Gated Recurrent Unit (Bi-GRU)

The meticulous monitoring and diagnosis of faults in photovoltaic (PV) systems enhances their reliability and facilitates a smooth transition to sustainable energy. This paper introduces a novel application of deep learning for fault detection and diagnosis in PV systems, employing a three-step approach. Firstly, a robust PV model is developed and fine-tuned using a heuristic optimization approach. Secondly, a comprehensive database is constructed, incorporating PV model data alongside monitored module temperature and solar irradiance for both healthy and faulty operation conditions. Lastly, fault classification utilizes features extracted from a combination consisting of a Convolutional Neural Network (CNN) and Bidirectional Gated Recurrent Unit (Bi-GRU). The amalgamation of parallel and sequential processing enables the neural network to leverage the strengths of both convolutional and recurrent layers concurrently, facilitating effective fault detection and diagnosis. The results affirm the proposed technique’s efficacy in detecting and classifying various PV fault types, such as open circuits, short circuits, and partial shading. Furthermore, this work underscores the significance of dividing fault detection and diagnosis into two distinct steps rather than employing deep learning neural networks to determine fault types directly.

Space Aware BGRU Microservice Fault Detection Algorithm

Microservice architecture is a new architecture pattern, which aims to provide users with more reliable, maintainable, and extensible software design services. However, with the continuous expansion of the scale of microservice application system, the proliferation of services and service interactions in the system make the system fault detection difficult. Detecting faults accurately and effectively is the key technology to ensure the system reliability and stability. From the perspective of microservice operation status and dependencies between services, this paper proposes a space-aware bidirectional gated recurrent unit (BGRU) microservice fault detection algorithm, which uses deep learning technology to mine hidden information that causes failures and combines space-aware attention to establish long-distance spatial dependency to improve the accuracy of model detection. The paper also conducts many experiments to demonstrate the effectiveness of the algorithm in microservice fault detection.

Integrated Model Text Classification Based on Multineural Networks

Based on the original deep network architecture, this paper replaces the deep integrated network by integrating shallow FastText, a bidirectional gated recurrent unit (GRU) network and the convolutional neural networks (CNNs). In FastText, word embedding, 2-grams and 3-grams are combined to extract text features. In recurrent neural networks (RNNs), a bidirectional GRU network is used to lessen information loss during the process of transmission. In CNNs, text features are extracted using various convolutional kernel sizes. Additionally, three optimization algorithms are utilized to improve the classification capabilities of each network architecture. The experimental findings using the social network news dataset demonstrate that the integrated model is effective in improving the accuracy of text classification.

Deep Learning in Carbon Neutrality Forecasting

With the growing urgency of global climate change, carbon neutrality, as a strategy to reduce greenhouse gas emissions into the atmosphere, is increasingly seen as a critical solution. However, current forecasting models still face significant challenges and limitations in accurately and effectively predicting carbon emissions and their associated effects. These challenges largely stem from the complexity of carbon emission data and the interplay of anthropogenic and natural factors. To overcome these obstacles, the authors introduce an advanced forecasting model, the SSA-Attention-BIGRU network. This model ingeniously integrates an external attention mechanism, bidirectional GRU, and SSA components, allowing it to synthesize various key factors and enhance prediction accuracy when forecasting carbon neutrality trends. Through experiments on multiple datasets, the results demonstrate that, compared to other popular methods, the SSA-Attention-BIGRU network significantly excels in prediction accuracy, robustness, and reliability.

From Text to Threat Detection: The Power of NLP in Cybersecurity

Natural Language Processing (NLP) is increasingly vital in cybersecurity, enabling the analysis of unstructured data from digital communications to enhance threat detection. This paper is specifically concerned with the changes made by applying natural language processing in conjunction with the machine learning approach in order to demonstrate vulnerability detection. Surprisingly, yet again, an independent phishing detection model with high accuracy level a level of accuracy of 97% was also designed. 39 % with Bidirectional Gated Recurrent Units (BiGRU), and the generated method for cyber threat intelligence records possesses more than 93 % of accuracy, precision, and recall. The findings show effectiveness in the integration of NLP with knowledge bases for further improvement of the meta search on data whereas, the advancement of greater models remains a challenge in the present time. To the extent of this research, this work Chen NLP and get information regarding its feasibility with present and new cybersecurity technologies.

Prediction of Particulate Matter PM2.5 Using Bidirectional Gated Recurrent Unit with Feature Selection

<p>In recent years, air pollution has increased with industrialization and urbanization globally. It is an important hazardous factor that causes severe health issues to community’s health. Among the number of pollutants in air, PM2.5 is very dangerous due to its very small, 2.5µm, diameter. The PM2.5 concentration in air causes severe life-threatening to humans. In this paper, RFBIGRU model is proposed to predict PM2.5 in the atmospheric air. RFBIGRU improves PM2.5 prediction accuracy using Random Forest (RF) feature selector and Bidirectional Gated Recurrent Unit (BIGRU) deep neural network. The PM2.5 concentration in air depends on other pollutants' concentration in the air. However, the consideration of several other pollutants increases the curse of dimensionality and overfitting issues. So, in RFBIGRU, first, the relevant pollutants to PM2.5 are identified using random forest feature importance. Then the nonlinear and temporal patterns of the time series air pollutant data are extracted both in forward and backward direction using Bidirectional GRU. The RFBIGRU reduces the curse of dimensionality, overfitting and improves the PM2.5 prediction accuracy compared to other deep learning methods. The experimental result proves RFBIGRU outperforms others by producing least Root Mean Square Error of 42.217 and 6.813 for Delhi and Amaravathi regions.</p>

Forecast of solar cycle 25 based on Hybrid CNN-Bidirectional-GRU (CNN-BiGRU) model and Novel Gradient Residual Correction (GRC) technique

In this study, a hybrid deep-learning(DL) model is proposed, which consists of a Convolution-Neural-Network (CNN) and Bidirectional-Gated-Recurrent-Unit (Bi-GRU) for the prediction of sunspot numbers(SSN) of different frequencies along with novel post-processing techniques of Gradient-Residual-Correction (GRC) to enhance the accuracy of the predictions further. GRC is utilized to reduce the residual present in the forecasted values. The AdaBoost regression model is implemented over the residual obtained from the prediction of training data using a hybrid CNN-BiGRU model with respect to the gradient of the training data points to predict the residual for the test data points. Ultimately, the predicted residual for test data points is summed up with the predicted test data points to achieve the final predictions. The results obtained from the proposed methods are compared with the results obtained from traditional DL models. The validation of the proposed method is carried out based on four performance metrics, namely Root Mean Squared Error(RMSE), Mean Absolute Scaled Error(MASE), Mean Absolute Error(MAE), and Mean Absolute Percentage Error(MAPE). A significant percentage of improvement is observed while using the GRC technique in comparison to all other experimented models for all four variants of SSN data. A maximum improvement of 85.71% has been achieved in comparison to the BiLSTM model over the 13-month smoothed SSN dataset on the basis of MAPE. Friedman Ranking is also performed as a non-parametric statistical test over the results of the performance measures. This model has been utilized for the forecast of solar cycle 25(SC25) over the annual mean of total SSN. It has been observed that the SC25 is expected to reach its peak in the year 2024 with an annual average peak value of 143.641. Comparative analysis of SC25 and the peak of SSN in the ongoing cycle with the previous works are also carried out.

Diverse ensemble classifier driven Email spam classification using multiple word embedding’s with COCOB optimizer

Spam Email is a serious concern which can steal user’s personal information and cause huge financial loss due to the increasing rate of internet users. Therefore, the demand for accurate spam filtering has become more sophisticated for the Email spam detection. In the existing techniques, it is difficult to intricate the relationship between words in the Email using certain word embedding techniques and learning rate tuning is one of the greatest challenges of stochastic optimization. To overcome this difficulty, the proposed framework uses diverse ensemble based Email spam classification by incorporating multiple word embedding’s with Continuous Coin Betting optimizer. Word2Vec is used to produce the first set of 200D, next set of 200D word embedding is produced by Glove and 768D is produced by using Bidirectional Encoder Representations from Transformers (BERT) respectively. After generating word embedding, then it is classified through diverse ensemble based classifier with base level classifier consists of Long Short Term Memory (LSTM) Networks, Gated Recurrent Unit (GRU) and Bi-directional Gated Recurrent Unit (Bi-GRU) and LSTM as Meta-classifier using COCOB optimizer. Experiments were conducted on 3 benchmark Email dataset and result shows that the proposed system outperforms well with a low false positive rate.

Sentiment Analysis Using Deep Learning Approaches on Multi-Domain Dataset in Telugu Language

Recent advancements in Natural Language Processing (NLP) have made sentiment analysis an essential component of a variety of NLP jobs, including recommendation systems, question answering, and business intelligence products. While sentiment analysis research has, to put it mildly, been widely pursued in English, Telugu has barely ever attempted the task. The majority of research works concentrate on analysing the sentiments of Tweets, news, or reviews containing Hindi and English words. There is a growing interest among academics in studying how people express their thoughts and views in Indian languages like Bengali, Telugu, Malayalam, Tamil and so on. Due to a paucity of labelled datasets, microscopic investigation on Indian languages has been published to our knowledge. This work suggested a sentence-level sentiment analysis on multi-domain datasets that has been collected in Telugu. Deep learning models have been used in this work because it demonstrates the significant expertise in sentiment analysis and is widely regarded as the cutting-edge model in Telugu Sentiment Analysis. Our proposed work investigates a productive Bidirectional Long Short-Term Memory (BiLSTM) Network and Bidirectional GRU Network (BiGRU) for improving Telugu Sentiment Analysis by encapsulating contextual information from Telugu feature sequences using Forward-Backward encapsulation. Further, the model has been deployed by merging the domains so as to predict the accuracy and other performance metrics. The experimental test findings show that the deep learning models outperform when compared with the baseline traditional ML methods in four benchmark sentiment analysis datasets. There is evidence that the proposed sentiment analysis method has improved precision, recall, F1-score and accuracy in certain cases. The proposed model has achieved the F1-score of 86% for song datasets when compared with the other existing models.

Proactive auto-scaling technique for web applications in container-based edge computing using federated learning model

Edge computing has emerged as an attractive alternative to traditional cloud computing by utilizing processing, network, and storage resources close to end devices, such as Internet of Things (IoT) sensors. Edge computing is still in its infancy, and resource provisioning and service scheduling remain research concerns. Kubernetes is a container orchestration tool in distributed environments. Proactive auto-scaling techniques in Kubernetes improve utilization by allocating resources based on future workload prediction. However, prediction models run on central cloud nodes, necessitating data transfer between edge and cloud nodes, which increases latency and response time. We present FedAvg-BiGRU, a proactive auto-scaling method in edge computing based on FedAvg and multi-step prediction by a Bidirectional Gated Recurrent Unit (BiGRU). FedAvg is a technique for training machine learning models in a Federated Learning (FL) model. FL reduces network traffic by exchanging only model updates rather than raw data, relieving learning models of the need to store data on a centralized cloud server. In addition, a technique for determining the number of Kubernetes pods based on the Cool Down Time (CDT) concept has been developed, preventing contradictory scaling actions. To our knowledge, our work is the first to employ FL for proactive auto-scaling in cloud and edge computing. The results demonstrate that the FedAvg-BiGRU method has a slightly higher prediction error than the load centralized processing mode, although this difference is not statistically significant. At the same time, it reduces the amount of data transmission between the edge nodes and the cloud server.

Evaluating and forecasting methods for assessing the health status of cables under the load of large-scale electric vehicle charging

The assessment of the health status and prediction of the lifespan of cable equipment are critical for ensuring the stability and efficiency of the power grid. This paper develops a temperature-current-capacity-life calculation model for cables, considering the fast and slow charging demands of electric vehicles (EVs). Analyses under scenarios of rapid and slow charging demands are conducted, introducing a cable health index and establishing a health status assessment framework based on this index. The framework accounts for various factors leading to cable faults, offering a comprehensive evaluation of the health status of cables with different fault rates. Building upon this, a prediction method using the Fire Hawk Optimization (FHO) Algorithm and Convolutional Neural Network (CNN) is proposed. This method enhances performance by optimizing the hyperparameters of Bidirectional Gated Recurrent Unit (BiGRU) through FHO, effectively searching and determining the optimal hyperparameter configuration. The impact of different scenarios and varying EV penetration rates on cable temperature is analyzed through case studies, facilitating the assessment and prediction of health status.