Bidirectional Gated Recurrent Unit Research Articles

Stratification of Depressed and Non-Depressed Texts from Social Media using LSTM and its Variants

This work examines the performance of various LSTM (long short-term memory) variants on social media text data. This study evaluates the performance of LSTM models with different architectures, namely, classic LSTM, Bidirectional LSTM, Stacked LSTM, gated recurrent unit (GRU), and bidirectional GRU, on a social network dataset comprising texts extracted from multiple social media platforms. We aim to identify the most effective LSTM variant of the five considered LSTM models for text analysis through a comparative study of the models’ precision, recall, F1-score, and accuracy. The research findings show that the Classic LSTM and the GRU model perform better than the other models in accuracy. In contrast, the bidirectional models (Bidirectional LSTM and Bidirectional GRU) provide better precision scores than their respective primitive models. This research has significant implications for developing more efficient models for natural language processing applications. It offers beneficial insights into the implications involving the scrutiny of depression on social media platforms through text data analysis.

Data tweet clustering using bidirectional gated recurrent unit and k-prototype for the Indonesian political year

As time passes, social media, which was formerly used as a means of communication between users, is experiencing a transition as a means for broadcasting information, conducting business, advertising, and even political campaigning. In elections, social media is also used to discredit political opponents to reduce the electability of opposing candidate. Spreading hate speech and fake news to undermine the electability of opposing candidate is a common violation of the law committed by supporters of one candidate over another. Considering that the number of social media users increases annually at a very rapid rate, the hazard of social media abuse has the potential to grow. In 2022, Indonesia had 191 million social media users in January 2022. Obviously, this will make the election situation more tumultuous and has the potential to cause societal divisions. The government must have a control system in place to screen social media content that can be considered illegal. In this study, fake news and hate speech are classified using the Bidirectional Gated Recurrent Unit (BiGRU). Lastly, K-Prototype was used to do clustering based on categorization dimensions and probable distribution to identify which clusters had the greatest risk of breaking the law, creating confusion, and dispersing broadly throughout society. It is hoped that the clusters that are created will represent the levels of priority of tweet data that requires prompt attention from the government to prevent it from spreading and inciting social unrest. Based on the results of the analysis, the BiGRU fake news model yields a F1-score of 95%, while the BiGRU hate speech model yields a F1-score of 90%. Clustering data using K-Prototype in this research can reduce the number of tweet data from 13,183 to 1,791 data. These new data are considered as a priority that must be pursued in preventing social media disputes.

A multi-indicator prediction method for NOx emission concentration and ammonia escape value for cement calciner system

It is particularly important to accurately predict NOx emission concentration in cement production. As the cement industry is a process industry, there are many chemical reactions, coupling of front and back processes, and large fluctuations in production, resulting in time-delayed denitration data, strong coupling, and uncertainty, which makes it difficult to adequately extract the characteristics of the denitration data to achieve accurate prediction of NOx concentration. At the same time, excessive ammonia is used in production to reduce NOx concentration, resulting in increased ammonia escape and secondary hazards to the environment, therefore it is necessary to perform accurate multi-step prediction of NOx concentration and ammonia escape. To address these issues, this paper presented a new multicriteria deep learning hybrid model MI-ITBA. The model is constructed by connecting an improved causal dilated temporal convolutional network (ITCN) with bidirectional gated recurrent units (BiGRU) and a special attention mechanism, which enhances the ability to extract features from denitration sequence, using the attention mechanism to weigh the importance of different time steps and highlight the influence of key features on the prediction results. In addition, the time series (TS) of denitration containing time-delay information is designed in the input of the model to minimize the influence of time delay. Dynamic information support for denitration control was provided by multi-step prediction of NOx concentration at seven moments and ammonia escape value at three moments to retain the dynamics of the data. Compared with TCN, BiGRU, and TCN-BiGRU, MI-ITBA has a high prediction accuracy, which can provide an effective guide to realizing NOx emission reduction and accurate ammonia injection.

Sentiment Analysis of Self Driving Car Dataset: A comparative study of Deep Learning approaches

Sentiment Analysis (SA) is a crucial task in understanding public opinions and perceptions towards emerging technologies. In this study, we focus on SA for a self-driving car dataset as it provides valuable insights into public perceptions and opinions towards a transformative technology. The dataset consists of textual reviews associated with sentiment labels, providing insights into how people perceive self-driving car technology. Our objective is to analyze the sentiments expressed in these reviews using Deep Learning (DL) models, namely, Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Gated Recurrent Unit (GRU), and Bidirectional GRU (BiGRU). We compared our results with an existing technique in the field of self-driving car sentiment classification, that implemented various Machine Learning (ML) and DL models, including Support Vector Machines (SVM), Naïve Bayes (NB), Logistic Regression (LR), Random Forest (RF), CNN, and LSTM. In our study, we expanded upon this research by evaluating the performance of ANN, BiLSTM, GRU, and BiGRU. Results reveal that BiLSTM, GRU, and BiGRU exhibit superior performance in sentiment classification within the self-driving car dataset. These findings offer valuable insights into public sentiment towards self-driving cars, contributing significantly to the advancement of SA techniques in the domain of autonomous vehicles. Additionally, the results are statistically tested and are statistically significant.

Integrating Ebola optimization search algorithm for enhanced deep learning-based ransomware detection in Internet of Things security

<abstract> <p>With the widespread use of Internet, Internet of Things (IoT) devices have exponentially increased. These devices become vulnerable to malware attacks with the enormous amount of data on IoT devices; as a result, malware detection becomes a major problem in IoT devices. A reliable and effective mechanism is essential for malware detection. In recent years, research workers have developed various techniques for the complex detection of malware, but accurate detection continues to be a problem. Ransomware attacks pose major security risks to corporate and personal information and data. The owners of computer-based resources can be influenced by monetary losses, reputational damage, and privacy and verification violations due to successful assaults of ransomware. Therefore, there is a need to swiftly and accurately detect the ransomware. With this motivation, the study designs an Ebola optimization search algorithm for enhanced deep learning-based ransomware detection (EBSAEDL-RD) technique in IoT security. The purpose of the EBSAEDL-RD method is to recognize and classify the ransomware to achieve security in the IoT platform. To accomplish this, the EBSAEDL-RD technique employs min-max normalization to scale the input data into a useful format. Also, the EBSAEDL-RD technique makes use of the EBSA technique to select an optimum set of features. Meanwhile, the classification of ransomware takes place using the bidirectional gated recurrent unit (BiGRU) model. Then, the sparrow search algorithm (SSA) can be applied for optimum hyperparameter selection of the BiGRU model. The wide-ranging experiments of the EBSAEDL-RD approach are performed on benchmark data. The obtained results highlighted that the EBSAEDL-RD algorithm reaches better performance over other models on IoT security.</p> </abstract>

Scraper factors investigation on Al2O3 paste flow based on marine predators algorithm-bidirectional gated recurrent unit pseudo-lattice Boltzmann method for stereolithography molding

The coating process is an essential part of stereolithography technology to mold alumina paste, achieved using a scraper. The characteristics of the scraper (geometry and roughness) are investigated to discover the potential influence mechanism on the coating process. To complete the analysis, the bidirectional gated recurrent unit (BiGRU) optimized by the marine predator algorithm (MPA) is proposed to provide a predicted model for the rheological data instead of the traditional fitting by using the known rheological equations. Then, a pseudo-lattice Boltzmann (pseudo-LB) method is proposed to conduct the coating flow simulation without a specific rheological equation, where the predicted rheological model is applied to obtain the relaxation time for each iteration. Based on the above MPA-BiGRU pseudo-LB method, three geometries and different roughnesses are explored for the coating process. The results indicate that the rectangle geometry suits the large coating speed, and the trapezium geometry is recommended for conditions with the small coating speed. The roughness significantly affects the coating process with the rectangle geometry, and the smoother scraper generates a minor disturbance. However, it is almost ineffective in the cases with trapezium geometry.

PV power prediction based on feature selection and GMM clustering

The increase in the installed capacity of photovoltaics (PV) brings great challenges to the stable operation of the power system. To improve the accuracy of PV power prediction, Spearman’s rank correlation coefficient and Grey Relation analysis (GRA) are combined to analyze and screen each meteorological feature to ensure the effectiveness and reasonableness of the input features. Based on the analysis of meteorological features, the Gaussian Mixture Models (GMM) clustering method is proposed to cluster the historical data sets. The historical data are clustered into two categories: simple PV power change and complex PV power change, which facilitates the model to learn the power change characteristics under different weather conditions. A Bidirectional Gated Recurrent Unit (BiGRU) prediction model is built to mine the timing properties of the two types of data and trained and tested separately. By setting several error evaluation indexes for comprehensive judgment, the methods in this paper are better than several other classical prediction methods and have a strong adaptability to different weather conditions.

MIDGNWD : Design of an efficient multimodal iterative deep graph network wearable interface for detecting drowsy drivers

Drowsy driving is a significant contributor to road traffic accidents, necessitating the development of robust and efficient detection systems. Existing approaches for detecting drowsy drivers often rely on single modalities such as image-based facial analysis or vehiclebased metrics, which may lack precision and be susceptible to environmental variations. Furthermore, the existing models might be computationally expensive and result in delayed responses. To address these limitations, this paper proposes a novel Multimodal Iterative Deep Graph Network Wearable Interface that leverages a rich set of data including heart rate, respiration, accelerometer movement, location (speed), traffic, and weather data. These data samples are processed through a fusion of different Recurrent Neural Network processes. The Bidirectional Long Short-Term Memory (BiLSTM) which is used to generate augmented features, is fused with Bidirectional Gated Recurrent Unit (BiGRU) which assists in estimating multidomain feature sets. These features are then used to train an efficient Deep Graph Recurrent Network (DGRN), which facilitates the real-time identification of drowsy drivers. The proposed model significantly enhances the precision of drowsy driver detection by 8.5%, accuracy by 9.5%, recall by 8.3%, and Area Under the Curve (AUC) by 4.9%, while also improving specificity by 5.5% and reducing delay by 10.4%. These improvements collectively contribute to a substantial enhancement in road safety, potentially saving lives by preventing accidents due to drowsy driving. The proposed system, with its real-time processing capability and high accuracy, stands as a significant improvement for identifying drowsy drivers, paving the way for safer and smarter transportation systems.

Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU

Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU

An intelligent chatbot using deep learning with bidirectional GRU and CNN model

An intelligent chatbot using deep learning with bidirectional GRU and CNN model

Snow Depth Retrieval Using Detrended SNR From GNSS-R With Bidirectional GRU

Snow Depth Retrieval Using Detrended SNR From GNSS-R With Bidirectional GRU

Bidirectional Gated Recurrent Unit with Glove Embedding and Attention Mechanism for Movie Review Classification

During the current Internet era, enormous volume of structured and unstructured textual data is generated and exchanged online that made the text classification more crucial. With the massive amount of user-generated content on social media platforms, businesses and organizations can leverage sentiment analysis to understand customer opinions and attitudes towards the products, services, or brand. Text classification is a natural language processing task in which a machine learning model is trained to categorize the text into predefined classes or categories. Due to the advent of promising Deep Learning techniques, intelligent and accurate text classification system can be developed. In movie review classification, algorithms are used to automatically categorize movie reviews into positive, negative, or sometimes neutral sentiments based on the opinions expressed in the text. The goal is to automate the process of determining whether a review reflects a positive or negative sentiment, helping users quickly understand the overall reception of a movie. This work proposes an automated movie review classification system based on Bidirectional Gated Recurrent Unit encoder with attention module. This method encodes the words using GloVe word embedding technique. Context information is captured in an efficient manner and the encoding is more reliable. Bidirectional GRU layer captures long-term dependencies in sentences, followed by a custom Attention layer that assigns higher weights to key components. The attention mechanism allows the model to selectively focus on important parts of the input sequence, by dynamically determining attention weights based on task relevance. The model was individually trained and evaluated using popular benchmark IMDb dataset. Experimental studies have proved the better performance of proposed movie review classification system that obtained accuracy of 98%.

DeepBiG: A Hybrid Supervised CNN and Bidirectional GRU Model for Predicting the DNA Sequence

DeepBiG: A Hybrid Supervised CNN and Bidirectional GRU Model for Predicting the DNA Sequence

Research on Automatic Annotation Method of Korean Language under Data Driving and Fusion

Abstract In the quest to streamline Korean text and speech annotation, this research introduces innovative automatic annotation methods that promise to revolutionize efficiency and technical prowess in constructing Korean annotation datasets. By leveraging the sophisticated Seq2Seq architecture with BERT and bidirectional GRU models, we significantly enhance the model’s ability to grasp contextual nuances, ensuring precise text annotations. The speech annotation frontier benefits from a novel amalgamation of the Hidden Markov Model’s forced alignment and semi-supervised learning, perfected with Seneff auditory features for meticulous phonological consonant boundary detection. Empirical validation across diverse datasets showcases our methodology’s superiority, achieving a remarkable 96.01% accuracy in text annotation and setting a new benchmark for phonological boundary detection at a 14.5ms minimum distance threshold. Our approach outperforms traditional algorithms, marking a pivotal step forward in Korean automatic annotation.

Enhancing User-Item Interaction Through Counterfactual Classifier For Sequential Recommendation

Abstract Sequential recommendation systems are vital for enhancing user experience and efficiency in e-commerce and online platforms by providing personalized suggestions. However, these systems often encounter challenges such as data sparsity and inaccuracy in identifying user preferences, leading to suboptimal recommendation quality. Existing solutions, like data augmentation and causal inference, have attempted to tackle these issues but have frequently neglected the crucial aspect of user feedback, which could potentially misrepresent user interests. In this paper, a new approach is introduced that incorporates a counterfactual classifier, textual information, and user feedback to enhance user-item interactions and the effectiveness of sequential recommendation systems. Our proposal is EUICC-SRec, a new methodology that incorporates a counterfactual classifier, a text module with bidirectional GRU channels, and a cross-attention mechanism. This method is designed to address data sparsity and enhance the precision of sequential recommendations by improving accuracy and relevance. The proposed approach has been evaluated through extensive experiments, demonstrating its superiority in mitigating data sparsity issues and accurately capturing user preferences compared to existing methods.

TimeNet: Time2Vec attention-based CNN-BiGRU neural network for predicting production in shale and sandstone gas reservoirs

With the continuous growth in global population and productivity, the demand for natural gas, the cleanest fossil fuel, is expected to increase significantly. Accurate daily gas production forecasting of shale and sandstone reservoirs ensures a reliable gas supply. However, the complex and non-linear gas data (reservoir and production data) makes this difficult. To address these challenges, we propose a novel model named TimeNet, which utilizes a mix of convolutional neural networks (CNN), bidirectional gated recurrent units (BiGRU), attention mechanisms (AM), and Time2Vec. Time2Vec is integrated to automatically capture important complex and non-linear temporal information and mitigate burdensome time series pre-processing. The CNN layer extracts spatial features influencing gas production, while the BiGRU captures high-level temporal features and irregular trends in the time series data. The AM helps in understanding embedded information for accurate learning. Each component of the TimeNet model serves a distinct function in the prediction task, optimizing its strengths. Testing on two real-world datasets from the Fenchuganj conventional sandstone gas field and the Marcellus shale gas field confirms the proposed model's effectiveness. Comparative analysis demonstrates the superior performance of the proposed model on the two datasets, exhibiting an R2 of 97.25 % and 97.57 % in shale and sandstone gas, respectively.

Enhanced personalized learning exercise question recommendation model based on knowledge tracing

Personalized exercise question recommendation is a crucial aspect of smart education used to customize educational exercises and questions to individual students' distinct abilities and learning progress. Integrating cognitive diagnosis with deep learning has shown promising results in personalized exercise recommendations. However, the black-box nature of the deep learning model hinders their interpretability. This makes it challenging for educators and students to understand the reasons behind the model's predictions for the next problem, and this limits their opportunity to take an active role in improving the learning process. To address this limitation, this article presents a novel personalized exercise question recommendation model based on knowledge tracing. The approach incorporates graph convolutional neural networks to model the student's abilities, thus enhancing the interpretability of the model. By employing Bidirectional gate recurrent unit (Bi-GRU), the model effectively traces fluctuations in students' abilities over time and predicts their responses to exercise questions. Experimental results demonstrate the effectiveness of this model, achieving an accuracy of 90.8% and 92.6% on ASSISTment 2009 and ASSISTment 2017 datasets, containing 4218 and 1709 student records, respectively. Moreover, the experiment was also conducted to validate the model's exercise difficulty setting. Results indicate an acceptable level of effectiveness in generating appropriate difficulty-level recommendations for individual students. The proposed model contributes to advancing personalized exercise recommendations by offering valuable insights that can lead to more efficient and effective student learning experiences.

Short Text Classification Based on Hybrid Semantic Expansion and Bidirectional GRU (BiGRU) based Method to Improve Hate Speech Detection

Short Text Classification Based on Hybrid Semantic Expansion and Bidirectional GRU (BiGRU) based Method to Improve Hate Speech Detection

Im5C-DSCGA: A Proposed Hybrid Framework Based on Improved DenseNet and Attention Mechanisms for Identifying 5-methylcytosine Sites in Human RNA.

5-methylcytosine (m5C) is a key post-transcriptional modification that plays a critical role in RNA metabolism. Owing to the large increase in identified m5C modification sites in organisms, their epigenetic roles are becoming increasingly unknown. Therefore, it is crucial to precisely identify m5C modification sites to gain more insight into cellular processes and other mechanisms related to biological functions. Although researchers have proposed some traditional computational methods and machine learning algorithms, some limitations still remain. In this study, we propose a more powerful and reliable deep-learning model, im5C-DSCGA, to identify novel RNA m5C modification sites in humans. Our proposed im5C-DSCGA model uses three feature encoding methods initially-one-hot, nucleotide chemical property (NCP), and nucleotide density (ND)-to extract the original features in RNA sequences and ensure splicing; next, the original features are fed into the improved densely connected convolutional network (DenseNet) and Convolutional Block Attention Module (CBAM) mechanisms to extract the advanced local features; then, the bidirectional gated recurrent unit (BGRU) method is used to capture the long-term dependencies from advanced local features and extract global features using Self-Attention; Finally, ensemble learning is used and full connectivity is used to classify and predict the m5C site. Unsurprisingly, the deep-learning-based im5C-DSCGA model performed well in terms of sensitivity (Sn), specificity (SP), accuracy (Acc), Matthew's correlation coefficient (MCC), and area under the curve (AUC), generating values of 81.0%, 90.8%, 85.9%, 72.1%, and 92.6%, respectively, in the independent test dataset following the use of three feature encoding methods. We critically evaluated the performance of im5C-DSCGA using five-fold cross-validation and independent testing and compared it to existing methods. The MCC metric reached 72.1% when using the independent test, which is 3.0% higher than the current state-of-the-art prediction method Deepm5C model. The results show that the im5C-DSCGA model achieves more accurate and stable performances and is an effective tool for predicting m5C modification sites. To the authors' knowledge, this is the first time that the improved DenseNet, BGRU, CBAM Attention mechanism, and Self-Attention mechanism have been combined to predict novel m5C sites in human RNA.

An integrated neural network method for landslide susceptibility assessment based on time-series InSAR deformation dynamic features

ABSTRACT We develop an integrated neural network landslide susceptibility assessment (LSA) method that integrates temporal dynamic features of interferometry synthetic aperture radar (InSAR) deformation data and the spatial features of landslide influencing factors. We construct a time-distributed convolutional neural network (TD-CNN) and bidirectional gated recurrent unit (Bi-GRU) to better understand the temporal dynamic features of InSAR cumulative deformation, and construct a multi-scale convolutional neural network (MSCNN) to determine the spatial features of landslide influencing factors, and construct a parallel unified deep learning network model to fuse these temporal and spatial features for LSA. Compared with the traditional MSCNN method, the accuracy of the proposed model is improved by 1.20%. The performance of the proposed model is preferable to MSCNN. The area under the receiver operating characteristic curve (AUC) of the testing set reaches 0.91. Our LSA results show that the proposed model clearly depicts areas with very high susceptibility landslides. Further, only 10.18% of the study area accurately covers 84.79% of historical landslide areas. Subjective consequences and objective indicators show that the proposed model that is integrated time-series InSAR deformation dynamic features can make full use of landslide characteristics and effectively improve the reliability of LSA.