Bi-GRU Model Research Articles

Music Emotion Recognition Model Using Gated Recurrent Unit Networks and Multi-Feature Extraction

A large number of music platforms have appeared on the Internet recently. The deep learning framework for music recommendation is still very limited when it comes to accurately identifying the emotional type of music and recommending it to users. Languages, musical styles, thematic scenes, and the ages to which they belong are all common classifications. And this is far from sufficient, posing difficulties in music classification and identification. As a result, this paper uses the methods of music emotion multi-feature extraction, BiGRU model design, and music theme scene classification model design to improve the accuracy of music emotion recognition. It develops the BiGRU emotion recognition model and compares it to other models. BiGRU can correctly identify happy and sad emotion music up to 79 percent and 81.01 percent of the time, respectively. It goes far beyond Rnet-LSTM, and the greater the difference in emotion categories, the easier it is to analyze the feature sequence containing emotional features and the higher the recognition accuracy. This is especially evident in the accuracy with which happiness and sadness are recognized. It can meet users' needs for music recognition in a variety of settings.

A Short Text Similarity Calculation Method Combining Semantic and Headword Attention Mechanism

Short text similarity computation plays an important role in various natural language processing tasks. Siamese neural networks are widely used in short text similarity calculation. However, due to the complexity of syntax and the correlation between words, siamese networks alone cannot achieve satisfactory results. Many studies show that the use of an attention mechanism will improve the impact of key features that can be utilized to measure sentence similarity. In this paper, a similarity calculation method is proposed which combines semantics and a headword attention mechanism. First, a BiGRU model is utilized to extract contextual information. After obtaining the headword set, the semantically enhanced representations of the two sentences are obtained through an attention mechanism and character splicing. Finally, we use a one-dimensional convolutional neural network to fuse the word embedding information with the contextual information. The experimental results on the ATEC and MSRP datasets show that the recall and F1 values of the proposed model are significantly improved through the introduction of the headword attention mechanism.

Water quality forecasting based on data decomposition, fuzzy clustering and deep learning neural network

Water quality forecasting can provide useful information for public health protection and support water resources management. In order to forecast water quality more accurately, this paper proposes a novel hybrid model by combining data decomposition, fuzzy C-means clustering and bidirectional gated recurrent unit. Firstly, the original water quality data is decomposed into several subseries by empirical wavelet transform, and then, the decomposed subseries are recombined by fuzzy C-means clustering. Next, for each clustered series, bidirectional gated recurrent unit is applied to develop prediction model. Finally, the forecast result is obtained by the summation of the predictions for the subseries. The proposed forecast model is evaluated by the water quality data of Poyang Lake, China. Results show that the proposed forecast model provides highly accurate forecast result for all of the six water quality data: the average of MAPE of the forecast results for the six water quality datasets is 4.59% for 7 day ahead prediction. Furthermore, our model shows better forecast performance than the other models. Particularly, compared with the single BiGRU model, MAPE decreased by 32.86% in average. Results demonstrate that the proposed forecast model can be used effectively for water quality forecasting.

Network Sentiment Analysis of College Students in Different Epidemic Stages Based on Text Clustering.

In order to analyze the evolution trend of public opinion in emergencies and explore its evolution law, this paper constructs a network sentiment analysis model based on text clustering, where the emotion analysis part is based on the pretraining BERT model and BiGRU model, in which BERT is used as the word embedding model to extract the feature vector of emotional text and BiGRU is used to extract the context of the text feature vector to accurately identify the sentiment polarity of public opinion data. In addition, the K-means clustering algorithm and Kolmogorov-Smirnov Z test were used to divide the different epidemic stages. Compared with other methods, the model proposed in this paper has a great degree of improvement in accuracy, recall, and F1 score index, which provides an opportunity reference and effective detection means for schools at all levels to carry out timely mental health education and psychological intervention for students.

Combining FastText and Glove Word Embedding for Offensive and Hate speech Text Detection

Over the past decade, increased use of social media has led to an increase in hate content. To address this issue new solutions must be implemented to filter out this kind of inappropriate content. Because manual filtering is difficult, several studies have been conducted in order to automate the process.This paper introduces a method based on a combination of Glove and FastText word embedding as input features and a BiGRU model to identify hate speech from social media websites.The obtained results show that our proposed model (BiGRU Glove FT) is effective in detecting inappropriate content. This model detect hate speech on OLID dataset, using an effective learning process that classifies the text into offensive and not offensive language. The performance of the system attained 84%, 87%, 93%, 90% accuracy, precision, recall, and f1-score respectively.

Flood Discharge Prediction Based on Remote-Sensed Spatiotemporal Features Fusion and Graph Attention

Floods have brought a great threat to the life and property of human beings. Under the premise of strengthening flood control engineering measures and following the strategic thinking of sustainable development, many achievements have been made in flood forecasting recently. However, due to the complexity of the traditional lumped model and distributed model, the hydrologic parameter calibration process is full of difficulties, leading to a long development cycle of a reasonable hydrologic prediction model. Even for modern data-driven models, the spatial distribution characteristics of the rainfall data are also not fully mined. Based on this situation, this paper abstracts the rainfall data into the graph structure data, uses remote sensing images to extract the elevation information, introduces the graph attention mechanism to extract the spatial characteristics of rainfall, and employs long-term and short-term memory (LSTM) network to fuse the spatial and temporal characteristics for flood prediction. Through well-designed experiments, the forecasting effect of flood peak value and flood arrival time is verified. Furthermore, compared with the LSTM model and BIGRU model without spatial feature extraction, the advantages of spatiotemporal feature fusion are highlighted. The specific performance is that the RMSE (the root means square error) and R2 (coefficient of determination) of the GA-RNN model have been significantly improved. Finally, we conduct experiments on the observed ten rainfall events in the history of the target watershed. According to the hydrological prediction specifications, the model can be evaluated as a Class B flood forecasting model.

A hybrid approach to classifying Wikipedia article quality flaws with feature fusion framework

Article quality has always been a major concern for Wikipedia. To improve article quality, it is critical to first identify defects. Thus, flaw classification has attracted considerable attention. To achieve this, several machine-learning-based approaches are available, including deep learning models based on either manually constructed or autoextracted features. However, adopting only features of either single type may not ensure a comprehensive description of articles. To improve flaw classification, we propose a feature fusion framework combining both handcrafted and autoextracted features. In this research, we first use a rule-based method from a previously proposed framework to extract handcrafted features. Additionally, we obtain autoextracted features using Bidirectional Encoder Representations from Transformers (BERT) and various deep learning models, including bidirectional long short-term memory (Bi LSTM), bidirectional gated recurrent unit (Bi GRU), bidirectional recurrent neural network (Bi RNN), and multihead self-attention models. Finally, the handcrafted features are standardized and concatenated with the autoextracted features. Then, the concatenated features are fed into a feedforward neural network for classification. A detailed comparison of different classifiers is conducted. We compare 12 different classifiers in terms of training performance, classification performance, and model training time. The experiments show that the proposed feature fusion framework can notably improve the effectiveness of quality flaw classification for Wikipedia articles. In particular, a Bi GRU model based on the proposed framework achieves excellent classification accuracy.

Transfer learning for remaining useful life prediction of multi-conditions bearings based on bidirectional-GRU network

Remaining useful life (RUL) prediction, has been a hotspot topic in the engineering field, which can ensure the security, availability, and continuous efficiency of the system. Different degradation trajectories of bearings under various working conditions may lead to the problem of inconsistent feature distribution and difficult acquisition of corresponding training labels, which affects the validity and accuracy of the prediction model. In this paper, a new transfer learning method based on bidirectional Gated Recurrent Unit (TBiGRU) is proposed to accurately predict the RUL of bearings under different working conditions. Firstly, based on dynamic time wraping (DTW) and Wasserstein distance to construct a comprehensive evaluation index of feature, the selection of transferable feature is carried out. Then a new index of energy entropy moving average cross-correlation based on maximal overlap discrete wavelet transform (MODWT) is proposed to realize adaptive recognition of bearings running states and the acquisition of corresponding training labels, which can also get rid of the constraint of setting threshold. Finally, transfer learning is carried out on the BiGRU model to solve the problem of distribution discrepancy, and timing information is also taken into account. The method is applied to the analysis of experimental data, and the results show that the framework can adaptively recognize different running states of bearings and obtain corresponding training labels, and at the same time realize better RUL prediction performance under different working conditions.

RoBERTa-IAN for aspect-level sentiment analysis of product reviews

Accurately mining the emotional information contained in product reviews is of great significance to product sales. Existing research on sentiment analysis of product reviews often ignores the importance of modelling aspect words and context separately. Therefore, this paper proposes a sentiment analysis model for product reviews based on RoBERTa-IAN. Firstly, the context and aspect words of product reviews are transformed into low dimensional vectors through the RoBERTa pre-training model. Then, the low dimensional vector is used as the input of Bi-GRU model to extract semantic features and get the hidden representation. Finally, the attention matrices of context and aspect words are obtained by using the Interactive Attention Networks (IAN), which are used as the input of sentiment classification layer to analyse and classify the sentiment polarity of product reviews. The experimental results on the real commodity Chinese dataset show that the accuracy of RoBERTa-IAN has reached more than 90%.

Research on Text Classification Based on BERT-BiGRU Model

Text classification is a typical application of natural language processing. At present, the most commonly used text classification method is deep learning. Meanwhile there are many difficulties in natural language processing, such as metaphor expression, semantic diversity and grammatical specificity. To solve these problems, this paper proposes the structure of BERT-BiGRU model. First, use the BERT model instead of the traditional word2vec model to represent the word vector, the word representation is calculated according to the context information, and it can be adjusted according to the meaning of word while the context information is fused. Secondly BiGRU model is attached to the BERT model, BiGRU model can extract the text information features from both directions at the same time. Multiple sets of experiments were set up and compared with the model proposed in this paper, according to the final experimental results, using the proposed BERT-BiGRU model for text classification, the final accuracy, recall and F1 score were all above 0.9. It shows that BERT-BiGRU model has good performance in the Chinese text classification task.

Customer Churn Prediction using Sandpiper Optimization with Bidirectional Gated Recurrent Unit for Business Intelligence

Recent innovation in business intelligence (BI) assists companies to stay successful and competitive with the increasing business trend. Businesses have started to examine the succeeding level of data analytics and BI solution. At the same time, Customer Churn Prediction (CCP) is an essential procedure involved in business decision making that effectually determines the churn clients and performs adequate processes to retain customers. With this motivation, this paper presents a sandpiper optimization with bidirectional gated recurrent unit (SPO-BiGRU) for CCP on BI applications. The SPO-BiGRU model aims for determining the occurrence of customers into churner or non-churner. In addition, the SPO-BiGRU technique involves pre-processing, classification, and hyperparameter optimization. Followed by, the BiGRU model is applied to perform the predictive process. At last, the SPO algorithm is applied to optimally adjust the hyperparameters involved in the BiGRU model. For validating the enhanced performance of the SPO-BiGRU method, a wide range of simulations take place and the results are inspected under varying aspects. The experimental results portrayed the supremacy of the SPO-BiGRU technique over the recent state of art approaches.

Sentiment Analysis of Film Reviews Based on BI-GRU+Attention+Capsule Fusion

In this paper, we introduce a novel model called BGAC for text semantic analysis task. The proposed network is a new integration of two Bi-GRU, self-Attention mechanism and Capsule network architecture. In our experiment on the task of sentiment analysis in dataset IMDB (a public film review dataset), our model achieve the state-of-the-art results. We compare it with six other deep learning models, such as LSTM, CNN, GRU, BI-GRU, CNN+GRU and GRU+CNN. The results of the experiments show that the experimental effect of the bidirectional GRU fusion self-attention mechanism and the capsule network outperforms than the other six neural network models. In addition, the experiments show that combination of GRU with CNN is better than that combination of CNN and GRU, and the combination of CNN with GRU performs better than employ CNN model alone. The accuracy of using single CNN is successively higher than that of LSTM, BI-GRU and GRU model. Our model which the combination of the BI-GRU, Attention and Capsule network introduced in this paper achieves the highest accuracy, precision and F1 score. In conclusion, the bidirectional GRU with self-attention mechanism and capsule network model significantly improves the accuracy of text sentiment classification task.

Bi-GRU model based on pooling and attention for text classification

Aiming at the problems that most of the text classification models based on neural network are easy to overfit and ignore key words in sentences in the training process, an improved text classifica...

Mine Gas Concentration Forecasting Model Based on an Optimized BiGRU Network.

To improve the utilization of mine gas concentration monitoring data with deep learning theory, we propose a gas concentration forecasting model with a bidirectional gated recurrent unit neural network (Adamax-BiGRU) using an adaptive moment estimation maximum (Adamax) optimization algorithm. First, we apply the Laida criterion and Lagrange interpolation to preprocess the gas concentration monitoring data. Then, the MSE is used as the loss function to determine the parameters of the hidden layer, hidden nodes, and iterations of the BiGRU model. Finally, the Adamax algorithm is used to optimize the BiGRU model to forecast the gas concentration. The experimental results show that compared with the recurrent neural network, LSTM, and gated recurrent unit (GRU) models, the error of the BiGRU model on the test set is reduced by 25.58, 12.53, and 3.01%, respectively. Compared with other optimization algorithms, the Adamax optimization algorithm achieved the best forecasting results. Thus, Adamax-BiGRU is an effective method to predict gas concentration values and has a good application value.

Research on Methods for Complex Chinese Entity Recognition

Named entity recognition(NER), as a key task of natural language processing, is of great significance in relational capture, information retrieval, knowledge mapping, machine translation, and question and answer systems. In the neural network-based Chinese named entity recognition method, the vectorized word representation only maps words into one vector, which cannot represent the above problem, In order to solve these problems, this paper proposes a Cascading model based on the character vectorization, This method uses BERT to represent the ambiguity of the word. In the agile BIGRU model, the initial entity recall is completed to increase the entity recall rate. Then the output of this layer is input into the high-level bilstm model of self attention mechanism to further filter and improve the accuracy of the final recognition. Finally, the CRF is used to complete the output of the final recognition result. Experiments show that the model can effectively improve the accuracy, accuracy and recall rate of complex Chinese entity recognition. The f1 value has increased by about 2% overall.

Spam Detection using Recurrent Neural Networks

Spam is well defined as the unsolicited bulk messages or junk mail will send to email address or phone number that are generally marketable in nature and also carry malicious documents. The main issue of spam is that it can download malicious files which can attack the computers, smart phones and networks, utilize network bandwidth and storage space, degrades email servers and can cause attacks in our devices like spyware, phishing and ransomware. In the existing approach, an exploratory analysis of supervised machine learning algorithms has done and the performance has been evaluated. The drawback of existing approach is that the performance of supervised machine learning algorithms decreases as we increase the size of the dataset. In order to overcome such drawbacks, an efficient spam detection using recurrent neural networks using the BiGRU model has been proposed. By implementing this, it has been achieved with better accuracy of 99.07%. From this, it is concluded that BiGRU model has better performance than existing approaches.

Ensemble Deep Learning Models for Heart Disease Classification: A Case Study from Mexico

Heart diseases are highly ranked among the leading causes of mortality in the world. They have various types including vascular, ischemic, and hypertensive heart disease. A large number of medical features are reported for patients in the Electronic Health Records (EHR) that allow physicians to diagnose and monitor heart disease. We collected a dataset from Medica Norte Hospital in Mexico that includes 800 records and 141 indicators such as age, weight, glucose, blood pressure rate, and clinical symptoms. Distribution of the collected records is very unbalanced on the different types of heart disease, where 17% of records have hypertensive heart disease, 16% of records have ischemic heart disease, 7% of records have mixed heart disease, and 8% of records have valvular heart disease. Herein, we propose an ensemble-learning framework of different neural network models, and a method of aggregating random under-sampling. To improve the performance of the classification algorithms, we implement a data preprocessing step with features selection. Experiments were conducted with unidirectional and bidirectional neural network models and results showed that an ensemble classifier with a BiLSTM or BiGRU model with a CNN model had the best classification performance with accuracy and F1-score between 91% and 96% for the different types of heart disease. These results are competitive and promising for heart disease dataset. We showed that ensemble-learning framework based on deep models could overcome the problem of classifying an unbalanced heart disease dataset. Our proposed framework can lead to highly accurate models that are adapted for clinical real data and diagnosis use.

Review rating prediction framework using deep learning

Nowadays Review websites, such as Amazon and Yelp, allow users to post online reviews for several products, services and businesses. Recently online reviews play a great role in influencing the shopping decisions made by consumers. These reviews provide consumers with information and experience about product quality. Online reviews commonly comprise of a free-text format and user star-level rating Out of five. People believe that reviews will do help to the rating predication based on the idea that high star rating may significantly be attach with really good reviews. However, user’s rating star-level information is not usually available on many online review’s websites. Due to, it’s not possible for a given user to rate every product. On the other hand, most online reviews are written in free-text format, and therefore difficult for computer system to understand and analyze it. Identifying ratings for online reviews lately become an important topic in machine learning. In this paper, we propose a review rating prediction framework using deep learning. The framework consists of two phases based on deep learning bidirectional gated recurrent unit Bi-GRU model architectures, the first phase used for polarity prediction and the second phase used to predict review rating from review text. Extensive experiments were conducted to evaluate the proposed framework on two dataset Amazon and yelp datasets which are real-world datasets. The experimental results demonstrated that the proposed framework can significantly enhance the rating prediction in term of precision, recall, F1-score and root mean square root RMSE compared with the baseline approaches on different datasets.

A Partially Amended Hybrid Bi-GRU—ARIMA Model (PAHM) for Predicting Solar Irradiance in Short and Very-Short Terms

Solar renewable energy (SRE) applications are substantial in eradicating the rising global energy shortages and reversing the approaching environmental apocalypse. Hence, effective solar irradiance forecasting models are crucial in utilizing SRE efficiently. This paper introduces a partially amended hybrid model (PAHM) by the implementation of a new algorithm. The algorithm innovatively utilizes bi-directional gated unit (Bi-GRU), autoregressive integrated moving average (ARIMA) and naive decomposition models to predict solar irradiance in 5-min and 60-min intervals. Meanwhile, the models’ generalizability strengths would be tested under an 11-fold cross-validation and are further classified according to their computational costs. The dataset consists of 32 months’ solar irradiance and weather conditions records. A fundamental result of this study was that the single models (Bi-GRU and ARIMA) outperformed the hybrid models (PAHM, classical hybrid model) in the 5-min predictions, negating the assumptions that hybrid models oust single models in every time interval. PAHM provided the highest accuracy level in the 60-min predictions and improved the accuracy levels of the classical hybrid model by 5%, on average. The single models were rigorous under the 11-fold cross-validation, performing well with different datasets; although the computational efficiency of the Bi-GRU model was, by far, the best among the models.

A hybrid VMD–BiGRU model for rubber futures time series forecasting

As one of the four major industrial raw materials in the world, natural rubber is closely related to the national economy and people’s livelihood. The analysis of natural rubber price and volatility can give hedging guidance to manufacturers and provide investors with uncertainty and risk information to reduce investment losses. To effectively analyses and forecast the natural rubber’s price and volatility, this paper constructed a hybrid model that integrated the bidirectional gated recurrent unit and variational mode decomposition for short-term prediction of the natural rubber futures on the Shanghai Futures Exchange. In data preprocessing period, time series is decomposed by variational mode decomposition to capture the tendency and mutability information. The bidirectional gated recurrent unit is introduced to return the one-day-ahead prediction of the closing price and 7-day volatility for the natural rubber futures. The experimental results demonstrated that: (a) variational mode decomposition is an effective method for time series analysis, which can capture the information closely related to the market fluctuations; (b) the bidirectional neural network structure can significantly improve the model performance both in terms of fitting performance and the trend prediction; (c) a correspondence was found between the predicted target, i.e., the price and volatility, and the intrinsic modes, which manifested as the impact of the long-term and short-term characteristics on the targets at different time-scales. With a change in the time scale of forecasting targets, it was found that there was some variation in matching degree between the forecasting target and the mode sub-sequences.