Gated Recurrent Unit Research Articles

Multimodal fusion: A study on speech-text emotion recognition with the integration of deep learning

Recognition of various human emotions holds significant value in numerous real-world scenarios. This paper focuses on the multimodal fusion of speech and text for emotion recognition. A 39-dimensional Mel-frequency cepstral coefficient (MFCC) was used as a feature for speech emotion. A 300-dimensional word vector obtained through the Glove algorithm was used as the feature for text emotion. The bidirectional gate recurrent unit (BiGRU) method in deep learning was added for extracting deep features. Subsequently, it was combined with the multi-head self-attention (MHA) mechanism and the improved sparrow search algorithm (ISSA) to obtain the ISSA-BiGRU-MHA method for emotion recognition. It was validated on the IEMOCAP and MELD datasets. It was found that MFCC and Glove word vectors exhibited superior recognition effects as features. Comparisons with the support vector machine and convolutional neural network methods revealed that the ISSA-BiGRU-MHA method demonstrated the highest weighted accuracy and unweighted accuracy. Multimodal fusion achieved weighted accuracies of 76.52 %, 71.84 %, 66.72 %, and 62.12 % on the IEMOCAP, MELD, MOSI, and MOSEI datasets, suggesting better performance than unimodal fusion. These results affirm the reliability of the multimodal fusion recognition method, showing its practical applicability.

Risk identification and prediction model for continuous‐lane‐change vehicles considering driving style

Lane change risk identification and prediction research is important for traffic safety, development of advanced driver assistance systems, traffic flow optimization. Among them, vehicle continuous‐lane‐change (CLC) risk identification and prediction is weak, but this element is important for vehicle decision-making and path planning in multi-lane, non-adjacent lane situations. To compensate for the shortcomings of risk identification based on both deterministic and probabilistic methods and address the issues of poor measurement accuracy and efficiency caused by manual tuning, this study established a risk identification and prediction model based on the Bayesian global optimization (BO) gated recurrent unit (GRU) neural network (BO-GRU) and instantiated the model using the trajectory data of 129 vehicles. Overall, the results showed that the accuracy of identifying CLC risks and driving-style was 99.33% and 97.44%, respectively, while the overall average accuracy of predicting potential CLC risks based on the identification of risks and predicted trajectories was 97.13%. The root-mean-square errors of CLC predicted trajectory for different driving-styles were 0.2503, 0.2331, 0.2093, and 0.2706, respectively, which were 84%−88% lower compared to 1.7361 when driving-styles were not considered. The performance of the BO-GRU model is comparable to that of parameter-optimized LSTM and its variant models, which highlights the significance of parameter optimization for neural network models. Overall, the use of a BO-GRU-based model for identifying and predicting continuous lane change risks in vehicles can aid in making driving decisions and planning routes, decrease the risk of collisions, enhance traffic flow efficiency, and encourage safe driving behavior.

Stock price crash risk prediction based on high-low frequency dual-layer graph attention network

The phenomenon of a stock price crash involves a rapid, significant decrease in stock prices, severely impacting the market, investors, and the economy. This study introduces the BiGAT-GRU model, which combines Graph Attention Networks (GAT) and Gated Recurrent Units (GRU) to predict stock price crash risk by analyzing multi-scale investor sentiment propagation using data from Baidu search index and public opinion texts. The model demonstrates superior performance in predicting crash risk, providing valuable insights for policymakers and investors.

The forecasting of surface displacement for tunnel slopes utilizing the WD-IPSO-GRU model

To quickly assess slope stability based on field displacement monitoring data, this paper constructs a hybrid optimization model that predicts surface displacement during tunnel excavation in base-overburden slopes. The model combines Wavelet Decomposition (WD) with a Gated Recurrent Unit (GRU), and the GRU's hyperparameters are optimized using an Improved Particle Swarm Optimization algorithm (IPSO). The specific steps are as follows: First, the Wavelet Decomposition (WD) technique is applied to decompose the raw displacement data, extracting features at different time–frequency scales. Next, the Dropout technique is incorporated into the GRU model to prevent overfitting. Additionally, nonlinear inertia weight ω improved cognitive factor c1, and social factor c2 are introduced. The PSO algorithm is improved by integrating crossover and mutation concepts from genetic algorithms. Finally, the IPSO is used to optimize the number of neural units hN, HN, LN and dropout rates D1 and D2 in the GRU network architecture. After constructing the WD-IPSO-GRU model, a comprehensive comparison is made with various swarm intelligence algorithms and state-of-the-art models. The experimental results demonstrate that the WD-IPSO-GRU model significantly improves the prediction accuracy of surface displacement in slopes during tunnel excavation. Compared to directly using raw data for prediction, the introduction of the WD preprocessing technique improved the prediction accuracy at measurement points 01 and 02 by 28% and 45.9%, respectively. Additionally, with the model optimized by IPSO, the prediction accuracy at measurement points 01 and 02 increased by 76% and 56.7%, respectively. The WD-IPSO-GRU model effectively addresses the challenges of extracting features from univariate displacement time-series data and determining the parameters of the GRU network. It improves the prediction accuracy of surface displacement in base-overburden type slopes and demonstrates excellent generalization ability and reliability. The research results validate the potential application of the model in geotechnical engineering and provide strong support for assessing slope stability during tunnel excavation.

Comparative Analysis of Machine Learning Models for Detecting Abnormal Driving Behaviour in Two-Wheeler Kinematics

Road accidents are a major global concern, particularly affecting vulnerable two-wheeler riders. This research investigates driving behaviour using time-series data from real-world scenarios to de- velop machine learning models for anomaly detection and risk as- sessment. The purpose of this study is to explore the performance of five predictive models—Long Short-Term Memory-Residual (LSTM- R), Gated Recurrent Unit (GRU), Adaboost, XGBoost, and a Mul- tilayer Perceptron (MLP) with Random Forest (RF) ensemble—are evaluated based on nine parameters: accelerometer (X, Y, Z), gyro rotation (X, Y, Z), motion yaw, pitch, and roll. A threshold rule-based approach using multi-output regression is employed for predictive modeling. The study makes three key contributions: First, it assesses the predictive accuracy of the five models, with the MLP-RF ensem- ble achieving a low MAE of 0.1157 on test data. Second, it identi- fies optimal hyperparameters for each model through systematic tuning. Third, it introduces a threshold-based anomaly detection method using residuals, which reduces overfitting and requires fewer instances of abnormal behaviour. The results indicate that these models, especially the MLP-RF ensemble and GRU, effectively predict and detect abnormal driving behaviours, enhancing safety for two-wheeler riders and contributing to sustainable urban devel- opment.

An evolutionary deep learning model based on XGBoost feature selection and Gaussian data augmentation for AQI prediction

Accurate prediction of air quality is crucial for ensuring the scientific validity and effectiveness of air pollution control measures. This study proposes a combined deep learning (DL) model (XGBoost-GDA-TCN-IMRFO-GRU) for predicting hourly air quality index (AQI) data in four cities. The model integrates Extreme gradient boosting (XGBoost) for feature selection, Gaussian data augmentation (GDA), improved manta ray foraging optimization (IMRFO) algorithm, temporal convolutional network (TCN), and gated recurrent unit (GRU). XGBoost calculates the scores of pollutant gases affecting AQI, selecting the top four important pollutants (PM2.5, PM10, NO2, O3) based on their importance rankings. GDA enhances the robustness of the DL models and addresses the limitations of insufficient and overfitting training datasets. Additionally, the IMRFO algorithm, with two improved strategies, is applied to enhance the GRU model. TCN extracts spatiotemporal features of AQI, while GRU constructs a temporal model for efficient computations. Compared to eleven benchmark models, the proposed model demonstrates superior performance in terms of MAE, RMSE, MAPE, and NSE, achieving high accuracy and optimal prediction performance. Specifically, the XGBoost-GDA-TCN-IMRFO-GRU model reduces RMSE, MAE, and MAPE by 33–60 %, 39–68 %, and 39–66 %, respectively, compared to the TCN model. Therefore, the XGBoost-GDA-TCN-IMRFO-GRU model can provide reliable early warnings for air quality, which is of great significance for air pollution prevention and the sustainable development of society.

A Robust Recurrent Neural Networks-Based Surrogate Model for Thermal History and Melt Pool Characteristics in Directed Energy Deposition.

In directed energy deposition (DED), accurately controlling and predicting melt pool characteristics is essential for ensuring desired material qualities and geometric accuracies. This paper introduces a robust surrogate model based on recurrent neural network (RNN) architectures-Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), and Gated Recurrent Unit (GRU). Leveraging a time series dataset from multi-physics simulations and a three-factor, three-level experimental design, the model accurately predicts melt pool peak temperatures, lengths, widths, and depths under varying conditions. RNN algorithms, particularly Bi-LSTM, demonstrate high predictive accuracy, with an R-square of 0.983 for melt pool peak temperatures. For melt pool geometry, the GRU-based model excels, achieving R-square values above 0.88 and reducing computation time by at least 29%, showcasing its accuracy and efficiency. The RNN-based surrogate model built in this research enhances understanding of melt pool dynamics and supports precise DED system setups.

Efficiency of Artificial Intelligence Methods for Hearing Loss Type Classification: an Evaluation

The evaluation of hearing loss is primarily conducted by pure tone audiometry testing, which is often regarded as golden standard for assessing auditory function. If the presence of hearing loss is determined, it is possible to differentiate between three types of hearing loss: sensorineural, conductive, and mixed. This study presents a comprehensive comparison of a variety of AI classification models, performed on 4007 pure tone audiometry samples that have been labeled by professional audiologists in order to develop an automatic classifier of hearing loss type. The tested models include Logistic Regression, Support Vector Machines, Stochastic Gradient Descent, Decision Trees, Random Forest, Feedforward Neural Network (FNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The presented work also investigates the influence of training dataset augmentation with the use of a Conditional Generative Adversarial Network on the performance of machine learning algorithms and examines the impact of various standardization procedures on the effectiveness of deep learning architectures. Overall, the highest classification performance, was achieved by LSTM with an out-of-training accuracy of 97.56%.

Industrial water withdrawal prediction using multi-head attention encoder model

ABSTRACT Industrial water withdrawal prediction is the cornerstone of water resources monitoring and early warning, as well as a key task in implementing rigid constraints on water resources. Although many factors may influence water withdrawal, they are difficult to obtain. Extracting the features in the historical water withdrawal data itself is a more convenient alternative way to achieve accurate prediction. However, due to the lack of significant regularity and periodicity in industrial water data, feature extraction is challenging. In this paper, a multi-head attention encoder (MAEN) model consisting of multi-head attention mechanisms and feed-forward neural networks is proposed to enable more convenient and accurate forecasting. The multi-head attention mechanism allows to focus on different parts of the data simultaneously, detecting complex dependencies, and extracting key features from historical data to enable more efficient feature extraction. Applied to water withdrawal data from real factories, the proposed model outperforms common time series prediction models including artificial neural network, long short-term memory, and gate recurrent unit. Compared to the best-performing comparison model, MAEN shows a reduction of mean squared error by 3.2% for 1 day ahead predictions, and 8.9% for 7 day ahead predictions, illustrating its superior performance in industrial water withdrawal prediction.

Stock Price Forecasting using N-Beats Deep Learning Architecture

Stock prices present unique forecasting challenges due to factors such as market volatility, investor sentiment, and economic indicators, which contribute to significant fluctuations in time series data. This paper addresses these complexities by applying Deep Learning (DL) models to predict stock prices, with a particular focus on the S&P 500 index. Although DL models have shown remarkable success in fields like image processing and natural language processing, they require specialized architectures to effectively handle time series forecasting. This study examines the Neural Basis Expansion Analysis for Interpretable Time Series Forecasting (N-BEATS) model, a novel DL architecture specifically tailored for time series data, using S&P 500 stock price data. The performance of N-BEATS is benchmarked against three baseline models: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRU). The evaluation metrics include Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). Results indicate that the N-BEATS model consistently surpasses the other models in all metrics. Additionally, the Diebold-Mariano (DM) test further validates the superior predictive accuracy of the N-BEATS model compared to the alternatives. This research underscores the potential of the N-BEATS model to significantly improve stock price forecasting, offering valuable insights for investors, financial analysts, and other market participants.

Using acoustic emission technique for structural health monitoring of laminate composite: A novel CNN-LSTM framework

This research has developed a real-time end-to-end deep learning model for structural health monitoring (SHM) method for composite impact damage diagnosis based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM). The acoustic emission (AE) signals collected under low-velocity impacts by means of piezoelectric sensors on composite materials are used for training deep learning networks. Based on the impact load curves, specimens are categorized into minor failure, intermediate failure, and severe failure. The convolved signals are segmented and reconstructed at a given length for the following LSTM module. The average accuracies for basic CNN, CNN– Recurrent Neural Network (RNN), CNN-LSTM, and CNN– Gated Recurrent Unit (GRU) are respectively 88.7 %, 92.6 %, 98 %, and 95.4 %. A sensitivity analysis on sub-signal length was conducted on the CNN-LSTM model, revealing that the model achieved its best performance when the sub-signal length was set at 16. The model attained prediction accuracies of 97.4 %, 100 %, and 100 %, respectively, for minor failure, intermediate failure, and severe failure cases.

Utilizing TGAN and ConSinGAN for Improved Tool Wear Prediction: A Comparative Study with ED-LSTM, GRU, and CNN Models

Developing precise deep learning (DL) models for predicting tool wear is challenging, particularly due to the scarcity of experimental data. To address this issue, this paper introduces an innovative approach that leverages the capabilities of tabular generative adversarial networks (TGAN) and conditional single image GAN (ConSinGAN). These models are employed to generate synthetic data, thereby enriching the dataset and enhancing the robustness of the predictive models. The efficacy of this methodology was rigorously evaluated using publicly available milling datasets. The pre-processing of acoustic emission data involved the application of the Walsh-Hadamard transform, followed by the generation of spectrograms. These spectrograms were then used to extract statistical attributes, forming a comprehensive feature vector for model input. Three DL models—encoder-decoder long short-term memory (ED-LSTM), gated recurrent unit (GRU), and convolutional neural network (CNN)—were applied to assess their tool wear prediction capabilities. The application of 10-fold cross-validation across these models yielded exceptionally low RMSE and MAE values of 0.02 and 0.16, respectively, underscoring the effectiveness of this approach. The results not only highlight the potential of TGAN and ConSinGAN in mitigating data scarcity but also demonstrate significant improvements in the accuracy of tool wear predictions, paving the way for more reliable and precise predictive maintenance in manufacturing processes.

Stock Market Prediction in Nepali Stock Market using Machine Learning Model

The stock market, a key driver of the global economy, presents a formidable challenge for accurate prediction due to its intricate, chaotic, and dynamic nature. This study explores machine learning approaches, specifically comparing three prediction models—Back Propagation Neural Network (BPNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). The analysis incorporates historical stock trading data, including open, high, low, and close prices, and technical analysis indicators such as moving average, Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), and Commodity Channel Index (CCI) so on. The evaluation focuses on the performance of these models in predicting future trends in stock prices, utilizing data from the Nepal Stock market spanning from 2013 to 2023. The selected stocks for analysis include two stock NIC Asia Bank Limited and, NMB Bank Limited, two index NEPSE and Hydropower index. Through rigorous assessment, it is determined that LSTM demonstrates superior overall performance compared to GRU and BPNN. This research contributes valuable insights into the effectiveness of machine learning models in forecasting stock market trends, with implications for investors and financial practitioners.

State of charge estimation for the lithium-ion battery based on fractional-order multi-dimensional Taylor network

To accurately estimate the state of charge (SOC) of the lithium-ion battery (LiB), a fractional-order multi-dimensional Taylor network (FMTN) model was proposed in the study. With two open datasets of LiBs, the performance of the FMTN model on the SOC estimation is evaluated and compared with multi-dimensional Taylor network (MTN), back propagation neural network (BPNN), and dual-stage attention gate recurrent unit neural network (DA-GRUNN) models. With the datasets of B0005 and B0006 LiBs, the SOC estimation accuracy of the FMTN-6 model is 34 %, 32 %, and 37 %, 45 % higher than that of the MTN-6 and BPNN-3 models, respectively. The estimation time consumptions of the FMTN-6 model are increased by 2.8 % and 3.9 % and reduced by 49.5 % and 52.8 % compared with that of the MTN-6 and BPNN-3 models, respectively. With the open dataset of real driving conditions, the root-mean-square error of the SOC estimation of the FMTN model is reduced by 48 % compared with that of the MTN model. Besides, with the datasets at 25 °C and 45 °C, the SOC estimation accuracy of the FMTN model is improved by 17 % and 18 % compared with that of the DA-GRUNN model, respectively.

Performance Analysis of Deep Learning based Signal Constellation Identification Algorithms for Underwater Acoustic Communications

This research delves into the evaluation of Deep learning signal constellation identification (DL-SCI) algorithms in underwater acoustic communications using Orthogonal Frequency Division Multiplexing (OFDM). It distinctly examines at how effective the recurrent neural networks (RNNs), particularly, Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) algorithms in predicting the signal constellation when applied to different underwater acoustic channels characteristics. Unlike manual feature selection in machine learning (ML), in this paper, DL-SCI exploits the labelled OFDM signals at the transmitter to detect and decode them at the receiver. In order to measure their effectiveness performance metrics, Bit Error Rate (BER) and parameters derived from the confusion matrix such as accuracy and precision are used. The study highlights the importance of utilizing zero cyclic prefix techniques which can exploit the inherent bandwidth limitation effectively. Furthermore, when examining complexity, it is observed that both GRU and LSTM algorithms require less floating-point operations (FLOPS) compared to traditional methods such as Minimum Mean Square Error (MMSE) and Least Squares (LS). Interestingly GRU shows performance in terms of complexity when compared to LSTM. Moreover, GRU outperforms LSTM by achieving a 4 dB improvement for long subcarriers. These results emphasize the effectiveness of learning techniques in enhancing performance and efficiency in acoustic communications.

Shield machine pose prediction based on CNN-GRU-Attention

This paper presents a shield machine pose prediction method based on Convolutional Neural Network (CNN) - Gated Recurrent Unit (GRU) with Attention mechanism (Attention). Firstly, the Pearson correlation coefficient is employed to select input parameters highly related to the position and posture of the shield machine. Then, a convolutional neural network is introduced to extract the long-term short-term feature dependency features in the operation data of the shield machine, optimizing the model’s input. The attention mechanism is integrated into the gated loop unit to make the model more targeted in using key information in the input sequence and improve the accuracy of the shield machine pose prediction model. The effectiveness of this method is verified by the example of Beijing Metro Line 10. Compared with GRU-Attention and LSTM-Attention models, the mean value of determination coefficient R2 increased from 0.872 and 0.886 to 0.959, and the mean value of root mean square error RMSE decreased from 2.78 and 2.52 to 2.14. This method can provide effective prediction for the attitude and position of shield machines in actual tunnel engineering.

Life prediction method of battery energy storage system in frequency modulation application

Abstract To tackle the challenge of lifespan reduction in lithium batteries during frequency modulation, this study introduces a novel Remaining Useful Life (RUL) prediction methodology. The proposed approach integrates Variational Mode Decomposition (VMD) with Gated Recurrent Unit (GRU) networks, thereby efficiently synthesizing data from both operational parameters and capacity metrics. Firstly, VMD is utilized to decompose the initial capacity data of lithium batteries into separate components characterized by high and low frequencies. Subsequently, for the high-frequency elements, GRU is employed for rolling iterative prediction, while for the low-frequency elements, features are extracted from the operational data and inputted into GRU for prediction. Finally, the component prediction results are restored to obtain capacity prediction values. Validation conducted using the NASA dataset shows that the root mean square error of capacity prediction is minimized to 0.0122, with a corresponding minimum average absolute error of 0.0096, and RUL prediction errors are generally within 2 cycles.

Optimized feature selection approaches for accident classification to enhance road safety

In the modern era, the issue of road accidents has become an increasingly critical global concern, requiring urgent attention and innovative solutions. This investigation has compiled an extensive dataset of 10,356 accident occurrences that occurred between the years 2018 and 2022 in Ernakulam district. By utilizing advanced feature selection methodologies, such as genetic algorithm and coyote optimization, this research has identified pivotal accident determinants. The study harnesses the potential of deep learning techniques, encompassing recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), and multilayer perceptron (MLP) for classifying accidents according to severity (categorized as fatal, grievous, and severe). Eight predictive models are trained using the dataset, and the top two are ensembled. Integrating deep learning and optimization strategies, this research aims to create a robust accident classification system. The system will help in developing proactive policies that can reduce the frequency and severity of accidents in Ernakulam district. <div style="position: absolute; display: none; z-index: 16777271;"> </div>

Defect Recognition and Classification Using Ultrasonic Scattered Signals Based on Convolutional Neural Network

Abstract Carbon Fiber Reinforced Polymer (CFRP) is widely used in the aerospace, aircraft, and railway industries because of its remarkable advantages, such as its lightweight and high strength. Detecting and identifying defects in CFRP is crucial for ensuring its secure and efficient use. Ultrasonic non-destructive testing is one of the most practical and effective methods for defect detection and identification. The ultrasonic scattered signals of defects, which contain abundant defect feature information, are advantageous for defect recognition and classification. However, the anisotropy and inhomogeneity of CFRP, which lead to a low signal-to-noise ratio and poor image quality, pose challenges in flaw recognition. To address the issue, this paper proposes using Convolutional Neural Networks (CNN) methods for defect classification, which can adequately extract information from ultrasonic scattered signals directly. The scattering signals from various types of defects in CFRP, including the different sizes of holes and folds, were selected as inputs for CNN models. Different network models combining Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) with 1D-CNN were employed to classify and identify defect signals. To generate sufficient input sample signals, the data sets were further enlarged by incorporating noise into the original signals. The experimental results showed that using the 1D-LSTM-CNN and 1D-GRU-CNN network models can reduce computational resources while ensuring the accuracy of defect recognition.

Acapella-based music generation with sequential models utilizing discrete cosine transform

Making musical instruments that accompany vocals in a song depends on the mood quality and the music composer’s creativity. The model created by other researchers has restrictions that include being limited to musical instrument digital interface files and relying on recurrent neural networks (RNN) or Transformers for the recursive generation of musical notes. This research offers the world’s first model capable of automatically generating musical instruments accompanying human vocal sounds. The model we created is divided into three types of sound input: short input, combed input, and frequency sound based on the discrete cosine transform (DCT). By combining the sequential models such as Autoencoder and gated recurrent unit (GRU) models, we will evaluate the performance of the resulting model in terms of loss and creativity. The best model has a performance evaluation that resulted in an average loss of 0.02993620155. The hearing test results from the sound output produced in the frequency range 0-1,600 Hertz can be heard clearly, and the tones are quite harmonious. The model has the potential to be further developed in future research in the field of sound processing.