Bidirectional Long Short-term Memory Network Research Articles

A novel hybrid framework for heterogeneous measurement data prediction of cable-stayed bridges using ACMD-PE-BiLSTM

Abstract Bridge heterogeneous measurement data prediction is a crucial aspect of structural analysis of bridge engineering. However, the nonlinearity and nonstationarity of the measurement data are common occurrences in daily monitoring operations, which affects the accuracy of the prediction of bridge health. In this context, this paper proposes a hybrid data driven deep learning approach for predicting bridge-multi-source heterogeneous data to tackle the challenges posed by the complexity and nonstationarity of cable-stayed bridge monitoring data and enhance the accuracy and efficiency of predicting measurement data. This approach leverages adaptive chirp mode decomposition (ACMD), permutation entropy (PE), and Bi-directional long short-term memory (BiLSTM). Firstly, the ACMD algorithm decomposes the bridge monitoring data into a discrete number of intrinsic mode functions (IMFs) to produce clearer signals. Then, the PE algorithm is applied to each IMF to optimize the number of IMFs and construct the new components. Finally, a BiLSTM network is present for each component to establish the prediction model, and the final prediction results are obtained by synthesizing the predictions. The effectiveness and feasibility of the proposed method are extensively evaluated using measureddisplacement, and acceleration data from a cable-stayed wind speed,bridge. Evaluation indicators are used to evaluate the performance, and a comparative analysis with other benchmark models is further conducted to systematically validate the reliability of the proposed approach. The proposed prediction method offers several advantages, with its stability and accuracy being particularly noteworthy. The results suggest that the proposed method is superior to all benchmark models regarding cable-stayed bridge heterogeneous measurement data and can provide reliable results for real-world bridge engineering.

Statistical and machine learning approaches for estimating pollution of fine particulate matter (PM2.5) in Vietnam

This study aims to predict fine particulate matter (PM2.5) pollution in Ho Chi Minh City, Vietnam, using autoregressive integrated moving average (ARIMA), linear regression (LR), random forest (RF), long short-term memory (LSTM), bidirectional LSTM (Bi-LSTM), and convolutional neural network (CNN) combining Bi-LSTM (CNN+Bi-LSTM). Two experiments were set up: the first one used data from 2018–2020 and 2021 as training and test data, respectively. Data from 2018–2021 and 2022 were used as training and test data for the second experiment, respectively. Consequently, ARIMA showed the worst performance, while CNN+Bi-LSTM achieved the best accuracy, with an R² of 0.70 and MAE, MSE, RMSE, and MAPE of 5.37, 65.4, 8.08 µg/m³, and 29%, respectively. Additionally, predicted air quality indexes (AQIs) of PM2.5 were matched the observed ones up to 96%, reflecting the application of predicted concentrations for AQI computation. Our study highlights the effectiveness of machine learning model in monitoring of air pollution.

Research on Climactic Chapter Recognition of a Chinese Long Novel Based on Plot Description

Many readers continue to pursue Chinese long novels in the past several decades because of diverse characters and fascinating plots. The climactic chapter is an important part of a Chinese long novel, where the key conflict develops to the extreme point. However, how to quickly and accurately recognize the climactic chapter remains a common problem for many readers in their reading choices. This paper conducts research on recognizing the climactic chapter of a Chinese long novel by accurately describing its plot. The proposed method consists of two parts; one is the extraction of key elements, such as viewpoint paragraphs, non-viewpoint paragraphs, chapter keywords, major characters etc. The other part is the climactic chapter recognition, which applies the Bidirectional Gate Recurrent Unit (BiGRU) model and the multi-head attention to recognize the climactic chapter, on the basis of the chapter plot description matrix. Comparative experiments on the corpus named The Condor Trilogy show that the proposed method in this paper has a better recognition performance compared with the existing models, such as Naive Bayesian (NB), Support Vector Machine (SVM), Roberta-large, and the Bidirectional Long-Short Term Memory (BiLSTM) network. Ablation experiments validated the effectiveness of primary components in the proposed method.

Advanced Monthly Rainfall Forecasting in Southern Lazio: Integrating Climatic Indices, Classical or Deep Neural Networks, and a Feature Selection Algorithm

Accurately predicting monthly precipitation is crucial for managing water resources, particularly in regions with complex climates like Southern Lazio, Italy. This study addresses the challenges of selecting relevant climate parameters and optimizing model complexity for precise forecasting. Four models, based on Bi-directional Long Short-Term Memory (BI-LSTM) and Radial Basis Function (RBF) neural networks, were compared, focusing on their ability to predict monthly precipitation using diverse inputs such as mean temperature, relative humidity, sea level pressure, radiation, precipitation change, and 12 climate indices. One significant challenge was determining the most impactful subset of input variables from a large dataset to enhance model performance without overfitting. To address this, the Particle Swarm Optimization (PSO) feature selection algorithm was employed, significantly improving model accuracy by identifying the optimal subset of input variables. The comparative analysis revealed that PSO-based models consistently outperformed those using all input variables, with the PSO-RBF model achieving a Kling-Gupta Efficiency (KGE) value of up to 0.80 during testing. Notably, both PSO-BI-LSTM and PSO-RBF models demonstrated strong performance in predicting precipitation peaks, achieving KGE values of 0.74. The study highlights the efficacy of the RBF model combined with PSO, which optimizes fewer parameters compared to the more complex BI-LSTM network while reducing input variables. This approach is not only less computationally intensive but also easier to implement, particularly in regions with limited instrumentation. These findings suggest that the PSO-RBF model is a reliable and efficient tool for monthly precipitation prediction in challenging environments.

Advanced hybrid neural network techniques for minimizing gas turbine emissions

Purpose Emissions have significant environmental impacts. Hence, minimizing emissions is essential. This study aims to use a hybrid neural network model to predict carbon monoxide (CO) and nitrogen oxide (NOx) emissions from gas turbines (GTs) to enhance emission prediction for GTs in predictive emissions monitoring systems (PEMS). Design/methodology/approach The hybrid model architecture combines convolutional neural networks (CNN) and bidirectional long-short-term memory (Bi-LSTM) networks called CNN-BiLSTM with modified extrinsic attention regression. Over five years, data from a GT power plant was uploaded to Google Colab, split into training and testing sets (80:20), and evaluated using test matrices. The model’s performance was benchmarked against state-of-the-art emissions prediction methodologies. Findings The model showed promising results for GT CO and NOx emissions. CO predictions had a slight underestimation bias of −0.01, with root mean-squared error (RMSE) of 0.064, mean absolute error (MAE) of 0.04 and R2 of 0.82. NOx predictions had an RMSE of 0.051, MAE of 0.036, R2 of 0.887 and a slight overestimation bias of +0.01. Research limitations/implications While the model demonstrates relative accuracy in CO emission predictions, there is potential for further improvement in future research. Practical implications Implementing the model in real-time PEMS and establishing a continuous feedback loop will ensure accuracy in real-world applications, enhance GT functioning and reduce emissions, fuel consumption and running costs. Social implications Accurate GT emissions predictions support stricter emission standards, promote sustainable development goals and ensure a healthier societal environment. Originality/value This paper presents a novel approach that integrates CNN and Bi-LSTM networks. It considers both spatial and temporal data to mitigate previous prediction shortcomings.

Fault diagnosis of floating offshore wind turbine based on bidirectional long short-term memory networks with sliding window

Owing to its ability to harvest more power in deeper waters than other types of offshore wind turbines, the floating offshore wind turbine (FOWT) has attracted significant interest from both academics and industry. However, deterioration in system performance and unscheduled shutdowns will significantly increase operation and maintenance costs. To avoid system damage and ensure operation safety, there is a great demand for developing FOWT fault diagnosis to detect and identify faults as early as feasible. Due to the enormous complexity of FOWT including dynamics and nonlinearity, effective FOWT fault diagnosis poses a significant challenge. This paper develops a novel FOWT fault diagnosis method using bidirectional long short-term memory (BiLSTM) with sliding window. Firstly, the dynamics are embedded by stacking time-dependent measurements using the sliding window technique. Then, the augmented data is fed into a BiLSTM network to exploit the nonlinearity of FOWT. Consequently, a softmax classifier is used for fault diagnosis. A high-fidelity 10 MW spar FOWT benchmark based on the NREL Fatigue, Aerodynamics, Structures, and Turbulence (FAST) model is used to validate the capability and effectiveness of the proposed method. Experimental results show that the proposed method outperforms other related methods in diagnosing critical faults in FOWTs under various operating conditions.

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Predicting temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by artificial neural networks

Predicting temporal evacuation travel time in staircases between adjacent floors is crucial for dynamic evacuation guidance to improve vertical evacuation efficiency during emergencies in super high-rise buildings. However, the prediction methods used in previous studies mainly focus on the total evacuation time in buildings and cannot predict the temporal evacuation travel time in every single part of the total evacuation route. This study proposes a novel method to predict temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by using artificial neural networks. Firstly, evacuation simulation data of a 71-storey office building are analyzed for feature engineering and dataset building. Secondly, three types of artificial neural networks, including convolutional neural networks, RNN-based model (the traditional recurrent neural networks, long-short term memory networks, and bidirectional long-short term memory networks), and Attention-based model (Attention model and Transformer model) are extended to predict the temporal evacuation travel time in staircases between adjacent floors based on the simulation data. Finally, the prediction results by different artificial neural networks are compared. The results show that the artificial neural networks can provide accurate prediction values for the temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings.

Solar irradiation forecast enhancement using clustering based CNN-BiLSTM-attention hybrid architecture with PSO

Accurate solar irradiation forecasting is essential for optimising solar energy use. This paper presents a novel forecasting approach: the ‘Clustering-based CNN-BiLSTM-Attention Hybrid Architecture with PSO’. It combines clustering, attention mechanisms, Convolutional Neural Networks (CNN), Bidirectional Long-Short Term Memory (BiLSTM) networks, and Particle Swarm Optimisation (PSO) into a unified framework. Clustering categorises days into groups, improving predictive capabilities. The CNN-BiLSTM model captures spatial and temporal features, identifying complex patterns. PSO optimises the hybrid model’s hyperparameters, while an attention mechanism assigns probability weights to relevant information, enhancing performance. By leveraging spatial and temporal patterns in solar data, the proposed model improves forecasting accuracy in univariate and multivariate analyses with multi-step predictions. Extensive tests on real-world datasets from various locations show the model’s effectiveness. For example, with NASA power data, the model achieves a Mean Absolute Error (MAE) of 24.028 W/m2, Root Mean Square Error (RMSE) of 43.025 W/m2, and an R 2 score of 0.984 for 1-hour ahead forecasting. The results show significant improvements over conventional methods.

Transformer assisted by DSC and BiLSTM for bearing fault pattern recognition under strong noise interference

ABSTRACT Deep learning has greatly promoted intelligent fault diagnosis for bearings and has led to the development of many intelligent fault recognition methods based on deep neural networks. However, bearing fault signals in industrial applications are susceptible to noise interference, and intelligent recognition methods for bearing faults under strong noise interference are lacking. An intelligent fault pattern recognition model termed IFPR-MDL, which is based on a Transformer assisted by depthwise separable convolutional (DSC) and bidirectional long short-term memory (BiLSTM) networks, is designed to identify bearing faults under strong noise conditions. In this model, multiscale features are first extracted by a multiscale feature extraction module designed on the basis of DSC; then, contextual information is further extracted by a BiLSTM, and information from different scales is integrated by Transformer encoders. The DSC network facilitates model parameter reduction, the BiLSTM network has expertise in mining temporal features, and the Transformer network excels at capturing long-range feature correlations. This model fully utilises the advantages of these models and complements them to improve the anti-interference ability of bearing fault pattern recognition. Tests on several public datasets show that the proposed model has higher recognition accuracy, stronger generalisation ability, and noise resistance in strong noise situations.

Research on Gold Price Prediction Based on LSTM Modeling

Gold plays a pivotal role in asset allocation, and the construction of gold price prediction models represents a complex yet rewarding task within the field of finance. The problem of international gold price is addressed in this paper forecasting by proposing a standard Long-Short Term Memory (LSTM) model and introducing bi-directional LSTM (Bi-LSTM) networks and multivariate analysis to compare the forecasting accuracy of the relevant models. This comparison is based on the daily gold price of the London Bullion Market Association (LBMA) for 2013-2022. This methodology takes into account various factors that influence the gold market and incorporates them as input sources into the Multi-factor LSTM model, thereby enhancing the interpretability of the LSTM model. Correlation analysis and the Granger test are employed to analyze these influencing factors. However, the results of this study suggest that while the Multi-factor LSTM model may be more accurate in predicting outliers, it tends to increase the number of small errors. On the other hand, the inclusion of Bi-LSTM networks can improve the overall accuracy of gold price prediction.

An Analytical Approach for IGBT Life Prediction Using Successive Variational Mode Decomposition and Bidirectional Long Short-Term Memory Networks

The precise estimation of the operational lifespan of insulated gate bipolar transistors (IGBT) holds paramount significance for ensuring the efficient and uncompromised safety of industrial equipment. However, numerous methodologies and models currently employed for this purpose often fall short of delivering highly accurate predictions. The analytical approach that combines the Pattern Optimization Algorithm (POA) with Successive Variational Mode Decomposition (SVMD) and Bidirectional Long Short-term Memory (BiLSTM) network is introduced. Firstly, SVMD is employed as an unsupervised feature learning method to partition the data into intrinsic modal functions (IMFs), which are used to eliminate noise and preserve the essential signal. Secondly, the BiLSTM network is integrated for supervised learning purposes, enabling the prediction of the decomposed sequence. Additionally, the hyperparameters of BiLSTM and the penalty coefficients of SVMD are optimized utilizing the POA technique. Subsequently, the various modal functions are predicted utilizing the trained prediction model, and the individual mode predictions are subsequently aggregated to yield the model’s definitive final life prediction. Through case studies involving IGBT aging datasets, the optimal prediction model was formulated and its lifespan prediction capability was validated. The superiority of the proposed method is demonstrated by comparing it with benchmark models and other state-of-the-art methods.

Memristor-based circuit design of BiLSTM network

The bidirectional long short-term memory (BiLSTM) network involves significant amount of parameter computations. This paper proposes the memristor-based bidirectional long short-term memory (MBiLSTM) network, with its capability of in-memory computing and parallel computing, can accelerates the parameter computations speed. The MBiLSTM network circuit is composed of normalization circuit, two memristor-based long short-term memory (LSTM) circuits, memristor-based resnet circuit, memristor-based dense circuitand winner-take-all (WTA) circuit. The voltage signals are scaled to the setting range by normalization circuit, memristor-based LSTM circuit is responsible for extracting features from the dataset, memristor-based resnet circuit can enhance the overall performance of the network, memristor-based dense circuit ensures that the final outputs dimension matches the dimension of the target signals, WTA circuit outputs the maximum voltage of memristor-based dense circuit. The effectiveness of the MBiLSTM network is validated through gait recognition experiment and handwritten digit recognition experiment. The stability, robustness and potential errors in the manufacturing process of memristance are analyzed.

Classification of motor imagery EEG signals using wavelet scattering transform and Bi-directional long short-term memory networks

A brain-computer interface (BCI) is a technology that creates a communication path between the brain and external devices. Raw EEG data in BCI contain a large amount of complex information, but only some of it needs to be focused on in research. So Feature extraction and classification play an important role in BCI by reducing the data dimensionality and improving the accuracy of subsequent classification. Wavelet scattering transform is an emerging feature extraction method that generates time-shift invariant representations of EEG signals. We applied the wavelet scattering transform to extract features from motor imagery EEG signals, and utilized these features for classification purposes. To achieve this, we proposed a new method that combines wavelet scattering transform with a bidirectional long short-term memory (BiLSTM) network in a fusion deep learning network. Wavelet scattering transform can deeply mine the feature information in EEG signals. In the classification stage, multiple time window features obtained in the scattering transform are sent to the BiLSTM network for classification. The final result will be determined by a vote. In addition, for the processing of raw EEG data, we proposed a time-step based time window strategy that can better utilize the small dataset. This operation can obtain EEG data of multiple time steps. The proposed method was validated using BCI competition II dataset III and BCI competition IV dataset 2b. The results show that the proposed method in this paper can effectively improve the accuracy of motor imagery EEG and provide a new idea for the feature extraction and classification research of motor imagery brain-computer interface.

Enhancing temperature and torque prediction in permanent magnet synchronous motors using deep learning neural networks and BiLSTM RNNs

This study aims to develop an effective method for predicting the temperature and torque of Permanent Magnet Synchronous Motors (PMSMs) using deep learning techniques, which is crucial for optimizing motor performance and ensuring longevity, particularly in the automotive industry. Various Neural Network (NN) architectures, including a Recurrent Neural Network (RNN) with a Bidirectional Long Short-Term Memory (BiLSTM) unit, were employed to model the complex relationships between motor parameters, such as stator winding, current, torque, and permanent magnet temperature. The findings demonstrate that an NN with two hidden layers (64 and 32 neurons) achieved an R2 score of 0.99 for both torque and temperature prediction, while the BiLSTM network effectively modeled temporal dynamics, leading to high-fidelity rotor temperature predictions. This research provides a novel application of BiLSTM RNNs in accurately predicting PMSM temperatures, offering valuable insights for industries reliant on these motors. Integrating these models into motor control systems can enhance operational efficiency, reduce overheating risks, and extend motor lifespan, contributing to energy savings and environmental sustainability by lowering energy consumption and reducing waste.

Enhanced Foreign Exchange Volatility Forecasting Using Ceemdan with Optuna-Optimized Ensemble Deep Learning Model

Foreign Exchange (FX) is the largest financial market in the world, with a daily trading volume that significantly exceeds that of stock and futures markets. The prediction of FX volatility is a critical financial concern that has garnered significant attention from researchers and practitioners due to its far-reaching implications in the financial markets. This paper presents a novel hybrid ensemble forecasting model integrating a decomposition strategy and three deep learning (DL) models: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and Convolutional Neural Network (CNN). This combination addresses individual models' limitations and further improves the accuracy and stability of FX volatility forecasting. The proposed approach utilizes the CEEMDAN technique to decompose volatility into multiple distinct intrinsic mode functions (IMFs) and merges these IMFs with GARCH and EGARCH volatilities to form the input dataset for the DL models. In addition, we employed an attention mechanism to improve the effectiveness of the DL techniques. Furthermore, the hyperparameters for the DL models are optimized using the Optuna algorithm. Finally, a hybrid ensemble model for forecasting exchange rate volatility is developed by combining the predictions of three distinct DL models. The proposed approach is evaluated against various benchmark models using evaluation measures such as MSE, MAE, HMSE, HMAE, RMSE, Q-LIKE, and the model confidence set (MCS) approach. The results demonstrate that our proposed approach provides accurate and reliable forecasts of FX volatility under different forecasting regimes, making it a valuable tool for financial practitioners and researchers.

A novel bidirectional LSTM network model for very high cycle random fatigue performance of CFRP composite thin plates

This study proposes a novel Double-layer Bidirectional Long Short-Term Memory (BiLSTM) neural network model, shorted as TDA-BiLSTM, which integrates Transfer learning and Attention mechanisms. The model aims to predict the fatigue life of carbon fiber reinforced polymer (CFRP) thin plate structures subjected to very high cycle random vibration fatigue loads. Distinct from conventional servo-hydraulic and ultrasonic fatigue testing methods, this research pioneers the use of a vibration table for very high cycle fatigue (VHCF) testing to fill the gap in the field. By training and validating data across various life ranges, the TDA-BiLSTM model demonstrates significant advantages in training speed and predicting accuracy. Its bidirectional structure and attention mechanism effectively capture complex patterns and long-term dependencies in sequence data. Experimental results indicate that the TDA-BiLSTM model achieves significantly lower Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) compared to Long Short-Term Memory (LSTM) and Long Short-Term Memory with Transfer learning (Tr-LSTM) models across different life ranges on the ε-N curve, indicating higher accuracy and stability in strain life prediction tasks. Additionally, an analysis of typical damage areas using Scanning Electron Microscopy (SEM) reveals the failure mechanisms of CFRP plates under very high cycle vibration fatigue loads.

Comparison of Cobb-Douglas Production Function with Deep Learning Models to Estimate the Productivity of Organised Manufacturing Sector - Indian Perspective

The production function defines the output of a firm, industry, or economy based on various combinations of inputs. This study focuses on estimating the production function of India's manufacturing sector and determining the relationship between its inputs and outputs. To predict the output, measured as Gross Value Added (GVA), the study applies the Cobb Douglas Production Function (CDPF) along with several deep learning techniques, including Feedforward, Recurrent, Long Short-Term Memory (LSTM), and Bidirectional LSTM networks. Model performance is evaluated using metrics like Mean Square Error (MSE), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). The results indicate that the Cobb Douglas Production Function outperforms the deep learning models in predicting GVA for the organized manufacturing sector in India.

Nonlinear Ensemble Deep Learning Model for Energy Consumption Prediction with Bayesian Optimization

Accurate prediction of electric energy consumption is crucial for efficient load dispatching, energy utilization, and grid operation. Traditional statistical and classical machine learning methods struggle with the nonlinear nature of energy consumption data, often leading to higher prediction errors. Additionally, deep learning models using a single approach face challenges such as convergence to local minima and poor generalization. This paper proposes a nonlinear ensemble deep learning model for residential energy consumption prediction, incorporating Bayesian optimization for hyperparameter tuning. The model combines Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and 1D Convolutional Neural Networks (1D-CNN), leveraging their powerful nonlinear feature learning capabilities. A k-means clustering approach is used to preprocess and reduce variability in the data, enhancing the ensemble model's performance. The ensemble model was tested on real energy consumption data from two districts in Addis Ababa, showing significant improvements in prediction accuracy with lower MAE, RMSE, and MAPE values compared to single models and un-clustered data. The integration of clustering and Bayesian optimization further enhanced model generalizability and minimized overfitting, demonstrating the effectiveness of a nonlinear approach in capturing complex energy consumption patterns.

Short-term power load forecasting based on hybrid feature extraction and parallel BiLSTM network

Electricity is a vital resource for societal and economic activities. Accurate electricity load forecasting can effectively reduce costs and enhance energy efficiency. Nevertheless, current research has limitations in effectively extracting and utilizing load forecasting information. This encompasses both undiscovered latent features and models that have not effectively captured long-term dependencies. To address these issues, this paper proposes a novel network architecture, which utilizes a hybrid feature extraction strategy and integrates machine learning methods with deep learning techniques. This method employs a hybrid feature extraction strategy composed of gradient-boosted regression trees, Fisher Score, and mutual information to achieve comprehensive feature representation. The network architecture utilizes a multi-head convolutional neural network and a bi-directional long short-term memory (BiLSTM) with lag parameters. This architecture can simultaneously learn multiple levels and aspects of features, further enhancing the BiLSTM model’s ability to capture long-term dependencies. Through multiple experiments conducted on datasets from Maine, Singapore, New South Wales, Australia, and European countries, our method outperformed the comparison models in three out of the five datasets.