Bidirectional Long Short-term Memory Network Research Articles

Advanced hybrid neural network techniques for minimizing gas turbine emissions

Purpose Emissions have significant environmental impacts. Hence, minimizing emissions is essential. This study aims to use a hybrid neural network model to predict carbon monoxide (CO) and nitrogen oxide (NOx) emissions from gas turbines (GTs) to enhance emission prediction for GTs in predictive emissions monitoring systems (PEMS). Design/methodology/approach The hybrid model architecture combines convolutional neural networks (CNN) and bidirectional long-short-term memory (Bi-LSTM) networks called CNN-BiLSTM with modified extrinsic attention regression. Over five years, data from a GT power plant was uploaded to Google Colab, split into training and testing sets (80:20), and evaluated using test matrices. The model’s performance was benchmarked against state-of-the-art emissions prediction methodologies. Findings The model showed promising results for GT CO and NOx emissions. CO predictions had a slight underestimation bias of −0.01, with root mean-squared error (RMSE) of 0.064, mean absolute error (MAE) of 0.04 and R2 of 0.82. NOx predictions had an RMSE of 0.051, MAE of 0.036, R2 of 0.887 and a slight overestimation bias of +0.01. Research limitations/implications While the model demonstrates relative accuracy in CO emission predictions, there is potential for further improvement in future research. Practical implications Implementing the model in real-time PEMS and establishing a continuous feedback loop will ensure accuracy in real-world applications, enhance GT functioning and reduce emissions, fuel consumption and running costs. Social implications Accurate GT emissions predictions support stricter emission standards, promote sustainable development goals and ensure a healthier societal environment. Originality/value This paper presents a novel approach that integrates CNN and Bi-LSTM networks. It considers both spatial and temporal data to mitigate previous prediction shortcomings.

Fault diagnosis of floating offshore wind turbine based on bidirectional long short-term memory networks with sliding window

Owing to its ability to harvest more power in deeper waters than other types of offshore wind turbines, the floating offshore wind turbine (FOWT) has attracted significant interest from both academics and industry. However, deterioration in system performance and unscheduled shutdowns will significantly increase operation and maintenance costs. To avoid system damage and ensure operation safety, there is a great demand for developing FOWT fault diagnosis to detect and identify faults as early as feasible. Due to the enormous complexity of FOWT including dynamics and nonlinearity, effective FOWT fault diagnosis poses a significant challenge. This paper develops a novel FOWT fault diagnosis method using bidirectional long short-term memory (BiLSTM) with sliding window. Firstly, the dynamics are embedded by stacking time-dependent measurements using the sliding window technique. Then, the augmented data is fed into a BiLSTM network to exploit the nonlinearity of FOWT. Consequently, a softmax classifier is used for fault diagnosis. A high-fidelity 10 MW spar FOWT benchmark based on the NREL Fatigue, Aerodynamics, Structures, and Turbulence (FAST) model is used to validate the capability and effectiveness of the proposed method. Experimental results show that the proposed method outperforms other related methods in diagnosing critical faults in FOWTs under various operating conditions.

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Solar irradiation forecast enhancement using clustering based CNN-BiLSTM-attention hybrid architecture with PSO

Accurate solar irradiation forecasting is essential for optimising solar energy use. This paper presents a novel forecasting approach: the ‘Clustering-based CNN-BiLSTM-Attention Hybrid Architecture with PSO’. It combines clustering, attention mechanisms, Convolutional Neural Networks (CNN), Bidirectional Long-Short Term Memory (BiLSTM) networks, and Particle Swarm Optimisation (PSO) into a unified framework. Clustering categorises days into groups, improving predictive capabilities. The CNN-BiLSTM model captures spatial and temporal features, identifying complex patterns. PSO optimises the hybrid model’s hyperparameters, while an attention mechanism assigns probability weights to relevant information, enhancing performance. By leveraging spatial and temporal patterns in solar data, the proposed model improves forecasting accuracy in univariate and multivariate analyses with multi-step predictions. Extensive tests on real-world datasets from various locations show the model’s effectiveness. For example, with NASA power data, the model achieves a Mean Absolute Error (MAE) of 24.028 W/m2, Root Mean Square Error (RMSE) of 43.025 W/m2, and an R 2 score of 0.984 for 1-hour ahead forecasting. The results show significant improvements over conventional methods.

Transformer assisted by DSC and BiLSTM for bearing fault pattern recognition under strong noise interference

ABSTRACT Deep learning has greatly promoted intelligent fault diagnosis for bearings and has led to the development of many intelligent fault recognition methods based on deep neural networks. However, bearing fault signals in industrial applications are susceptible to noise interference, and intelligent recognition methods for bearing faults under strong noise interference are lacking. An intelligent fault pattern recognition model termed IFPR-MDL, which is based on a Transformer assisted by depthwise separable convolutional (DSC) and bidirectional long short-term memory (BiLSTM) networks, is designed to identify bearing faults under strong noise conditions. In this model, multiscale features are first extracted by a multiscale feature extraction module designed on the basis of DSC; then, contextual information is further extracted by a BiLSTM, and information from different scales is integrated by Transformer encoders. The DSC network facilitates model parameter reduction, the BiLSTM network has expertise in mining temporal features, and the Transformer network excels at capturing long-range feature correlations. This model fully utilises the advantages of these models and complements them to improve the anti-interference ability of bearing fault pattern recognition. Tests on several public datasets show that the proposed model has higher recognition accuracy, stronger generalisation ability, and noise resistance in strong noise situations.

Research on Gold Price Prediction Based on LSTM Modeling

Gold plays a pivotal role in asset allocation, and the construction of gold price prediction models represents a complex yet rewarding task within the field of finance. The problem of international gold price is addressed in this paper forecasting by proposing a standard Long-Short Term Memory (LSTM) model and introducing bi-directional LSTM (Bi-LSTM) networks and multivariate analysis to compare the forecasting accuracy of the relevant models. This comparison is based on the daily gold price of the London Bullion Market Association (LBMA) for 2013-2022. This methodology takes into account various factors that influence the gold market and incorporates them as input sources into the Multi-factor LSTM model, thereby enhancing the interpretability of the LSTM model. Correlation analysis and the Granger test are employed to analyze these influencing factors. However, the results of this study suggest that while the Multi-factor LSTM model may be more accurate in predicting outliers, it tends to increase the number of small errors. On the other hand, the inclusion of Bi-LSTM networks can improve the overall accuracy of gold price prediction.

An Analytical Approach for IGBT Life Prediction Using Successive Variational Mode Decomposition and Bidirectional Long Short-Term Memory Networks

The precise estimation of the operational lifespan of insulated gate bipolar transistors (IGBT) holds paramount significance for ensuring the efficient and uncompromised safety of industrial equipment. However, numerous methodologies and models currently employed for this purpose often fall short of delivering highly accurate predictions. The analytical approach that combines the Pattern Optimization Algorithm (POA) with Successive Variational Mode Decomposition (SVMD) and Bidirectional Long Short-term Memory (BiLSTM) network is introduced. Firstly, SVMD is employed as an unsupervised feature learning method to partition the data into intrinsic modal functions (IMFs), which are used to eliminate noise and preserve the essential signal. Secondly, the BiLSTM network is integrated for supervised learning purposes, enabling the prediction of the decomposed sequence. Additionally, the hyperparameters of BiLSTM and the penalty coefficients of SVMD are optimized utilizing the POA technique. Subsequently, the various modal functions are predicted utilizing the trained prediction model, and the individual mode predictions are subsequently aggregated to yield the model’s definitive final life prediction. Through case studies involving IGBT aging datasets, the optimal prediction model was formulated and its lifespan prediction capability was validated. The superiority of the proposed method is demonstrated by comparing it with benchmark models and other state-of-the-art methods.

Memristor-based circuit design of BiLSTM network

The bidirectional long short-term memory (BiLSTM) network involves significant amount of parameter computations. This paper proposes the memristor-based bidirectional long short-term memory (MBiLSTM) network, with its capability of in-memory computing and parallel computing, can accelerates the parameter computations speed. The MBiLSTM network circuit is composed of normalization circuit, two memristor-based long short-term memory (LSTM) circuits, memristor-based resnet circuit, memristor-based dense circuitand winner-take-all (WTA) circuit. The voltage signals are scaled to the setting range by normalization circuit, memristor-based LSTM circuit is responsible for extracting features from the dataset, memristor-based resnet circuit can enhance the overall performance of the network, memristor-based dense circuit ensures that the final outputs dimension matches the dimension of the target signals, WTA circuit outputs the maximum voltage of memristor-based dense circuit. The effectiveness of the MBiLSTM network is validated through gait recognition experiment and handwritten digit recognition experiment. The stability, robustness and potential errors in the manufacturing process of memristance are analyzed.

Predicting temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by artificial neural networks

Predicting temporal evacuation travel time in staircases between adjacent floors is crucial for dynamic evacuation guidance to improve vertical evacuation efficiency during emergencies in super high-rise buildings. However, the prediction methods used in previous studies mainly focus on the total evacuation time in buildings and cannot predict the temporal evacuation travel time in every single part of the total evacuation route. This study proposes a novel method to predict temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings by using artificial neural networks. Firstly, evacuation simulation data of a 71-storey office building are analyzed for feature engineering and dataset building. Secondly, three types of artificial neural networks, including convolutional neural networks, RNN-based model (the traditional recurrent neural networks, long-short term memory networks, and bidirectional long-short term memory networks), and Attention-based model (Attention model and Transformer model) are extended to predict the temporal evacuation travel time in staircases between adjacent floors based on the simulation data. Finally, the prediction results by different artificial neural networks are compared. The results show that the artificial neural networks can provide accurate prediction values for the temporal evacuation travel time in staircases between adjacent floors of super high-rise buildings.

Enhancing temperature and torque prediction in permanent magnet synchronous motors using deep learning neural networks and BiLSTM RNNs

This study aims to develop an effective method for predicting the temperature and torque of Permanent Magnet Synchronous Motors (PMSMs) using deep learning techniques, which is crucial for optimizing motor performance and ensuring longevity, particularly in the automotive industry. Various Neural Network (NN) architectures, including a Recurrent Neural Network (RNN) with a Bidirectional Long Short-Term Memory (BiLSTM) unit, were employed to model the complex relationships between motor parameters, such as stator winding, current, torque, and permanent magnet temperature. The findings demonstrate that an NN with two hidden layers (64 and 32 neurons) achieved an R2 score of 0.99 for both torque and temperature prediction, while the BiLSTM network effectively modeled temporal dynamics, leading to high-fidelity rotor temperature predictions. This research provides a novel application of BiLSTM RNNs in accurately predicting PMSM temperatures, offering valuable insights for industries reliant on these motors. Integrating these models into motor control systems can enhance operational efficiency, reduce overheating risks, and extend motor lifespan, contributing to energy savings and environmental sustainability by lowering energy consumption and reducing waste.

Enhanced Foreign Exchange Volatility Forecasting Using Ceemdan with Optuna-Optimized Ensemble Deep Learning Model

Foreign Exchange (FX) is the largest financial market in the world, with a daily trading volume that significantly exceeds that of stock and futures markets. The prediction of FX volatility is a critical financial concern that has garnered significant attention from researchers and practitioners due to its far-reaching implications in the financial markets. This paper presents a novel hybrid ensemble forecasting model integrating a decomposition strategy and three deep learning (DL) models: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and Convolutional Neural Network (CNN). This combination addresses individual models' limitations and further improves the accuracy and stability of FX volatility forecasting. The proposed approach utilizes the CEEMDAN technique to decompose volatility into multiple distinct intrinsic mode functions (IMFs) and merges these IMFs with GARCH and EGARCH volatilities to form the input dataset for the DL models. In addition, we employed an attention mechanism to improve the effectiveness of the DL techniques. Furthermore, the hyperparameters for the DL models are optimized using the Optuna algorithm. Finally, a hybrid ensemble model for forecasting exchange rate volatility is developed by combining the predictions of three distinct DL models. The proposed approach is evaluated against various benchmark models using evaluation measures such as MSE, MAE, HMSE, HMAE, RMSE, Q-LIKE, and the model confidence set (MCS) approach. The results demonstrate that our proposed approach provides accurate and reliable forecasts of FX volatility under different forecasting regimes, making it a valuable tool for financial practitioners and researchers.

A novel bidirectional LSTM network model for very high cycle random fatigue performance of CFRP composite thin plates

This study proposes a novel Double-layer Bidirectional Long Short-Term Memory (BiLSTM) neural network model, shorted as TDA-BiLSTM, which integrates Transfer learning and Attention mechanisms. The model aims to predict the fatigue life of carbon fiber reinforced polymer (CFRP) thin plate structures subjected to very high cycle random vibration fatigue loads. Distinct from conventional servo-hydraulic and ultrasonic fatigue testing methods, this research pioneers the use of a vibration table for very high cycle fatigue (VHCF) testing to fill the gap in the field. By training and validating data across various life ranges, the TDA-BiLSTM model demonstrates significant advantages in training speed and predicting accuracy. Its bidirectional structure and attention mechanism effectively capture complex patterns and long-term dependencies in sequence data. Experimental results indicate that the TDA-BiLSTM model achieves significantly lower Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) compared to Long Short-Term Memory (LSTM) and Long Short-Term Memory with Transfer learning (Tr-LSTM) models across different life ranges on the ε-N curve, indicating higher accuracy and stability in strain life prediction tasks. Additionally, an analysis of typical damage areas using Scanning Electron Microscopy (SEM) reveals the failure mechanisms of CFRP plates under very high cycle vibration fatigue loads.

Comparison of Cobb-Douglas Production Function with Deep Learning Models to Estimate the Productivity of Organised Manufacturing Sector - Indian Perspective

The production function defines the output of a firm, industry, or economy based on various combinations of inputs. This study focuses on estimating the production function of India's manufacturing sector and determining the relationship between its inputs and outputs. To predict the output, measured as Gross Value Added (GVA), the study applies the Cobb Douglas Production Function (CDPF) along with several deep learning techniques, including Feedforward, Recurrent, Long Short-Term Memory (LSTM), and Bidirectional LSTM networks. Model performance is evaluated using metrics like Mean Square Error (MSE), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). The results indicate that the Cobb Douglas Production Function outperforms the deep learning models in predicting GVA for the organized manufacturing sector in India.

Nonlinear Ensemble Deep Learning Model for Energy Consumption Prediction with Bayesian Optimization

Accurate prediction of electric energy consumption is crucial for efficient load dispatching, energy utilization, and grid operation. Traditional statistical and classical machine learning methods struggle with the nonlinear nature of energy consumption data, often leading to higher prediction errors. Additionally, deep learning models using a single approach face challenges such as convergence to local minima and poor generalization. This paper proposes a nonlinear ensemble deep learning model for residential energy consumption prediction, incorporating Bayesian optimization for hyperparameter tuning. The model combines Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and 1D Convolutional Neural Networks (1D-CNN), leveraging their powerful nonlinear feature learning capabilities. A k-means clustering approach is used to preprocess and reduce variability in the data, enhancing the ensemble model's performance. The ensemble model was tested on real energy consumption data from two districts in Addis Ababa, showing significant improvements in prediction accuracy with lower MAE, RMSE, and MAPE values compared to single models and un-clustered data. The integration of clustering and Bayesian optimization further enhanced model generalizability and minimized overfitting, demonstrating the effectiveness of a nonlinear approach in capturing complex energy consumption patterns.

Short-term power load forecasting based on hybrid feature extraction and parallel BiLSTM network

Electricity is a vital resource for societal and economic activities. Accurate electricity load forecasting can effectively reduce costs and enhance energy efficiency. Nevertheless, current research has limitations in effectively extracting and utilizing load forecasting information. This encompasses both undiscovered latent features and models that have not effectively captured long-term dependencies. To address these issues, this paper proposes a novel network architecture, which utilizes a hybrid feature extraction strategy and integrates machine learning methods with deep learning techniques. This method employs a hybrid feature extraction strategy composed of gradient-boosted regression trees, Fisher Score, and mutual information to achieve comprehensive feature representation. The network architecture utilizes a multi-head convolutional neural network and a bi-directional long short-term memory (BiLSTM) with lag parameters. This architecture can simultaneously learn multiple levels and aspects of features, further enhancing the BiLSTM model’s ability to capture long-term dependencies. Through multiple experiments conducted on datasets from Maine, Singapore, New South Wales, Australia, and European countries, our method outperformed the comparison models in three out of the five datasets.

Missing well logs reconstruction based on cascaded bidirectional long short-term memory network

Well logs are the fundamental data for development and evaluation in the oil and gas field. In the actual exploitation process, missing or incomplete well logs are common due to borehole collapse and instrument failure. In this paper, a cascaded bidirectional long short-term memory network with residual attention is proposed to reconstruct well logs. Firstly, bidirectional long short-term memory (Bi-LSTM) is employed to extract the data features from the forward and backward direction of the missing well logs, considering the bi-directional correlation between the missing data and the context information with depth. Then, the residual structure and the attention mechanism are designed for Bi-LSTM network, named RA-Bi-LSTM, to obtain deeper semantic information of the long-sequence logging curve. Finally, we develop the cascaded RA-Bi-LSTM to solve the multiple logging curves reconstruction problems, where the synthetic logging curve obtained at each RA-Bi-LSTM module is combined with the known logging curves as new input to the next RA-Bi-LSTM module. Hence, unknown missing logging curves can be reconstructed by iterating the cascaded system. Extensive experiments and thorough analysis are validated on four wells in the Sulige gas field in China. Experimental results demonstrate the effectiveness and superiority of the proposed cascade structure on reconstruction accuracy.

Application of Deep Learning for Reservoir Porosity Prediction and Self Organizing Map for Lithofacies Prediction

While there is a connection between petrophysical logs and reservoir porosity, finding analytical solutions for this relationship is still difficult. This paper presents a novel approach for forecasting porosity and lithofacies by using a convolutional neural network (CNN) model in conjunction with a bi-directional long short-term memory (BLSTM) network. The BLSTM network uses a self-organizing map (SOM) technique to form connections between input and destination data. The SOM is used to organize depth intervals with similar facies into four separate clusters, each exhibiting internal consistency in petrophysical parameters. The CNN is responsible for extracting spatial characteristics, while the BLSTM network gathers comprehensive spatiotemporal components, guaranteeing that the model accurately represents the spatiotemporal aspects of log data. The accuracy of the model was verified by analyzing simulation logging data. The findings indicate that the BLSTM network model successfully recovers significant characteristics from logging data, resulting in improved estimate accuracy. In addition, Facies-01 has lower gamma ray levels in comparison to other facies. Facies-01 is also suggestive of pristine sandstone formations, which are greatly sought as reservoir rocks. The BLSTM network model is effective in predicting physical characteristics of reservoirs, offering a new method for precise reservoir characterization parameter prediction.

An advanced hybrid deep learning model for accurate energy load prediction in smart building

In smart cities, sustainable development depends on energy load prediction since it directs utilities in effectively planning, distributing and generating energy. This work presents a novel hybrid deep learning model including components of the Improved-convolutional neural network (CNN), bidirectional long short-term memory (Bi-LSTM), Graph neural network (GNN), Transformer and Fusion Layer architectures for precise energy load forecasting. Better feature extraction results from the Improved-CNN's dilated convolution and residual block accommodation of wide receptive fields reduced the vanishing gradient problem. By capturing temporal links in both directions, Bi-LSTM networks help to better grasp complicated energy use patterns. Graph neural networks improve predictive capacities across linked systems by characterizing the spatial relationships between energy-consuming units in smart cities. Emphasizing critical trends to guarantee reliable forecasts, transformer models use attention methods to manage long-term dependencies in energy consumption data. Combining CNN, Bi-LSTM, Transformer and GNN component predictions in a Fusion Layer synthesizes numerous data representations to increase accuracy. With Root Mean Square Error of 5.7532 Wh, Mean Absolute Percentage Error of 3.5001%, Mean Absolute Error of 6.7532 Wh and R2 of 0.9701, the hybrid model fared better than other models on the ‘Electric Power Consumption’ Kaggle dataset. This work develops a realistic model that helps informed decision-making and enhances energy efficiency techniques, promoting energy load forecasting in smart cities.

Glypred: Lysine Glycation Site Prediction via CCU-LightGBM-BiLSTM Framework with Multi-Head Attention Mechanism.

Glycation, a type of posttranslational modification, preferentially occurs on lysine and arginine residues, impairing protein functionality and altering characteristics. This process is linked to diseases such as Alzheimer's, diabetes, and atherosclerosis. Traditional wet lab experiments are time-consuming, whereas machine learning has significantly streamlined the prediction of protein glycation sites. Despite promising results, challenges remain, including data imbalance, feature redundancy, and suboptimal classifier performance. This research introduces Glypred, a lysine glycation site prediction model combining ClusterCentroids Undersampling (CCU), LightGBM, and bidirectional long short-term memory network (BiLSTM) methodologies, with an additional multihead attention mechanism integrated into the BiLSTM. To achieve this, the study undertakes several key steps: selecting diverse feature types to capture comprehensive protein information, employing a cluster-based undersampling strategy to balance the data set, using LightGBM for feature selection to enhance model performance, and implementing a bidirectional LSTM network for accurate classification. Together, these approaches ensure that Glypred effectively identifies glycation sites with high accuracy and robustness. For feature encoding, five distinct feature types─AAC, KMER, DR, PWAA, and EBGW─were selected to capture a broad spectrum of protein sequence and biological information. These encoded features were integrated and validated to ensure comprehensive protein information acquisition. To address the issue of highly imbalanced positive and negative samples, various undersampling algorithms, including random undersampling, NearMiss, edited nearest neighbor rule, and CCU, were evaluated. CCU was ultimately chosen to remove redundant nonglycated training data, establishing a balanced data set that enhances the model's accuracy and robustness. For feature selection, the LightGBM ensemble learning algorithm was employed to reduce feature dimensionality by identifying the most significant features. This approach accelerates model training, enhances generalization capabilities, and ensures good transferability of the model. Finally, a bidirectional long short-term memory network was used as the classifier, with a network structure designed to capture glycation modification site features from both forward and backward directions. To prevent overfitting, appropriate regularization parameters and dropout rates were introduced, achieving efficient classification. Experimental results show that Glypred achieved optimal performance. This model provides new insights for bioinformatics and encourages the application of similar strategies in other fields. A lysine glycation site prediction software tool was also developed using the PyQt5 library, offering researchers an auxiliary screening tool to reduce workload and improve efficiency. The software and data sets are available on GitHub: https://github.com/ZBYnb/Glypred.

Phase-Based Gait Prediction after Botulinum Toxin Treatment Using Deep Learning.

Gait disorders in neurological diseases are frequently associated with spasticity. Intramuscular injection of Botulinum Toxin Type A (BTX-A) can be used to treat spasticity. Providing optimal treatment with the highest possible benefit-risk ratio is a crucial consideration. This paper presents a novel approach for predicting knee and ankle kinematics after BTX-A treatment based on pre-treatment kinematics and treatment information. The proposed method is based on a Bidirectional Long Short-Term Memory (Bi-LSTM) deep learning architecture. Our study's objective is to investigate this approach's effectiveness in accurately predicting the kinematics of each phase of the gait cycle separately after BTX-A treatment. Two deep learning models are designed to incorporate categorical medical treatment data corresponding to the injected muscles: (1) within the hidden layers of the Bi-LSTM network, (2) through a gating mechanism. Since several muscles can be injected during the same session, the proposed architectures aim to model the interactions between the different treatment combinations. In this study, we conduct a comparative analysis of our prediction results with the current state of the art. The best results are obtained with the incorporation of the gating mechanism. The average prediction root mean squared error is 2.99° (R2 = 0.85) and 2.21° (R2 = 0.84) for the knee and the ankle kinematics, respectively. Our findings indicate that our approach outperforms the existing methods, yielding a significantly improved prediction accuracy.