Modular Aero-Propulsion System Simulation Research Articles

Deep attention SMOTE: Data augmentation with a learnable interpolation factor for imbalanced anomaly detection of gas turbines

Anomaly detection of gas turbines faces the significant challenges of data imbalance and inter-class overlap. In this paper, we develop a novel data augmentation method, namely deep attention synthetic minority over-sampling technique with the Encoder-Decoder (DA-SMOTE-ED), which serves as a key step in our hybrid re-sampling scheme. To reduce the risk of generating noise data, on one hand, the DA-SMOTE-ED leverages an Encoder-Decoder to learn a class-separable feature space to weaken the effect of inter-class overlap. On the other hand, an attention module is applied to assign proper interpolation factors to generate synthetic samples that stay off the aggregation area of normal samples. Moreover, synthetic samples are generated in the learnable feature space, mapped back to the original space, and merged with under-sampled samples to form the balanced dataset. Finally, the superiority of the developed method is validated through two case studies including the real monitoring data of gas turbines and the modified version of the commercial modular aero-propulsion system simulation (C-MAPPS) dataset. More specifically, its average balanced accuracy is 91.77 % on the gas turbine dataset, yielding 3.67 %, 6.4 %, and 5.56 % improvements compared to the SMOTE-ENN, TimeGAN, and AugmentTS, respectively.

Time Series-Based Sensor Selection and Lightweight Neural Architecture Search for RUL Estimation in Future Industry 4.0

Industry 4.0 is the current development trend in manufacturing technology. Regarding prognostic and health management (PHM) systems, a major task is to predict the remaining useful life (RUL) of a machine. To have precise RUL information, massive sensors should be involved, which leads to the dramatic cost of sensor network construction. Therefore, many sensor selection methods have been proposed to remove the redundant sensors under a certain constraint of RUL estimation. However, those approaches do not consider time-series sensing data, which is an intrinsic feature of mechanical signals. Therefore, the current research suffers from sensor under-killing problems due to the pessimistic consideration. To solve this problem, this work first considers the time-series sensing data to propose an integrated group-based valuable sensor selection algorithm by using the Least Absolute Shrinkage and Selection Operator (LASSO) property. In addition, to reduce the computing and hardware overhead, we further propose a heuristic neural architecture search (NAS) method to reduce the number of neurons in each neural network for further RUL estimation. To verify the proposed approaches, we use the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset and utilize the PHM score to evaluate the performance of RUL estimation. Compared with the conventional methods, the proposed sensor selection method can reduce the PHM score by 9% to 99% with fewer sensors. On the other hand, the proposed NAS method can reduce the number of neurons by 86% to 89% and reduce the PHM score by 17% to 79% compared with the traditional NAS approaches.

Modified DEMATEL Method Based on Objective Data Grey Relational Analysis for Time Series

Smart data selection can quickly sieve valuable information from initial data. Doing so improves the efficiency of analyzing situations to aid in better decision-making. Past methods have mostly been based on expert experience, which may be subjective and inefficient when dealing with large, complex datasets. Recently, the system analysis method has been exploited to find the key data. However, few studies address the indirect effects and heterogeneity of time series data. In this study, a data selection method, the modified Decision-Making Trial and Evaluation Laboratory (DEMATEL) method based on the objective data grey relational analysis (GRA), is used to enhance the ability to analyze time-series data. GRA was first applied to assess the direct impact in the raw data indicators. Then, a modified DEMATEL was adopted to find the overall impact by including the indirect impact and data heterogeneity. We applied the method to analyze the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset and perform the remaining useful life (RUL) prediction of aircraft engines. The results suggest that our method predicts well. Our work offers a nuanced approach of identifying key information in time series data and has potential applications.

Controlled physics-informed data generation for deep learning-based remaining useful life prediction under unseen operation conditions

Limited availability of representative time-to-failure (TTF) trajectories either limits the performance of deep learning (DL)-based approaches on remaining useful life (RUL) prediction in practice or even precludes their application. Generating synthetic data that is physically plausible is a promising way to tackle this challenge. In this study, a novel hybrid framework combining the controlled physics-informed data generation approach with a deep learning-based prediction model for prognostics is proposed. In the proposed framework, a new controlled physics-informed generative adversarial network (CPI-GAN) is developed to generate synthetic degradation trajectories that are physically interpretable and diverse. Five basic physics constraints are proposed as the controllable settings in the generator. A physics-informed loss function with penalty is designed as the regularization term, which ensures that the changing trend of system health state recorded in the synthetic data is consistent with the underlying physical laws. Then, the generated synthetic data is used as input of the DL-based prediction model to obtain the RUL estimations. The proposed framework is evaluated based on new Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS), a turbofan engine prognostics dataset where a limited availability of TTF trajectories is assumed. The experimental results demonstrate that the proposed framework is able to generate synthetic TTF trajectories that are consistent with underlying degradation trends. The generated trajectories enable to significantly improve the accuracy of RUL predictions.

Pseudo-Label-Vector-Guided Parallel Attention Network for Remaining Useful Life Prediction

Prognostic health management (PHM) has become important in many industries as a critical technology to increase machine stability and operational efficiency. Recently, various methods using deep learning to estimate the remaining useful life (RUL) as a core task of PHM have been proposed. However, the existing attention methods do not explicitly capture the correlation between temporal and spatial time series, reducing the RUL prediction accuracy. This article proposes a novel RUL prediction algorithm using a spatiotemporal attention mechanism based on the pseudo-label vectors to solve this problem. The proposed attention network uses the pseudo-label vector learned in the intermediate prediction process as a query vector to focus on time sequence data related to the RUL. Therefore, compared with conventional attention models that extract correlations for all the sequences, the proposed model captures features directly related to RUL with less computational cost. Experiments have been performed on two widely used datasets, and the experimental results show that the proposed approach outperforms the state of the art for root-mean-square error, with averages 4.27 and 3039 in the NASA Commercial Modular Aero-Propulsion System Simulation dataset and the IEEE PHM 2012 Prognostic challenge dataset, respectively. In addition, the analysis in the experiment reveals that the proposed model has better interpretability than the existing models by obtaining the correlation between time-series data and the RUL through the attention score in terms of time and features.

A new supervised multi-head self-attention autoencoder for health indicator construction and similarity-based machinery RUL prediction

Remaining useful life (RUL) prediction plays a significant role in the prognostic and health management (PHM) of rotating machineries. A good health indicator (HI) can ensure the accuracy and reliability of RUL prediction. However, numerous existing deep learning-based HI construction approaches rely heavily on the prior knowledge, and they are difficult to capture the key information in the process of machinery degradation from raw signals, thereby affecting the performance of RUL prediction. To tackle the aforementioned problem, a new supervised multi-head self-attention autoencoder (SMSAE) is proposed for extracting the HI that effectively reflects the degraded state of rotating machinery. By embedding the multi-head self-attention (MS) module into autoencoder and imposing the constraint of power function-type labels on the hidden variable, SMSAE can directly extract the HIs from raw vibration signals. As the current HI evaluation indexes don’t consider the global monotonicity and variation law of HI, two improved monotonicity and robustness indexes are designed for the better evaluation of HI. With the proposed HI, a two-stage residual life prediction framework based on similarity is developed. Extensive experiments have been performed on an actual wind turbine gearbox bearing dataset and a well-known open commercial modular aero-propulsion system simulation (C-MAPSS) dataset. The comparative results verify that the constructed SMSAE HI has better comprehensive performance than the typical HIs, and the proposed prediction method is competitive with the state-of-the-art methods.

Improved RUL Predictions of Aero- Engines by Hyper-Parameter Optimization of LSTM Neural Network Models.

Abstract: In the industry domains such as transportation, power generation and heavy engineering industries utilize complex machinery, as their working principles are very complex and it is very difficult to predict their exact life and operational availability due to difficulties involved in their operations. PHM- Prognostics and health management is one such domain which supports in the management of this machinery also known as assets for continuous monitoring for their best utilization and maximum efficiency. For any machine, the current health of the system can be determined by operational and raw sensor data. This data can also be utilized for the prediction of remaining useful life (RUL) of the system provided a reliable prediction model is available. It is always be challenging to determine the remaining useful life of complex machines such as turbofan jet engines from the knowledge of the data available from operating conditions and sensor data. An HPT LSTM- advanced hyper parameter tuned LSTM- Long Short Term Memory neural network model is developed to accurately determine the remaining useful life of the turbojet fan engines. To test this predictive model, a NASA developed database CMAPSS- Commercial Modular Aero Propulsion System Simulation is used. The performance of this HPT LSTM is found to be much improved as compared to reference machine learning algorithms such as simple linear regression and Lagged Multi- Layer Perceptron Models

Hierarchical graph neural network with adaptive cross-graph fusion for remaining useful life prediction

Multi-sensor monitoring data provide abundant information resources for complex machine systems, which facilitates monitoring the degradation process of machinery and ensuring the reliability of the industrial process. However, previous prognostic methods focus more on the sequential data obtained from multi-sensors, while ignoring the underlying prior structural information of the equipment. To fully leverage the structural information into the modeling process, and thus improve the remaining useful life (RUL) prediction performance, a hierarchical graph neural network with adaptive cross-graph fusion (HGNN-ACGF) method for RUL prediction is proposed in this study. In the HGNN-ACGF method, a hierarchical graph consisting of a sensor graph and a module graph is constructed by introducing the structural information to fully model the degradation trend information of the complex machine system. Besides, the graph neural network (GNN) is adopted to learn the representation at both the module graph and sensor graph, and an adaptive cross-graph fusion (ACGF) block is proposed. Owing to the cross-graph fusion block, the representation from different graphs can be fused adaptively by considering the relative importance between different modules and sensors. To verify the proposed method, the experiments were conducted on a set of degradation data sets of aircraft engines provided by the Commercial Modular Aero-Propulsion System Simulation. The experimental results show that the proposed method has superior performance in RUL prediction over the state-of-the-art methods.

A concise self-adapting deep learning network for machine remaining useful life prediction

Most remaining useful life (RUL) prediction methods learn the feature using a single fixed pattern, resulting in a lack of self-adapting learning capability and a decrease in generalization and prediction accuracy. To address this issue, we propose a concise self-adapting deep learning network (CSDLN) for RUL prediction with fewer learnable parameters per sub-module. First, a multi-branch 1D involution neural network (MINN) is proposed to adaptively extract the hidden feature from the multi-input using the involution operation, which has inverse inherence with the convolution operation. Second, an adaptive learning algorithm called the multi-head gated recurrent unit (MGRU) is proposed to learn the hidden feature. Finally, the aero-engine RUL is determined by dimension reduction of the full connection (FC) layer and linear regression of the regression layer. Additionally, rather than using the ReLU activation function, the Mish activation function is used to strengthen the self-adapting deep learning ability in CSDLN. The performance in the Commercial Modular Aero Propulsion System Simulation (C-MAPSS) dataset and the real wind turbine gearbox bearing tests demonstrate the superiority of CSDLN over the state-of-the-art RUL prediction methods. Meanwhile, dropout is adopted in the model for avoiding overfitting and achieving the uncertainty quantification of RUL prediction in the two applications.

Least squares smoothed k-nearest neighbors online prediction of the remaining useful life of a NASA turbofan

An accurate prediction of the Remaining Useful Life (RUL) of aircraft engines plays a fundamental role in the aerospace field since it is both mission and safety critical. In fact, a reliable estimate of the RUL can effectively reduce the maintenance costs while fostering safety. This paper proposes a novel data-driven method to increase accuracy of the RUL prediction for real-time prognostic systems, considering multiple degradation mechanisms and making the model easy to implement. The proposed method exploits a novel modified k-Nearest Neighbors Interpolation (kNNI) with an a posteriori Least Square Smoothing (LSS) automatically optimized to obtain the minimum prediction error. The LSS novel formulation was also generalized and proved to be equivalent to a Cumulative and Moving Average (CMA) mixture filter, which can be easily implemented online. The method was developed and validated based on a new NASA dataset generated by the dynamic model Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) with run-to-failure data related to a small fleet of aircraft engines under realistic flight conditions. Finally, a reference kNN-based method already known in the literature was compared to the novel proposed one to demonstrate the goodness of the results and the performance improvements.

Remaining Useful Life Prediction for Aero-Engines Using a Time-Enhanced Multi-Head Self-Attention Model

Data-driven Remaining Useful Life (RUL) prediction is one of the core technologies of Prognostics and Health Management (PHM). Committed to improving the accuracy of RUL prediction for aero-engines, this paper proposes a model that is entirely based on the attention mechanism. The attention model is divided into the multi-head self-attention and timing feature enhancement attention models. The multi-head self-attention model employs scaled dot-product attention to extract dependencies between time series; the timing feature enhancement attention model is used to accelerate and enhance the feature selection process. This paper utilises Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) turbofan engine simulation data obtained from NASA Ames’ Prognostics Center of Excellence and compares the proposed algorithm to other models. The experiments conducted validate the superiority of our model’s approach.

A comparative Study of Data-driven Prognostic Approaches under Training Data Deficiency

In this study, a remaining useful life (RUL) prognostic performance is explored for the three algorithms under the insufficient training datasets: data augmentation prognostics (DAPROG) developed by the authors, and trajectory similarity-based prediction (TSBP) and multilayer perceptron (MLP) based neural network, which is one of the popular approaches in the literature. The Commercial Modular Aero-Propulsion System Simulation (CMAPSS) datasets are taken to assess each algorithm. Only a fraction is chosen randomly from the training dataset to simulate the data deficiency and evaluate the prognostic performance. As a result, the DAPROG is found to be the most superior in terms of the prediction error when the data deficiency is severe. The prediction variance for the chosen training dataset is also small, indicating more uniform performance.

ProgNet: A Transferable Deep Network for Aircraft Engine Damage Propagation Prognosis under Real Flight Conditions

Machine learning prognosis for condition monitoring of safety-critical systems, such as aircraft engines, continually faces challenges of data unavailability, complexity, and drift. Consequently, this paper overcomes these challenges by introducing adaptive deep transfer learning methodologies, strengthened with robust feature engineering. Initially, data engineering encompassing: (i) principal component analysis (PCA) dimensionality reduction; (ii) feature selection using correlation analysis; (iii) denoising with empirical Bayesian Cauchy prior wavelets; and (iv) feature scaling is used to obtain the required learning representations. Next, an adaptive deep learning model, namely ProgNet, is trained on a source domain with sufficient degradation trajectories generated from PrognosEase, a run-to-fail data generator for health deterioration analysis. Then, ProgNet is transferred to the target domain of obtained degradation features for fine-tuning. The primary goal is to achieve a higher-level generalization while reducing algorithmic complexity, making experiments reproducible on available commercial computers with quad-core microprocessors. ProgNet is tested on the popular New Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) dataset describing real flight scenarios. To the extent we can report, this is the first time that all N-CMAPSS subsets have been fully screened in such an experiment. ProgNet evaluations with numerous metrics, including the well-known CMAPSS scoring function, demonstrate promising performance levels, reaching 234.61 for the entire test set. This is approximately four times better than the results obtained with the compared conventional deep learning models. In addition, designed codes of ongoing experiments, from data preparation to application, are available online.

Aero-Engine Remaining Useful Life Estimation Based on CAE-TCN Neural Networks

With the rapid growth of the aviation fields, the remaining useful life (RUL) estimation of aero-engine has become the focus of the industry. Due to the shortage of existing prediction methods, life prediction is stuck in a bottleneck. Aiming at the low efficiency of traditional estimation algorithms, a more efficient neural network is proposed by using Convolutional Neural Networks (CNN) to replace Long-Short Term Memory (LSTM). Firstly, multi-sensor degenerate information fusion coding is realized with the convolutional autoencoder (CAE). Then, the temporal convolutional network (TCN) is applied to achieve efficient prediction with the obtained degradation code. It does not depend on the iteration along time, but learning the causality through a mask. Moreover, the data processing is improved to further improve the application efficiency of the algorithm. ExtraTreesClassifier is applied to recognize when the failure first develops. This step can not only assist labelling, but also realize feature filtering combined with tree model interpretation. For multiple operation conditions, new features are clustered by K-means++ to encode historical condition information. Finally, an experiment is carried out to evaluate the effectiveness on the Commercial Modular Aero-Propulsion System Simulation (CMAPSS) datasets provided by the National Aeronautics and Space Administration (NASA). The results show that the proposed algorithm can ensure high-precision prediction and effectively improve the efficiency.

Semi-Supervised Framework with Autoencoder-Based Neural Networks for Fault Prognosis

This paper presents a generic framework for fault prognosis using autoencoder-based deep learning methods. The proposed approach relies upon a semi-supervised extrapolation of autoencoder reconstruction errors, which can deal with the unbalanced proportion between faulty and non-faulty data in an industrial context to improve systems' safety and reliability. In contrast to supervised methods, the approach requires less manual data labeling and can find previously unknown patterns in data. The technique focuses on detecting and isolating possible measurement divergences and tracking their growth to signalize a fault's occurrence while individually evaluating each monitored variable to provide fault detection and prognosis. Additionally, the paper also provides an appropriate set of metrics to measure the accuracy of the models, which is a common disadvantage of unsupervised methods due to the lack of predefined answers during training. Computational results using the Commercial Modular Aero Propulsion System Simulation (CMAPSS) monitoring data show the effectiveness of the proposed framework.

Determining RUL Predictive Maintenance on Aircraft Engines Using GRU

Prognostic and health management (PHM) in the aviation industry is expanding because of its effect on economic and human safety. Advanced maintenance shall be applied to this industry to inform aircraft engine conditions. PdM (Predictive Maintenance) is an advanced maintenance technique that can be applied to the aviation industry because of its high-precision prediction. Combining PdM as a technique to calculate the RUL (Remaining Useful Lifetime ) and ML (Machine Learning) as a tool to make high-accuracy predictions is mixed together that accurately forecasts the state of aircraft machine condition and on the best time to get the maintenance or service. In this work, we use the NASA Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) data set. This work proposes GRU to determine RUL on aircraft engines to implement a Predictive maintenance strategy. For the training parameters tested in this study, we used a batch size of 512, a learning rate with Adam optimizer of 0.001, then epochs of 200. The essence of the results of this experiment is to obtain a new method with a simpler calculation process and the epoch value and a faster prediction process compared to other methods used, and the results obtained can approach the original value from an economic point of view and the RUL prediction process using the GRU.

AI-Based Exhaust Gas Temperature Prediction for Trustworthy Safety-Critical Applications

Data-driven condition-based maintenance (CBM) and predictive maintenance (PdM) strategies have emerged over recent years and aim at minimizing the aviation maintenance costs and environmental impact by the diagnosis and prognosis of aircraft systems. As the use of data and relevant algorithms is essential to AI-based gas turbine diagnostics, there are different technical, operational, and regulatory challenges that need to be tackled in order for the aeronautical industry to be able to exploit their full potential. In this work, the machine learning (ML) method of the generalised additive model (GAM) is used in order to predict the evolution of an aero engine’s exhaust gas temperature (EGT). Three different continuous synthetic data sets developed by NASA are employed, known as New Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS), with increasing complexity in engine deterioration. The results show that the GAM can be predict the evolution of the EGT with high accuracy when using several input features that resemble the types of physical sensors installed in aero gas turbines currently in operation. As the GAM offers good interpretability, this case study is used to discuss the different data attributes a data set needs to have in order to build trust and move towards certifiable models in the future.

Evaluating Image Classification Deep Convolutional Neural Network Architectures for Remaining Useful Life Estimation of Turbofan Engines

Accurate estimation of the remaining useful life (RUL) is a key component of condition-based maintenance (CBM) and prognosis and health management (PHM). Data-based models for the estimation of RUL are of particular interest because expert knowledge of systems is not always available, and physical modeling is often not feasible. Additionally, using data-based models, which make decisions based on raw sensor data, allow features to be learned instead of manually determined. In this work, deep convolutional neural network (CNN) architectures are investigated for their ability to estimate the RUL of turbofan engines. To improve the accuracy of the models, CNN architectures, which have proven successful in image classification, are implemented and tested. Specifically, the blocks used in the Visual Geometry Group (VGG) architecture, inception modules used in the GoogLeNet architecture, and residual blocks used in the ResNet architecture are incorporated. To account for varying flight lengths, the input to the models is a window of time series data collected from the engine under test. Window locations at the climb, cruise, and descent stages are considered. To further improve the RUL estimations, multiple overlapping windows at each location are used. This increases the amount of training data available and is found to increase the accuracy of the resulting RUL estimations by averaging the estimates from all overlapping segments. The model is trained and tested using the new Commercial Modular Aero-Propulsion System Simulation (N-CMAPSS) data set, and high prognosis accuracy was achieved. This work expands on the model developed and used in the 2021 PHM Society Data Challenge, which received second place.

Remaining Useful Life Prediction Based on Multisensor Fusion and Attention TCN-BiGRU Model

Predicting remaining useful life (RUL) accurately is crucial for improving the reliability of the digitized maintenance and optimizing the operating cycle of devices. Based on the attention mechanism, this article proposes a temporal convolutional neural network (TCN)-bidirectional gate recurrent unit (BiGRU) model for the RUL prediction. The proposed model can accurately predict the RUL by fusing the time-series information recorded by multiple sensors. To suppress noise signals, the sensor’s raw signal filtered by the contribution rate is processed by variational mode decomposition (VMD) and decomposed into intrinsic mode components (IMFs) with different modes. Then, grid search and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> -fold cross validation are performed to determine the optimal hyperparameters of the proposed Attention TCN-BiGRU model to accurately capture the correlation of IMFs in the spatial dimension and the decay characteristics in the temporal dimension. The performance of the proposed model is validated on the commercial modular aero-propulsion system simulation (C-MAPSS) turbofan engine dataset and the sealing ring dataset. The experimental results indicate that the proposed model can achieve more accurate RUL predictions than other related models.

Spatiotemporally Multidifferential Processing Deep Neural Network and its Application to Equipment Remaining Useful Life Prediction

In this article, facing the gaps that the traditional long short-term memory (LSTM) and convolution neural network (CNN) cannot differentially deal with the input data based on the corresponding trend and stage information in remaining useful life (RUL) prediction, a more accurate and robust RUL prediction model is constructed. First, a temporally multidifferential LSTM (TMLSTM) with the multitrend division unit and multicellular unit is proposed, and a spatially multidifferential CNN (SMCNN) with the multistage division unit and differentiated convolutions is designed. Then, by combining TMLSTM and SMCNN, a spatiotemporally multidifferential deep neural network is developed for predicting the equipment RUL, which enhances the ability of feature extraction from the spatiotemporal perspective by using the multitrend and multistage information. Via several evaluation indexes, the commercial modular aero propulsion system simulation dataset and the wind turbine gearbox bearing dataset are used to validate the superiority of the proposed method over several existing prediction methods.