Modular Aero-Propulsion System Simulation Research Articles

Robustness testing framework for RUL prediction Deep LSTM networks

Efficiency and robustness in remaining useful life (RUL) prediction are crucial in system health monitoring. Thus, the internal logic computation of a Deep LSTM model for RUL prediction is mainly shaped and evaluated over a training data-set and its performance examined on a testing data-set. This paper proposes a framework for testing robustness of deep Long Short Term Memory (LSTM) architecture for remaining useful life prediction that enables to gain confidence in the trained LSTM model for RUL prediction and ensures better quality. The resiliency of proposed Deep LSTM networks for RUL estimation using stress functions is first checked then the effect of the stress on model performance is analyzed. A comparison between the performance of the constructed mutant fuzzed Deep LSTM networks and the original Deep LSTM model for RUL prediction is provided to determine the quality of the RUL prediction model.Furthermore, the main purpose of this paper is to determine to what extent Deep LSTM models in the neighborhood of the trained LSTM model still have high test accuracy and quality scoring. Thus, the use of φ-stress operators shows that we could build stable and data-independent Deep LSTM models for RUL prediction. Finally, the proposed framework is validated using the Commercial Modular Aero Propulsion System Simulation (C-MAPSS) data-set.

Feature Extraction Methods for Prognosis Maintenance Model

Research in prognosis maintenance, a branch of condition-based maintenance has received more attention from researchers lately. They focus on predicting when is the most suitable time to perform maintenance. Our review suggests that investigation on feature extraction in development of prognosis prediction model is still limited. This paper presents our study to find the most effective method for features extraction from maintenance monitoring data. The chosen features set should effectively improve the prognosis maintenance model performance. There have been several investigations to study feature extraction methods; however, the appropriate one is yet to be identified. In this research, we used datasets publicly available from National Aeronautics and Space Administration (NASA) army research laboratory. These datasets were generated through a simulation of the turbofan engine by using Commercial Modular Aero-Propulsion System Simulation (CMAPSS) software developed by NASA army research laboratory. Features extraction methods such as correlation among sensors, correlation among the outputs, variable weighing and treated data methods were studied in this research. Next, the extracted features were applied to the regression tree for searching an appropriate prognosis model. Based on the Remaining Useful Life (RUL) prediction results, the correlation among sensors method was found as the best method that can represent the most useful features for the prediction model.

A relative entropy based feature selection framework for asset data in predictive maintenance

Predictive maintenance (PdM) is applied to monitor a system’s life cycle to provide current diagnostics, prognostics and provide information capable of guiding maintenance related decisions. Often, an asset’s life cycle is monitored using multiple measurements which translate to high-dimensional (multivariate) data. The large volume of data used to describe an asset’s life cycle has led to current state-of-the-art data-driven PdM relying on machine learning (ML). As research shows, high-dimensional data diminish ML algorithm performance. Generally, high-dimensionality is managed by feature engineering, except asset data characteristics differ from characteristics managed in typical feature engineering problems. In data-driven PdM, information regarding observed faults in an asset is important. Such information is often misinterpreted or lost when general feature engineering is performed on asset data. This work proposes a correlation and relative entropy (C-RE) feature engineering framework specific to asset data. C-RE, applies correlation based hierarchical clustering and relative entropy through the measure of Kullback–Leibler divergence to generate a lower-dimensional feature subset of the original data. The resulting feature subset has minimal redundancies and the highest content of domain-specific information relating to the influence of faults observed during an asset’s life cycle. The utility of C-RE is demonstrated on the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset which describes the run-to-failure life cycles of multiple aircraft engines.

Extension of labeled multiple attribute decision making based on fuzzy neighborhood three-way decision

Weight assignment of attribute is considered as a key part of multiple attribute decision making (MADM), and this is also applicable to labeled multiple attribute decision making (LMADM) that is a decision theory specially proposed for the dataset with labels. However, regarding the decision making of massive data characterized by redundancy and uncertainty, more means including attribute selection and uncertainty processing should be considered to solve these problems. Based on the traditional framework of LMADM, this paper deduces a new framework to adapt to the decision making of massive data. With respect to the uncertainty generated from data and decision process, a fuzzy neighborhood three-way decision model (FN3WD) is proposed, in which the fuzzy neighborhood relationship can address the uncertainty of data and the three-way decision theory can deal with the uncertainty of decision process. Finally, the experimental results illustrate the superiority of FN3WD and verify the effectiveness of the proposed framework of the extended LMADM by using some benchmarked datasets and the Commercial Modular Aero-Propulsion System Simulation dataset.

Deep Learning with Dynamically Weighted Loss Function for Sensor-Based Prognostics and Health Management.

Deep learning has been employed to prognostic and health management of automotive and aerospace with promising results. Literature in this area has revealed that most contributions regarding deep learning is largely focused on the model’s architecture. However, contributions regarding improvement of different aspects in deep learning, such as custom loss function for prognostic and health management are scarce. There is therefore an opportunity to improve upon the effectiveness of deep learning for the system’s prognostics and diagnostics without modifying the models’ architecture. To address this gap, the use of two different dynamically weighted loss functions, a newly proposed weighting mechanism and a focal loss function for prognostics and diagnostics task are investigated. A dynamically weighted loss function is expected to modify the learning process by augmenting the loss function with a weight value corresponding to the learning error of each data instance. The objective is to force deep learning models to focus on those instances where larger learning errors occur in order to improve their performance. The two loss functions used are evaluated using four popular deep learning architectures, namely, deep feedforward neural network, one-dimensional convolutional neural network, bidirectional gated recurrent unit and bidirectional long short-term memory on the commercial modular aero-propulsion system simulation data from NASA and air pressure system failure data for Scania trucks. Experimental results show that dynamically-weighted loss functions helps us achieve significant improvement for remaining useful life prediction and fault detection rate over non-weighted loss function predictions.

Remaining useful life prediction using multi-scale deep convolutional neural network

Accurate and reliable remaining useful life (RUL) assessment result provides decision-makers valuable information to take suitable maintenance strategy to maximize the equipment usage and avoid costly failure. The conventional RUL prediction methods include model-based and data-driven. However, with the rapid development of modern industries, the physical model is becoming less capable of describing sophisticated systems, and the traditional data-driven methods have limited ability to learn sophisticated features. To overcome these problems, a multi-scale deep convolutional neural network (MS-DCNN) which have powerful feature extraction capability due to its multi-scale structure is proposed in this paper. This network constructs a direct relationship between Condition Monitoring (CM) data and ground-RUL without using any prior information. The MS-DCNN has three multi-scale blocks (MS-BLOCKs), where three different sizes of convolution operations are put on each block in parallel. This structure improves the network’s ability to learn complex features by extracting features of different scales. The developed algorithm includes three stages: data pre-processing, model training, and RUL prediction. After the min–max normalization pre-processing, the data is sent to the MS-DCNN network for parameter training directly, and the associated RUL value can be estimated base on the learned representations. Regularization helps to improve prediction accuracy and alleviate the overfitting problem. We evaluate the method on the available modular aero-propulsion system simulation data (C-MAPSS dataset) from NASA. The results show that the proposed method achieves good prognostics performance compared with other network architectures and state-of-the-art methods. RUL prediction result is obtained precisely without increasing the calculation burden.

NBLSTM: Noisy and Hybrid Convolutional Neural Network and BLSTM-Based Deep Architecture for Remaining Useful Life Estimation

Abstract Smart manufacturing and industrial Internet of things (IoT) have transformed the maintenance management concept from the conventional perspective of being reactive to being predictive. Recent advancements in this regard has resulted in development of effective prognostic health management (PHM) frameworks, which coupled with deep learning architectures have produced sophisticated techniques for remaining useful life (RUL) estimation. Accurately predicting the RUL significantly empowers the decision-making process and allows deployment of advanced maintenance strategies to improve the overall outcome in a timely fashion. In light of this, the paper proposes a novel noisy deep learning architecture consisting of multiple models designed in parallel, referred to as noisy and hybrid deep architecture for remaining useful life estimation (NBLSTM). The proposed NBLSTM is designed by integration of two parallel noisy deep architectures, i.e., a noisy convolutional neural network (CNN) to extract spatial features and a noisy bidirectional long short-term memory (BLSTM) to extract temporal information learning the dependencies of input data in both forward and backward directions. The two paths are connected through a fusion center consisting of fully connected multilayers, which combines their outputs and forms the target predicted RUL. To improve the robustness of the model, the NBLSTM is trained based on noisy input signals leading to significantly robust and enhanced generalization behavior. Through 100 Monte Carlo simulation runs performed under three different signal-to-noise ratio (SNR) values, it can be noted that utilization of the noisy training enhanced the results by reducing the standard deviation (std) between 9% and 67% across different settings in terms of the root-mean-square error (RMSE) and between 21% and 63% in terms of the score value. The proposed NBLSTM model is evaluated and tested based on the commercial modular aero-propulsion system simulation (C-MAPSS) dataset provided by NASA, illustrating state-of-the-art results in comparison with its counterparts.

Remaining Useful Life Prediction for Complex Systems With Multiple Indicators Based on Particle Filter and Parameter Correlation

In practical applications, the failure of large-scale complex equipment is often caused by the simultaneous degradation of multiple components. It is necessary to predict the remaining useful life (RUL) of the equipment with multiple degradation indicators. This article proposes a new joint-RUL-prediction method in the presence of multiple degradation indicators based on parameter correlation. The stochastic process model is established for each degradation indicator, and the model parameters are estimated by kernel smoothing particle filter (KS-PF) and maximum likelihood estimation (MLE). Meanwhile, to facilitate the dependencies between multiple degradation indicators, correlations of the degradation model parameter between multiple degradation indicators are established in KS-PF. In addition, optimal tuning (OT) is introduced to choose the best kernel parameter. A case study on the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset is applied to verify the proposed method, the experiment shows that the proposed joint-RUL-prediction method based on parameter correlation possesses a superior prediction performance compared with that by using a single degradation indicator.

Gated Recurrent Unit Networks for Remaining Useful Life Prediction

Abstract Remaining useful life prediction is a key procedure for prognostics and health management. However, traditional data-driven methods rely on handcrafted feature selection from the whole range of time series data, which may not obtain the temporal information for complex systems. This study proposes a gated recurrent unit networks based approach to predict remaining useful life. First, time window approach is applied on sample preparation for multiple sensor data. In particular, unsupervised stacked sparse autoencoder is utilized to automatically extract nonlinear features, then the selected features are fed into gated recurrent unit based recurrent neural networks to predict remaining useful life. The effectiveness of the proposed method is demonstrated on the commercial modular aero-propulsion system simulation data from NASA. Experimental results validate that the proposed approach achieves better prediction performance than other methods.

Autoencoder based Semi-Supervised Anomaly Detection in Turbofan Engines

This paper proposes a semi-supervised autoencoder based approach for the detection of anomalies in turbofan engines. Data used in this research is generated through simulation of turbofan engines created using a tool known as Commercial Modular Aero-Propulsion System Simulation (CMAPSS). C-MAPSS allows users to simulate various operational settings, environmental conditions, and control settings by varying various input parameters. Optimal architecture of autoencoder is discovered using Bayesian hyperparameter tuning approach. Autoencoder model with optimal architecture is trained on data representing normal behavior of turbofan engines included in training set. Performance of trained model is then tested on data of engines included in test set. To study the effect of redundant features removal on performance, two approaches are implemented and tested: with and without redundant features removal. Performance of proposed models is evaluated using various performance evaluation metrics like F1-score, Precision and Recall. Results have shown that best performance is achieved when autoencoder model is used without redundant feature removal.

Recommendation of PHM algorithms based on fuzzy information fusion

The diversity of algorithms and the complication of application scenarios makes it difficult for PHM (Prognostics and Health Management) developers to accurately choose an appropriate algorithm when performing algorithm design. To make up for the aforementioned shortcoming, this paper proposes an algorithm recommendation system suitable for PHM. Specifically, according to the PHM architecture, a PHM database is designed to assist in the mining of recommended knowledge and the execution of recommended strategies. Subsequently, a recommendation system is developed that mainly includes hybrid data processing, similarity measurement, and recommendation decision making. Therein, two recommendation strategies, based on Fuzzy C-Means (FCM) method and weighted information fusion, are designed to cope with two kinds of actual requirements. Finally, the recommendation of remaining useful life (RUL) prediction associated with the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) data set is employed as an application case to verify the effectiveness of the proposed recommendation system.

A machine learning approach to circumventing the curse of dimensionality in discontinuous time series machine data

The growing interest in artificial intelligence has led to current data-driven predictive maintenance (PdM) relying on machine learning (ML) algorithms. Although ML algorithms are useful for data-intensive analysis, research shows that their performance and reliability are reduced when high-dimensional data is used for training and testing. Raw machine data can be high-dimensional due to multi-sensor measurements and discontinuous due to the wide ranges of parameter variations during continuous sensor measurements. While standard dimension reduction methods such as principal component analysis are often applied to circumvent high-dimensionality, they are often unreliable when the data is discontinuous. This paper presents a ML-based dimension reduction framework to circumvent the challenges of high-dimensional discontinuous machine data. This framework minimizes discontinuity by clustering observations based on the dataset’s modality. The modality is identified using a kernel density estimation parameterized using a heat diffusion solution to the approximate mean integrated squared error. Then, low-dimension representations of each cluster are learned using Laplacian eigenmaps embedding. Finally, the original time sequence of observations across the low-dimensional clusters is used to re-index the observations into a continuous low-dimension feature set. We demonstrate the framework’s utility on common ML-based PdM analysis using the Commercial Modular Aero-Propulsion System Simulation dataset.

Convolution and Long Short-Term Memory Hybrid Deep Neural Networks for Remaining Useful Life Prognostics

Reliable prediction of remaining useful life (RUL) plays an indispensable role in prognostics and health management (PHM) by reason of the increasing safety requirements of industrial equipment. Meanwhile, data-driven methods in RUL prognostics have attracted widespread interest. Deep learning as a promising data-driven method has been developed to predict RUL due to its ability to deal with abundant complex data. In this paper, a novel scheme based on a health indicator (HI) and a hybrid deep neural network (DNN) model is proposed to predict RUL by analyzing equipment degradation. Explicitly, HI obtained by polynomial regression is combined with a convolutional neural network (CNN) and long short-term memory (LSTM) neural network to extract spatial and temporal features for efficacious prognostics. More specifically, valid data selected from the raw sensor data are transformed into a one-dimensional HI at first. Next, both the preselected data and HI are sequentially fed into the CNN layer and LSTM layer in order to extract high-level spatial features and long-term temporal dependency features. Furthermore, a fully connected neural network is employed to achieve a regression model of RUL prognostics. Lastly, validated with the aid of numerical and graphic results by an equipment RUL dataset from the Commercial Modular Aero-Propulsion System Simulation(C-MAPSS), the proposed scheme turns out to be superior to four existing models regarding accuracy and effectiveness.

Remaining useful lifetime prediction via deep domain adaptation

In Prognostics and Health Management (PHM) sufficient prior observed degradation data is usually critical for Remaining Useful Lifetime (RUL) prediction. Most previous data-driven methods assume that training (source) and testing (target) condition monitoring data have similar distributions. However, due to different operating conditions, fault modes and noise, distribution and feature shift exist across different domains. This shift reduces the performance of predictive models when no target observed run-to-failure data is available. To address this issue, this paper proposes a new data-driven approach for domain adaptation in prognostics using Long Short-Term Neural Networks (LSTM). We use a Domain Adversarial Neural Network (DANN) approach to adapt remaining useful life estimates to a target domain containing only sensor information. We analyse our approach using the NASA Commercial Modular Aero-Propulsion System Simulation (C-MAPPS). The results show that the proposed method can provide more reliable RUL predictions than models trained only on source data for varying operating conditions and fault modes.

A novel deep capsule neural network for remaining useful life estimation

With the availability of cheaper multi-sensor systems, one has access to massive and multi-dimensional sensor data for fault diagnostics and prognostics. However, from a time, engineering and computational perspective, it is often cost prohibitive to manually extract useful features and to label all the data. To address these challenges, deep learning techniques have been used in the recent years. Within these, convolutional neural networks have shown remarkable performance in fault diagnostics and prognostics. However, this model present limitations from a prognostics and health management perspective: to improve its feature extraction generalization capabilities and reduce computation time, ill-based pooling operations are employed, which require sub-sampling of the data, thus loosing potentially valuable information regarding an asset’s degradation process. Capsule neural networks have been recently proposed to address these problems with strong results in computer vision–related classification tasks. This has motivated us to extend capsule neural networks for fault prognostics and, in particular, remaining useful life estimation. The proposed model, architecture and algorithm are tested and compared to other state-of-the art deep learning models on the benchmark Commercial Modular Aero Propulsion System Simulation turbofans data set. The results indicate that the proposed capsule neural networks are a promising approach for remaining useful life prognostics from multi-dimensional sensor data.

An Adaptive Model-Based Framework for Prognostics of Gas Path Faults in Aircraft Gas Turbine Engines

This paper presents an adaptive framework for prognostics in civil aero gas turbine engines, which incorporates both performance and degradation models, to predict the remaining useful life of the engine components that fail predominantly by gradual deterioration over time. Sparse information about the engine configuration is used to adapt a performance model, which serves as a baseline for implementing optimum sensor selection, operating data correction, fault isolation, noise reduction and component health diagnostics using nonlinear Gas Path Analysis (GPA). Degradation models, which describe the progression of faults until failure, are then applied to the diagnosed component health indices from previous run-to-failure cases. These models constitute a training library from which fitness evaluation to the current test case is done. The final remaining useful life (RUL) prediction is obtained as a weighted sum of individually evaluated RULs for each training case. This approach is validated using dataset generated by the Commercial Modular Aero-Propulsion System Simulation (CMAPSS) software, which comprises both training and testing instances of run-to-failure sensor data for a turbofan engine, some of which are obtained at different operating conditions and for multiple fault modes. The results demonstrate the capability of improved prognostics of faults in aircraft engine turbomachinery using models of system behavior, with continuous health monitoring data.

A Relative Entropy Weibull-SAX framework for health indices construction and health stage division in degradation modeling of multivariate time series asset data

Predictive maintenance is the monitoring of an asset’s condition over its life cycle to provide a prognosis for when maintenance is required. Prior to prognosis, an asset’s life cycle is modeled by a health indicator (HI) which is derived from sensor measurements and describes an asset’s degradation over a life cycle. Often an asset’s HI is accompanied by a health stage (HS) division, which describes the asset’s life cycle condition in discrete states. Generally, HSs are discrete representations generated from discrete state transition models, dynamic state space models, or subjectively defined thresholds, which use sensor measurements to provide a HS division related to an asset’s life cycle degradation. HS division methods are often designed for a specific asset in which HS division is based on user assumptions and not generalizable to different asset types or asset data representations. Also, HS division methods are often limited to a bi-state HS division (normal and failure), in which unobservable states are often generalized transition states. As assets become more complex and require multivariate measurements, more advanced methods are required to model an asset’s degradation using HIs and HSs. This work introduces Relative Entropy Weibull-SAX (REWS), a data-driven HI and HS degradation modeling method for multivariate asset data. REWS constructs a HI using relative entropy to represent an asset’s condition as a change of entropy during its life cycle. The relative entropy representation is then discretized into HS divisions using a Weibull distribution based Symbolic Aggregate approXimation. REWS’s utility is demonstrated on the Commercial Modular Aero-Propulsion System Simulation dataset which describes the life cycle observations of multiple aircraft engines.

A multimodal and hybrid deep neural network model for Remaining Useful Life estimation

Aging critical infrastructures and valuable machineries together with recent catastrophic incidents such as the collapse of Morandi bridge calls for an urgent quest to design advanced and innovative data-driven solutions and efficiently incorporate multi-sensor streaming data sources for condition-based maintenance. Remaining Useful Life (RUL) is a crucial measure used in this regard within manufacturing and industrial systems, and its accurate estimation enables improved decision-making for operations and maintenance. Capitalizing on the recent success of multiple-model (also referred to as hybrid or mixture of experts) deep learning techniques, the paper proposes a hybrid deep neural network framework for RUL estimation, referred to as the Hybrid Deep Neural Network Model (HDNN). The proposed HDNN framework is the first hybrid deep neural network model designed for RUL estimation that integrates two deep learning models simultaneously and in a parallel fashion. More specifically, in contrary to the majority of existing data-driven prognostic approaches for RUL estimation, which are developed based on a single deep model and can hardly maintain good generalization performance across various prognostic scenarios, the proposed HDNN framework consists of two parallel paths (one LSTM and one CNN) followed by a fully connected multilayer fusion neural network which acts as the fusion centre combining the output of the two paths to form the target RUL. The HDNN uses the LSTM path to extract temporal features while simultaneously the CNN is utilized to extract spatial features. The proposed HDNN framework is tested on the NASA commercial modular aero-propulsion system simulation (C-MAPSS) dataset. Our comprehensive experiments and comparisons with several recently proposed RUL estimation methodologies developed based on the same data-sets show that the proposed HDNN framework significantly outperforms all its counterparts in the complicated prognostic scenarios with increased number of operating conditions and fault modes.

Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process

Remaining useful life (RUL) prediction is a key process for prognostics and health management (PHM). However, conventional model-based methods and data-driven methods for RUL prediction are bad at a very complex system with multiple components, multiple states and therefore extremely large amount of parameters. In order to solve the problem, a general two-step solution is proposed in this paper. In the first step, kernel principle component analysis (KPCA) is applied for nonlinear feature extraction. Then, a novel recurrent neural network called gated recurrent unit (GRU) is presented as the second step to predict RUL. GRU network is capable of describing a very complex system because of its specially designed structure. The effectiveness of the proposed solution for RUL prediction of a nonlinear degradation process is proved by a case study of commercial modular aero-propulsion system simulation data (C-MAPSS-Data) from NASA. Results also show that the proposed method requires less training time and has better prediction accuracy than other data-driven methods.

Degradation Modeling and Remaining Useful Life Prediction of Aircraft Engines Using Ensemble Learning

Degradation modeling and prediction of remaining useful life (RUL) are crucial to prognostics and health management of aircraft engines. While model-based methods have been introduced to predict the RUL of aircraft engines, little research has been reported on estimating the RUL of aircraft engines using novel data-driven predictive modeling methods. The objective of this study is to introduce an ensemble learning-based prognostic approach to modeling an exponential degradation process due to wear as well as predicting the RUL of aircraft engines. The ensemble learning algorithm combines multiple base learners, including random forests (RFs), classification and regression tree (CART), recurrent neural networks (RNN), autoregressive (AR) model, adaptive network-based fuzzy inference system (ANFIS), relevance vector machine (RVM), and elastic net (EN), to achieve better predictive performance. The particle swarm optimization (PSO) and sequential quadratic optimization (SQP) methods are used to determine optimum weights that are assigned to the base learners. The predictive model trained by the ensemble learning algorithm is demonstrated on the data generated by the commercial modular aero-propulsion system simulation (C-MAPSS) tool. Experimental results have shown that the ensemble learning algorithm predicts the RUL of the aircraft engines with considerable robustness as well as outperforms other prognostic methods reported in the literature.