Tennessee Eastman Chemical Process Research Articles

Fault detection of complicated processes based on an enhanced transformer network with graph attention mechanism

Recently, deep learning becomes increasingly popular in the industrial data analysis field due to its distinguished feature representation capability. As an emerging deep learning technology, transformer network has attracted extensive attention, but its application in industrial fault detection is still not sufficiently exploited. Furthermore, the traditional transformer network mainly focuses on the time series information of the data, while ignoring the spatial characteristic of the data. For this problem, an improved transformer network model with graph attention mechanism (GA-Tran) is proposed and used for complicated process fault detection. Considering the strong coupling property of process variables, the presented method performs joint spatial-temporal learning by mining the data information in views of both time-dimension and variable-dimension. On the one hand, a transformer encoder module equipped with a self-attention mechanism is utilized to capture long-term temporal dependencies, while multi-scale convolution is integrated to abstract short-term time dependencies. On the other hand, a graph attention network is introduced into the transformer network to meticulously analyze the spatial relationships among variables. Considering the topology of the industrial process variables, we use spectral clustering to infer prior topology information, and remove intelligently irrelevant spatial information. Different from the existing transformer models with the anomaly scores as the monitoring indicator, two monitoring statistics based on encoder features and reconstruction errors are constructed for process status monitoring. Studies on the Tennessee Eastman chemical process demonstrate the effectiveness of the proposed GA-Tran method.

Process monitoring and fault diagnosis method combining bagging DPCA‐ICA with moving window Kolmogorov–Smirnov test

AbstractProcess monitoring and fault diagnosis systems are essential for ensuring the quality and safety of industrial processes. Real industrial process data always has dynamic features, autocorrelation features, and non‐Gaussian distributions, making many multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA), not work well. To solve the problem, a new process monitoring method (named DPCA‐ICA‐moving window Kolmogorov–Smirnov test [MWKS]) is proposed in the paper, which is a combination of DPCA and independent component analysis (ICA) with adaptive weight parameters and introduces a moving window technique and two‐sample K‐S test to obtain the K‐S distance of the commonly used Hotelling's and square predictive error as new monitoring parameters. Experimental results on the Tennessee Eastman chemical process demonstrate the effectiveness and accuracy of the proposed method, which shows it has better performance than existing commonly used methods. The average fault detection rate (FDR) is 93.131%, while the average false alarm rate (FAR) of the proposed method is only 0.1%.

A robust neural network model for fault detection in the presence of mislabelled data

AbstractSeveral data‐driven methodologies for process monitoring and detection of faults or abnormalities have been developed for the safety of processing systems. The effectiveness of data‐based models, however, is impacted by the volume and quality of training data. This work presents a robust neural network model for addressing the mislabelled and low‐quality data in detecting faults and process abnormalities. The approach is based on harnessing data quality features along with supervisory labels in the network training. The data quality has been computed using the Mahalanobis distances and trusted centres of each class of data such as normal and faulty data. The method has been examined for detecting abnormalities in two case studies; a continuous stirred tank heater problem for detecting leaks and the Tennessee Eastman chemical process for detecting step and sticking faults. The performance of the proposed robust artificial neural networks (ANN) model is evaluated in terms of accuracy, fault detection rate, false alarm rate, and classification index at varying extents of mislabelling, namely, 1%, 5%, and 10% mislabelled data. The proposed model demonstrates higher detection performance, especially at increased labels of mislabelled data where the performance of the conventional ANN is severely impacted. The proposed methodology can be advantageous in handling mislabelled and low‐quality data issues which are crucial in the data‐driven modelling of processing systems.

PSINDy: Probabilistic sparse identification of nonlinear dynamics for temporal process modeling and fault detection

BackgroundMultivariate statistical process monitoring (MSPM) methods have been widely studied and applied for dynamic process monitoring. However, limitations exist in the extant literature, as numerous extant MSPM approaches fail to concurrently account for the governing dynamics and inherent non-linearity of systems. MethodsTo handle the above-mentioned issue, a novel probabilistic dynamic process monitoring algorithm named probabilistic sparse identification of nonlinear dynamics (PSINDy) is proposed. In this algorithm, the Expectation–Maximization (EM) method is employed to estimate the parameters. In the E-step, the particle filtering technique is adopted to calculate corresponding latent expectations whose posterior distributions are not Gaussian. Furthermore, Hotelling's T2 and two novel monitoring statistics, termed Mahalanobis-based predictive error (MPE) and dynamic predictive error (DPE), are designed and utilized for fault detection. Significant FindingsThe effectiveness of the proposed method is validated through the Tennessee Eastman (TE) chemical process and the three-phase flow facility (TPFF).

Improving Accuracy and Interpretability of CNN-Based Fault Diagnosis through an Attention Mechanism

This study aims to enhance the accuracy and interpretability of fault diagnosis. To address this objective, we present a novel attention-based CNN method that leverages image-like data generated from multivariate time series using a sliding window processing technique. By representing time series data in an image-like format, the spatiotemporal dependencies inherent in the raw data are effectively captured, which allows CNNs to extract more comprehensive fault features, consequently enhancing the accuracy of fault diagnosis. Moreover, the proposed method incorporates a form of prior knowledge concerning category-attribute correlations into CNNs through the utilization of an attention mechanism. Under the guidance of thisprior knowledge, the proposed method enables the extraction of accurate and predictive features. Importantly, these extracted features are anticipated to retain the interpretability of the prior knowledge. The effectiveness of the proposed method is verified on the Tennessee Eastman chemical process dataset. The results show that proposed method achieved a fault diagnosis accuracy of 98.46%, which is significantly higher than similar existing methods. Furthermore, the robustness of the proposed method is analyzed by sensitivity analysis on hyperparameters, and the interpretability is revealed by visually analyzing its feature extraction process.

An innovative approach based on meta-learning for real-time modal fault diagnosis with small sample learning

The actual multimodal process data usually exhibit non-linear time correlation and non-Gaussian distribution accompanied by new modes. Existing fault diagnosis methods have difficulty adapting to the complex nature of new modalities and are unable to train models based on small samples. Therefore, this paper proposes a new modal fault diagnosis method based on meta-learning (ML) and neural architecture search (NAS), MetaNAS. Specifically, the best performing network model of the existing modal is first automatically obtained using NAS, and then, the fault diagnosis model design is learned from the NAS of the existing model using ML. Finally, when generating new modalities, the gradient is updated based on the learned design experience, i.e., new modal fault diagnosis models are quickly generated under small sample conditions. The effectiveness and feasibility of the proposed method are fully verified by the numerical system and simulation experiments of the Tennessee Eastman (TE) chemical process.

Detecting Plant-Wide Oscillation Propagation Effects of Disturbances and Faults in a Chemical Process Plant Using Network Topology of Variance Decompositions

This work demonstrates for the first time the application of network topology of variance decompositions in analyzing the connectedness of chemical plant process variable oscillations arising from disturbances and faults. Specifically, the time-based connectedness and frequency-based connectedness of variables can be used to compute the net pairwise dynamic connectedness (NPDC), which originated as a volatility spillover index for financial markets studies in the field of econometrics. This work used the anomaly-detection benchmark Tennessee-Eastman chemical process (TEP) dataset, which consists of 41 measured variables and 11 manipulated variables subjected to various faulty operating conditions. The data analytics was performed using key functions from the R-package ‘ConnectednessApproach’ that implements connectedness computations based on time and frequency. The NPDC coefficient matrices were then transformed into network adjacency matrices for the rendering of the network topology of connectedness for TEP. The resulting network topologies allow a comprehensive analysis of oscillation effects across all plant-measured and manipulated variables. Analyzing the directed connectedness of the system dynamics at short-range, mid-range, and long-range frequencies showed how the oscillation effects of disturbances and faults propagate and dissipate in the short-term, mid-term, and long-term periods.

Fault detection and diagnosis strategy based on k-nearest neighbors and fuzzy C-means clustering algorithm for industrial processes

Fault detection and diagnosis is crucial in recent industry sector to ensure safety and reliability, and improve the overall equipment efficiency. Moreover, fault detection and diagnosis based on k-nearest neighbor rule (FDD-kNN) has been effectively applied in industrial processes with characteristics such as multi-mode, non-linearity, and non-Gaussian distributed data. The main challenge associated with FDD-kNN is the on-line computational complexity and storage space that are needed for searching neighbors. To deal with these issues, this paper proposes a monitoring approach where the Fuzzy C-Means clustering technique is used to decrease the overall on-line computations and required storage by measuring the neighbors of the clusters’ centres rather than the raw data. After building the monitoring model off-line based on the data clusters’ centres, the faults are detected by comparing the average squared Euclidean distance between the on-line data sample and the clusters’ centres with a predefined threshold. Then, the detected faults can be diagnosed by calculating the contribution of each variable in the fault detection index. Furthermore, for easily analysing the fault diagnosis results, the relative contribution for each sample data vector is considered. A numerical example and the Tennessee Eastman chemical process are used to demonstrate the performance of the proposed FCM-kNN for fault detection and diagnosis.

Fault Diagnosis of Tennessee Eastman Process with XGB-AVSSA-KELM Algorithm

In fault detection and the diagnosis of large industrial systems, whose chemical processes usually exhibit complex, high-dimensional, time-varying and non-Gaussian characteristics, the classification accuracy of traditional methods is low. In this paper, a kernel limit learning machine (KELM) based on an adaptive variation sparrow search algorithm (AVSSA) is proposed. Firstly, the dataset is optimized by removing redundant features using the eXtreme Gradient Boosting (XGBOOST) model. Secondly, a new optimization algorithm, AVSSA, is proposed to automatically adjust the network hyperparameters of KELM to improve the performance of the fault classifier. Finally, the optimized feature sequences are fed into the proposed classifier to obtain the final diagnosis results. The Tennessee Eastman (TE) chemical process is used to verify the effectiveness of the proposed method through multidimensional diagnostic metrics. The results show that our proposed diagnosis method can significantly improve the accuracy of TE process fault diagnosis compared with traditional optimization algorithms. The average diagnosis rate for 21 faults was 91.00%.

A Novel Fault Diagnosis Method for TE Process Based on Optimal Extreme Learning Machine

Chemical processes usually exhibit complex, high-dimensional and non-Gaussian characteristics, and the diagnosis of faults in chemical processes is particularly important. To address this problem, this paper proposes a novel fault diagnosis method based on the Bernoulli shift coyote optimization algorithm (BCOA) to optimize the kernel extreme learning machine classifier (KELM). Firstly, the random forest treebagger (RFtb) is used to select the features, and the data set is optimized. Secondly, a new optimization algorithm BCOA is proposed to automatically adjust the network hyperparameters of KELM and improve the classifier performance. Finally, the optimized feature sequence is input into the proposed classifier to obtain the final diagnosis results. The Tennessee Eastman (TE) chemical process have been collected and used to verify the effectiveness of the proposed method. A comprehensive comparison and analysis with widely used algorithms is also performed. The results demonstrate that the proposed method outperforms other methods in terms of classification accuracy. The average diagnosis rate of 21 faults is found to be 89.32%.

An integrated methodology for fault detection, root cause diagnosis, and propagation pathway analysis in chemical process systems

Although the current research on fault detection and diagnosis (FDD) has made significant developments, there exists some challenges such as earlier fault detection with lower false alarms, accurate fault diagnosis without prior fault information, and construction of fault propagation pathway. Many articles have contributed to address one or more aforementioned challenges. However, very few of them has attempted to capture these research gaps in a unified framework. Hence, the current article presents an integrated method for process FDD and propagation pathway identification. The methodology is developed combining the early fault detection capacity of the multivariate exponentially weighted moving average principal component analysis (MEMWA-PCA) and robust diagnosis and causality representability of the Bayesian network (BN). MEWMA-PCA can generally detect the fault earlier than PCA with a lower number of false alarms. Multiple likelihood evidence-based BN updating is utilized to ensure accurate diagnosis without in-depth fault information; this BN updating mechanism is found to be superior to the conventional hard evidence-based updating technique. Additionally, for taking corrective actions effectively, fault propagation pathway can be generated using BN. The proposed method is applied to a total of ten fault cases in the benchmark Tennessee Eastman chemical process. It is observed that the developed method is robust and comprehensive compared to the existing PCA and PCA-BN-based hybrid methods.

Integrating feature optimization using a dynamic convolutional neural network for chemical process supervised fault classification

Chemical processes usually exhibit complex, high-dimensional, time-varying, and non-Gaussian characteristics, and the diagnosis of faults in chemical processes is particularly important. However, many current fault diagnosis methods do not consider the temporal correlation of process data, feature selection, and feature sequence arrangement. To solve this problem, this paper presents a fault diagnosis method using a dynamic convolutional neural network, based on a genetic algorithm (GA), for optimizing a feature sequence. First, the input data are transformed into a two-dimensional matrix by adding the dimension of time characteristics. Second, the GA is used to select the features, and the sequence of the selected features is optimized. Finally, the optimized feature sequence is input into the convolutional neural network (CNN) to obtain the final diagnosis results. The Tennessee Eastman chemical process is used for experimental analysis, and the proposed model is compared with the weighted cascade forest, deep belief network (DBN), optimized DBN, long short-term memory + CNN and feature selection using random forest models. The experimental results show that the proposed model has higher diagnostic accuracy. The average diagnosis rate of 20 faults is found to be 89.72%.

Risk-based fault detection and diagnosis for nonlinear and non-Gaussian process systems using R-vine copula

This paper presents a risk-based fault detection and diagnosis methodology for nonlinear and non-Gaussian process systems using the R-vine copula and the event tree. The R-vine model provides a multivariate probability that is used in the event tree to generate a dynamic risk profile. An abnormal situation is detected from the monitored risk profile; subsequently, root cause(s) diagnosis is carried out. A fault diagnosis module is also proposed using the density quantiles, developed from marginal probabilities. The performance of this methodology is benchmarked using the Tennessee Eastman chemical process. The proposed risk-based framework has also been applied to an experimental setup and a real industrial isomer separator unit. The diagnosis module is found sensitive to both single and simultaneous faults. The results confirm that the proposed methodology provides better performance than the conventional principal component analysis and transfer entropy-based fault diagnosis techniques using the advantage of marginal density quantile analysis.

A novel fault diagnosis method based on CNN and LSTM and its application in fault diagnosis for complex systems

Fault diagnosis plays an important role in actual production activities. As large amounts of data can be collected efficiently and economically, data-driven methods based on deep learning have achieved remarkable results of fault diagnosis of complex systems due to their superiority in feature extraction. However, existing techniques rarely consider time delay of occurrence of faults, which affects the performance of fault diagnosis. In this paper, by synthetically considering feature extraction and time delay of occurrence of faults, we propose a novel fault diagnosis method that consists of two parts, namely, sliding window processing and CNN-LSTM model based on a combination of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM). Firstly, samples obtained from multivariate time series by the sliding window processing integrates feature information and time delay information. Then, the obtained samples are fed into the proposed CNN-LSTM model including CNN layers and LSTM layers. The CNN layers perform feature learning without relying on prior knowledge. Time delay information is captured with the use of the LSTM layers. The fault diagnosis of the Tennessee Eastman chemical process is addressed, and it is verified that the predictive accuracy and noise sensitivity of fault diagnosis can be greatly improved when the proposed method is applied. Comparisons with five existing fault diagnosis methods show the superiority of the proposed method.

Nonparametric manifold learning approach for improved process monitoring

AbstractA novel nonparametric method based on manifold learning is proposed for industrial process monitoring. In conventional algorithms, to preserve the global and local structure information of data, heat kernels containing two auxiliary parameters are introduced to define the global and local weight matrices, respectively. However, it is difficult to identify and choose these two parameters empirically. The inadequate selection of parameters can lead to one‐sided and inappropriate global and local feature extractions, resulting in an inadequate fault detection performance. To resolve the above problems, a nonparametric strategy is used in this study to generate two nonparametric weight matrices to replace the heat kernel‐based weight matrices. Consequently, the proposed method requires no auxiliary parameters in defining the weight matrices, making it more practical. Moreover, it automatically determines a good trade‐off between global and local feature extractions. A process monitoring model based on the proposed method was developed. The feasibility and effectiveness of the new nonparametric method are evaluated using a synthetic example and the Tennessee Eastman chemical process.

A Novel Ensemble Adaptive Sparse Bayesian Transfer Learning Machine for Nonlinear Large-Scale Process Monitoring.

Process monitoring plays an important role in ensuring the safety and stable operation of equipment in a large-scale process. This paper proposes a novel data-driven process monitoring framework, termed the ensemble adaptive sparse Bayesian transfer learning machine (EAdspB-TLM), for nonlinear fault diagnosis. The proposed framework has the following advantages: Firstly, the probabilistic relevance vector machine (PrRVM) under Bayesian framework is re-derived so that it can be used to forecast the plant operating conditions. Secondly, we extend the PrRVM method and assimilate transfer learning into the sparse Bayesian learning framework to provide it with the transferring ability. Thirdly, the source domain (SD) data are re-enabled to alleviate the issue of insufficient training data. Finally, the proposed EAdspB-TLM framework was effectively applied to monitor a real wastewater treatment process (WWTP) and a Tennessee Eastman chemical process (TECP). The results further demonstrate that the proposed method is feasible.

An optimized long short-term memory network based fault diagnosis model for chemical processes

With the development of the chemical industry, fault diagnosis of chemical processes has become a challenging problem because of the high-dimensional data and complex time correlation caused by the more complex chemical processes and increasing number of equipment. However, the ordinary feedforward neural network cannot solve these problems very well. Therefore, this paper proposes a fault diagnosis model based on the optimized long short-term memory (LSTM) network. Since the number of hidden layer nodes in the LSTM network has a great influence on the diagnosis result, the link of determining the optimal number of hidden layer nodes by the iterative method based on the LSTM network is added. Then the LSTM is optimized to get higher chemical process fault diagnosis accuracy. Finally, through the simulation experiment of the Tennessee Eastman (TE) chemical process, the results verify that the optimized LSTM network has better performance in chemical process fault diagnosis than the BP neural network, the multi-layer perceptron method and the original LSTM network.

A Multichannel LSTM-CNN Method for Fault Diagnosis of Chemical Process

The identification and classification of faults in chemical processes can provide decision basis for equipment maintenance personnel to ensure the safe operation of the production process. In this paper, we combine long short-term memory neural network (LSTM) with convolutional neural network (CNN) and propose a new fault diagnosis method based on multichannel LSTM-CNN (MCLSTM-CNN). The primary methodology here includes three aspects. In the initial state, the fault data are input into the LSTM to obtain the output of the hidden layer, which stores the relevant temporal and spatial domain information. Due to the diversity of data features, convolutional kernels with different sizes are utilized to form multiple channels to extract the output characteristics of the hidden layer simultaneously. Finally, the fault data are classified by fully connected layers. The Tennessee Eastman (TE) chemical process is used for experimental analysis, and the MCLSTM-CNN model is compared with the LSTM-CNN, LSTM, CNN, RF and KPCA + SVM models. The experimental results show that the MCLSTM-CNN model has higher diagnostic accuracy, and the fault classification results are superior to other models.

Modeling the Tennessee Eastman chemical process reactor using bio-inspired feedforward neural network (BI-FF-NN)

This study explores the application of bio-inspired algorithms (BIAs) in training artificial neural networks (ANNs) in the area of the nonlinear chemical process modeling. Motivated by the increasing complexity and operational efficiency of chemical processes, the need for schemes that can improve model identification of highly nonlinear systems is demanded. As a case study, the Tennessee Eastman (TE) chemical reactor problem is considered. The key issue is to find a particular architecture for an ANN that best model the TE chemical process. We propose the use of BIAs of bat algorithm (BA), firefly algorithm (FA), and artificial bee colony (ABC) algorithm as mechanisms to automatically update the synaptic weights of ANN. These algorithms were conducted to increase the ability of ANN to adapt the dynamic aspect of the TE process and prevent trapping into a local optimum. The proposed modeling framework was devised with extensive experiments and statistical analysis to illustrate the usability and suitability of the entire modeling and identification procedures. The results were explored with a discussion using the mean square error (MSE) and the variance account for (VAF) criteria to assess the degree of identification performance of the TE case study model. The reliability of the presented models was compared with previously developed models for the same subsystems of the TE reactor using competitive, intelligent approaches. The comparative results show that the proposed approach provides superior modeling performance and outperforms its competitors.

Process system fault detection and diagnosis using a hybrid technique

This paper presents a hybrid methodology to detect and diagnose the faults in dynamic processes based on principal component analysis (PCA) with T2 statistics and a Bayesian network (BN). It deals with the uncertainty generated by the multivariate contribution plots and improves the diagnostic capacity by updating the BN with multiple likelihood evidence. It can diagnose the root cause of the process fault precisely as well as identify the fault propagation pathway. This methodology has been applied to the continuous stirred tank heater and the Tennessee Eastman chemical process for twelve fault scenarios. The result shows that it provides better diagnostic performance over conventional principal component analysis with hard evidence-based approaches.