Tennessee Eastman Benchmark Process Research Articles

Weighted random forests for fault classification in industrial processes with hierarchical clustering model selection

In this paper, a hierarchical clustering selection based weighted random forests scheme is proposed for fault classification in complex industrial processes. Model diversity and the strength of each model are deemed to be two key issues for the performance of ensemble learning method. To improve the diversity between classification trees and the performance of individual classification trees in random forests, the hierarchical clustering method is applied for offline model selection in random forests, which can simultaneously reduce the online fault classification complexity. Meanwhile, the weighted voting rule is used in random forests instead of majority voting, in order to boost the good performance models and weaken the bad ones. Detailed comparative studies between proposed method and conventional methods have been carried out through the Tennessee Eastman (TE) benchmark process.

Automated feature learning for nonlinear process monitoring – An approach using stacked denoising autoencoder and k-nearest neighbor rule

Modern industrial processes have become increasingly complicated, consequently, the nonlinearity of data collected from these systems continues to increase. However, the feature extraction methods of existing process monitoring are not capable of extracting crucial features from these highly nonlinear data, which affects the performance of monitoring. In this paper, a novel nonlinear process monitoring method based on stacked denoising autoencoder (SDAE) and k-nearest neighbor (kNN) rule is proposed. Specifically, stacked denoising autoencoder is utilized to model the nonlinear process data and automatically extract crucial features. The original nonlinear space is then mapped to the feature space and the residual space via SDAE. Two new statistics in the above spaces are constructed by introducing the kNN rule with their corresponding control limits determined by kernel density estimation. Case studies on a nonlinear numerical system and the Tennessee Eastman benchmark process verify the effectiveness of the proposed method.

Neural component analysis for fault detection

Principal component analysis (PCA) is largely adopted for chemical process monitoring and numerous PCA-based systems have been developed to solve various fault detection and diagnosis problems. Since PCA-based methods assume that the monitored process is linear, nonlinear PCA models, such as autoencoder models and kernel principal component analysis (KPCA), has been proposed and applied to nonlinear process monitoring. However, KPCA-based methods need to perform eigen-decomposition (ED) on the kernel Gram matrix whose dimensions depend on the number of training data. Moreover, prefixed kernel parameters cannot be most effective for different faults which may need different parameters to maximize their respective detection performances. Autoencoder models lack the consideration of orthogonal constraints which is crucial for PCA-based algorithms. To address these problems, this paper proposes a novel nonlinear method, called neural component analysis (NCA), which intends to train a feedforward neural work with orthogonal constraints such as those used in PCA. NCA is a unified model including a nonlinear encoder and a linear decoder. NCA can adaptively learn its parameters through backpropagation and the dimensionality of the nonlinear features has no relationship with the number of training samples. Extensive experimental results on the Tennessee Eastman (TE) benchmark process show the superiority of NCA in terms of missed detection rate (MDR) and false alarm rate (FAR). The source code of NCA can be found in https://github.com/haitaozhao/Neural-Component-Analysis.git.

Recursive Spectral Meta-Learner for Online Combining Different Fault Classifiers

This paper considers the problem of fault classification when different fault classifiers are performed simultaneously. Based on spectral meta-learner (SML) proposed by Parisi et al. , its recursive version, i.e., recursive SML (RSML) is developed for online combining the potentially conflicting classification information. Considering different statistical properties of faults occurring at different time intervals, the binary classification information is recursively utilized. By introducing a forgetting factor, the leading eigenvector of the estimate of the time-varying covariance matrix is used as the weight vector for each classifier. Rank-one modification is then used for calculating the eigenvector in order to reduce the online computational complexity. The performance of RSML is strictly analyzed in a statistical sense, including the effect of recursive calculation and conditional dependence of different classifiers on the weight vector. Compared with majority voting and SML, a higher balanced accuracy of RSML can be verified by the benchmark Tennessee Eastman process.

Ensemble learning model based on selected diverse principal component analysis models for process monitoring

AbstractPrincipal component analysis (PCA) is extensively applied in industrial process monitoring. For optimal performance monitoring, different faults may require different principal components (PCs). However, at present, it only selects PCs of the highest variance to create a single PCA model, thereby leading to information loss and poor monitoring performance. For the solution of this problem, a method based on ensemble learning and Bayesian inference is presented in this paper. First, numerous models are generated according to the randomly selected PCs. Next, the model with the lowest false alarm rate is retained to ensure good model performance. A novel pruning algorithm is then employed to obtain several models comprising great difference (“great difference” means the smaller similarity of the selected PCs when building different PCA models). This method enables the identification of models that can effectively predict various faults, thereby improving the monitoring performance of the ensemble model. Bayesian inference is adopted to determine the final monitoring indicator. Finally, a numerical example was used, and the Tennessee Eastman benchmark process was applied to evaluate monitoring effectiveness and illustrate the excellent performance of ensemble learning and Bayesian inference.

Recursive Dynamic Transformed Component Statistical Analysis for Fault Detection in Dynamic Processes

This paper considers the problem of dynamic process monitoring. Based on the recently proposed recursive transformed component statistical analysis (RTCSA), its dynamic counterpart recursive dynamic transformed component statistical analysis (RDTCSA) is proposed. With time lag shift technique, the augmented sample covariance matrices are used for eigendecomposition and further data transformation. The obtained dynamic transformed components include dynamic information of measurements, whose statistics are used for process monitoring. The difference between RTCSA and RDTCSA for monitoring time-correlated process data is analyzed, which implies that RDTCSA is more sensitive to dynamic changes. In addition, the detectability of RDTCSA for monitoring time-correlated process data is analyzed in a statistical sense. A numerical simulation and the benchmark Tennessee Eastman process (TEP) both indicate the superior fault detectability of RDTCSA compared with the existing methods. Specifically, RDTCSA can effectively detect fault 15 in TEP with detection rate over 95%.

Mixture Semisupervised Bayesian Principal Component Regression for Soft Sensor Modeling

In this paper, a mixture semisupervised Bayesian principal component regression-based soft sensor modeling method for nonlinear industrial process with multiple operating modes is presented. In many chemistry processes, part of output data samples may be unavailable due to the difficulties in measurement or recording. The semisupervised method is introduced to efficiently deal with the unlabeled data set. Moreover, the Bayesian regularization method is proposed to determine the unknown dimensionality of latent variables space in each submode by introducing three different formulations of two hyperparameters to construct the Gaussian prior distributions over the loading and regression matrices. The formulation of this method is derived in expectation maximization algorithm scheme, and the formulas to update unknown parameters are derived. The effectiveness of the proposed method is verified through a numerical example, the Tennessee Eastman benchmark process, and the comparisons with the existing method.

Sequential Fault Diagnosis Based on LSTM Neural Network

Fault diagnosis of chemical process data becomes one of the most important directions in research and practice. Conventional fault diagnosis and classification methods first extract features from the raw process data. Then certain classifiers are adopted to make diagnosis. However, these conventional methods suffer from the expertise of feature extraction and classifier design. They also lack the adaptive processing of the dynamic information in raw data. This paper proposes a fault diagnosis method based on long short-term memory (LSTM) neural network. The novel method can directly classify the raw process data without specific feature extraction and classifier design. It is also able to adaptively learn the dynamic information in raw data. First, raw process data are used to train the LSTM neural network until the cost function of LSTM converges below certain predefined small positive value. In this step, the dynamic information of raw process data is adaptively learned by LSTM. Then testing data are used to obtain the diagnosis results of the trained LSTM neural network. The application of LSTM to fault identification and analysis is evaluated in the Tennessee Eastman benchmark process. Extensive experimental results show LSTM can better separate different faults and provide more promising fault diagnosis performance.

Optimal Variable Transmission for Distributed Local Fault Detection Incorporating RA and Evolutionary Optimization

Determining the variable transmission structure is the key step in designing a distributed monitoring scheme for multiunit processes. This paper proposes randomized algorithm (RA) integrated with evolutionary optimization-based data-driven distributed local fault detection scheme to achieve efficient monitoring of multiunit chemical processes. First, the RA is employed to generate faulty validation data. Second, evolutionary optimization-based variable transmission structure determination is performed to achieve the minimal non-detection rate by selecting transferred variables. Then, a principal component analysis (PCA) or kernel PCA monitoring model is established for each operation unit to identify the status of the unit. Last, a comprehensive index is developed to identify the status of the entire process. The established local monitors consider the relationship with other units but avoid introducing redundant information, thereby exhibiting superior monitoring performance. Case studies on two numerical examples, including a linear and a nonlinear case and the Tennessee Eastman benchmark process, are provided. Comparative results of traditional local or global monitoring methods verify the efficiency of the proposed monitoring scheme.

A Systematic Approach to Dynamic Monitoring of Industrial Processes Based on Second-Order Slow Feature Analysis

Slow feature analysis has proven to be an effective process monitoring and fault diagnosis approach. By isolating temporal behaviors from steady-state variations in process data, slow feature analysis enables a concurrent monitoring of operating condition and process dynamics, based on which false alarms triggered by nominal operating condition deviations can be effectively removed. However, the present formulation of slow feature analysis only makes use of the first-order time difference of time series data, thereby falling short of addressing high-order dynamics in process operations. In this work, we propose a second-order formulation of slow feature analysis, and further develop a systematic framework for process monitoring and fault diagnosis, which can provide more meaningful information about process dynamics to assist decision-making of operators. Case studies on the Tennessee Eastman benchmark process are conducted to demonstrate the efficacy of the proposed method.

A Comparison of Different Statistics for Detecting Multiplicative Faults in Multivariate Statistics-Based Fault Detection Approaches

The explosion of different fault detection (FD) statistics in multivariate statistics-based FD approaches has meant that the practitioner is faced with the unenviable job of determining which to use in a given circumstance. Moreover, compared to extensive investigations on additive faults, the performance of commonly used FD statistics for detecting multiplicative faults has not been holistically evaluated. Therefore, this paper seeks to investigate the different statistics that can be applied to detect multiplicative faults in order to provide users and practitioners in the FD field with guidance to select an appropriate method. The considered statistics are broadly classified into three groups: traditional methods ( e.g. $T^{2}$ -statistic) and their extensions; the Wishart distribution-based methods; and those methods created in the information and communication fields to describe the measurement variance and covariance ( e.g. Kullback-Leibler divergence). These three groups are compared by considering the required probability distributions, interconnections, and detection performance for multiplicative faults. Using simulated data from numerical examples and the Tennessee Eastman benchmark process, the theoretical results are validated, and the applicability of multivariate statistics-based FD methods incorporating all considered statistics for detecting multiplicative faults is examined at the end of this paper.

Dynamic CCA-Based Distributed Monitoring for Multiunit Non-Gaussian Processes

Multiunit non-Gaussian dynamic processes are more popular, and monitoring of such processes becomes important. A dynamic canonical correlation analysis (DCCA)-based distributed monitoring approach for multiunit non-Gaussian dynamic processes is proposed. First, DCCA is performed between each local unit and the coupled units, which models both the auto-correlation and cross-correlation among units. Second, two fault detection residuals are generated for each local unit, through which both the process status and the type of a detected fault are identified. The proposed DCCA monitoring scheme achieves local unit monitoring by considering the variables from both the local unit and the coupled units, and therefore exhibits superiority. Applications on the Tennessee Eastman benchmark process are provided, through which the efficiency of the DCCA monitoring method is demonstrated.

Change Point Detection using the Kantorovich Distance Algorithm

In this article, a novel change detection algorithm is proposed based on the Kantorovich distance concept. Incorporating the proposed change detection algorithm with the existing process monitoring tools may assist the operator in detecting dynamic changes in process plants and provide fewer unnecessary (false) alarms as well as fewer missed alarms. The proposed change detection method was tested through simulation data. It is applied to the benchmark Tennessee Eastman (TE) process in online mode. Results prove the efficacy of the proposed method to detect both the sharp and incipient changes.

Generative adversarial network based novelty detection usingminimized reconstruction error

Generative adversarial network (GAN) is the most exciting machine learning breakthrough in recent years, and it trains the learning model by finding the Nash equilibrium of a two-player zero-sum game. GAN is composed of a generator and a discriminator, both trained with the adversarial learning mechanism. In this paper, we introduce and investigate the use of GAN for novelty detection. In training, GAN learns from ordinary data. Then, using previously unknown data, the generator and the discriminator with the designed decision boundaries can both be used to separate novel patterns from ordinary patterns. The proposed GAN-based novelty detection method demonstrates a competitive performance on the MNIST digit database and the Tennessee Eastman (TE) benchmark process compared with the PCA-based novelty detection methods using Hotelling’s T2 and squared prediction error statistics.

Online Process Monitoring Using Recursive Mutual Information-Based Variable Selection and Dissimilarity Analysis With No Prior Information

The traditional dissimilarity (DISSIM) method explores the underlying fault characteristic based on the data distribution and is sensitive to the structure change of the process. However, it fails to explore which variables are significant to the concerned faults and its monitoring performance, including fault detection and diagnosis performance, is seriously decreased by the noise brought by non-informative variables, especially in plant-wide process. Since mutual information (MI) can explore both the linear and nonlinear dependencies of variables, a recursive MI-based variable selection algorithm is proposed in this paper. It can efficiently extract the most informative variables to the faults online and reduce the computational complexity. Then based on the variables selected online, the dissimilarity index is calculated to detect the distribution changes from normal to a fault condition and an MI-based diagnosis method is developed to further investigate the responsible variables to the fault. With the variable selection, not only the local characteristic of the process can be highlighted by the informative variables but also the influence of the non-informative variables can be eliminated, thus the control limit can be adaptively updated and the sensitivity and accuracy of the monitoring performance can be significantly improved. Moreover, the MI-based diagnosis method explores the contribution of selected variables with a high-order statistic and overcomes shortage brought by variable variance. Case study on Tennessee Eastman (TE) benchmark process demonstrates the feasibility and efficiency of our method.

An improved plant‐wide fault detection scheme based on PCA and adaptive threshold for reliable process monitoring: Application on the new revised model of Tennessee Eastman process

AbstractAn improved process monitoring scheme is presented in this paper, it is based on the integration of multivariate and univariate statistical analysis methods. Instead of conventional fixed control limits, adaptive thresholds are developed for common fault detection indices used with principal component analysis, including the Hotelling T2 statistic and the sum of squared prediction error known as the Q statistic. The thresholds are updated based on a modified exponentially weighted moving average chart with a limited window length. The primary goal of this strategy is to enhance the performance of principal component analysis–based process monitoring method and overcome its shortcomings, by increasing fault detection rate to improve monitoring sensitivity and eliminating false alarms to ensure higher robustness and reliability. Fault detection in the revised model of Tennessee Eastman process benchmark is also investigated. The developed monitoring scheme is tested and compared with conventional fixed threshold technique, and its performance is evaluated across various types of process faults. The obtained results demonstrate the promising capabilities of the developed scheme.

Evolving model identification for process monitoring and prediction of non-linear systems

This paper tackles the problem of model identification for monitoring of non-linear processes using evolving fuzzy models. To ensure a high production quality and to match the economic requirements, industrial processes are becoming increasingly complicated in both their structure and their degree of automation. Therefore, evolving systems, because of their data-driven and adaptive nature, appear to be a useful tool for modeling such complex and non-linear processes. In this paper the identification of evolving cloud-based fuzzy models is treated for process monitoring purposes. Moreover, the evolving part of the algorithm was improved with the inclusion of some new cloud-management mechanisms. To evaluate the proposed method two different processes, but both complex and non-linear, were used. The first one is a simulated Tennessee Eastman benchmark process model, while the second one is a real water-chiller plant.

Multi‐block principal component analysis based on variable weight information and its application to multivariate process monitoring

The traditional principal component analysis (PCA)‐based process monitoring method builds a global statistical model and omits the mining of the local variable behaviours, which may degrade the fault detection performance. Considering this problem, this paper proposes a variable weight information‐based multi‐block PCA (VWI‐MBPCA) method. Firstly, a sequence hierarchical clustering algorithm is proposed to divide the full PCA component space into several sub‐blocks, where the components sharing similar variable weight information are gathered together and then the sub‐block T2 statistic is constructed for monitoring sub‐block components. Further, the variables with small weight information on each component sub‐block are extracted to build an additional sub‐PCA model, where the statistic is developed to compensate the sub‐block T2 statistic. In order to integrate the monitoring results of each sub‐block, Bayesian inference is applied to construct an overall T2 statistic. To identify the faulty variables, a multi‐block PCA contribution plot is designed by choosing some specific blocks to highlight fault information. Finally, simulations on a numerical example and the benchmark Tennessee Eastman (TE) process are used to demonstrate the strengths of the proposed method.

Fault detection based on augmented kernel Mahalanobis distance for nonlinear dynamic processes

This paper presents a fault detection method based on augmented kernel Mahalanobis distance (AKMD) for monitoring nonlinear dynamic processes. In order to reflect the information of dynamic correlations, the measurements are stacked into augmented vectors at adjacent sampling instants. The augmented kernel Mahalanobis distance serves as the detection index, and its control limit is determined by the empirical method with assigning a significance level. Contrary to the mainstream of process monitoring methods based on principal component analysis (PCA), dimensionality reduction is not used here. The disadvantage of dimensionality reduction and space partition is discussed, and the improvement of fault detectability via data augmentation is analyzed. In addition, the computational complexity of the proposed method is acceptable. For training dataset containing m variables and n samples, if n≫m, the online computational burden of the proposed method is about O(n2). Simulations about a nonlinear dynamic process and the benchmark Tennessee Eastman process (TEP) both illustrate higher detection rates of the proposed method, compared with conventional multivariate statistical process monitoring (MSPM) methods such as PCA and its variants.

Data-driven root-cause fault diagnosis for multivariate non-linear processes

In a majority of multivariate processes, propagating nature of malfunctions makes the fault diagnosis a challenging task. This paper presents a novel data-driven strategy for real-time root-cause fault diagnosis in multivariate (non-)linear processes by estimating the strength of causality using normalized transfer entropy (NTE) between measured process variables and variations of a residual signal. In this paper, a new framework for root-cause fault diagnosis applicable for multivariate nonlinear processes is proposed, which can reduce the necessary number of calculation for causality analysis among time-series. More specially, a new and fast symbolic dynamic-based normalized transfer entropy (SDNTE) technique is proposed to enable real-time application of transfer entropy, which has been considered as a burdensome approach for causality analysis. The concept of SDNTE is built upon principles of time-series symbolization, xD-Markov machine and Shannon entropy. This paper also introduces a new concept of joint xD-Markov machine to capture dynamic interactions between two time-series. The proposed root-cause fault diagnosis framework is applied on Tennessee Eastman process benchmark and its computational advantages are shown by comparing with conventional kernel PDF-based method. Moreover, the proposed strategy is applied to health monitoring of a big scale industry centrifuge to corroborates its effectiveness and feasibility in industrial applications.