Multivariate Statistical Process Monitoring Research Articles

Variable contribution analysis in multivariate process monitoring using permutation entropy

Multivariate statistical process monitoring is widely used in industrial processes for performance monitoring and fault detection. Once a fault has been detected, fault diagnoses must be performed to identify the variables that are responsible for the fault or the deviation from normal operation. In this paper, we present a new data-driven method for variable contribution analysis to a fault or abnormal behaviour in a process. The method is based on permutation entropy (PE), which measures the order of fluctuations within a time series. The PE method for variable contribution analysis is independent of fault detection statistics, and is used to evaluate which variables experienced the greatest change in their expected behaviour over time, immediately after the fault has been identified. We show how PE can be used to identify and interpret the variables responsible for, or affected by faults in an industrial process. The well-known Tennessee Eastman Process (TEP) is used to illustrate the application of PE for variable contribution analysis. The results show that PE is an efficient analysis tool for specifying variable contributions to faults.

Process monitoring and fault diagnosis method combining bagging DPCA‐ICA with moving window Kolmogorov–Smirnov test

AbstractProcess monitoring and fault diagnosis systems are essential for ensuring the quality and safety of industrial processes. Real industrial process data always has dynamic features, autocorrelation features, and non‐Gaussian distributions, making many multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA), not work well. To solve the problem, a new process monitoring method (named DPCA‐ICA‐moving window Kolmogorov–Smirnov test [MWKS]) is proposed in the paper, which is a combination of DPCA and independent component analysis (ICA) with adaptive weight parameters and introduces a moving window technique and two‐sample K‐S test to obtain the K‐S distance of the commonly used Hotelling's and square predictive error as new monitoring parameters. Experimental results on the Tennessee Eastman chemical process demonstrate the effectiveness and accuracy of the proposed method, which shows it has better performance than existing commonly used methods. The average fault detection rate (FDR) is 93.131%, while the average false alarm rate (FAR) of the proposed method is only 0.1%.

A change-point–based control chart for detecting sparse mean changes in high-dimensional heteroscedastic data

Because of the “curse of dimensionality,” high-dimensional processes present challenges to traditional multivariate statistical process monitoring (SPM) techniques. In addition, the unknown underlying distribution of and complicated dependency among variables such as heteroscedasticity increase the uncertainty of estimated parameters and decrease the effectiveness of control charts. In addition, the requirement of sufficient reference samples limits the application of traditional charts in high-dimension, low-sample-size scenarios (small n, large p). More difficulties appear when detecting and diagnosing abnormal behaviors caused by a small set of variables (i.e., sparse changes). In this article, we propose two change-point–based control charts to detect sparse shifts in the mean vector of high-dimensional heteroscedastic processes. Our proposed methods can start monitoring when the number of observations is a lot smaller than the dimensionality. The simulation results show that the proposed methods are robust to nonnormality and heteroscedasticity. Two real data examples are used to illustrate the effectiveness of the proposed control charts in high-dimensional applications. The R codes are provided online.

A Relevant Variable Selection and SVDD-Based Fault Detection Method for Process Monitoring

This study investigates the sample value imbalance problem of process monitoring. A fault detection approach based on variable selection and support vector data description (SVDD) is developed for efficient process monitoring. First, Kullback&#x2013;Leibler divergence serves as the variable selection algorithm, which highlights the most beneficial information about the concerned faults. The attained variables are segmented by block division to avoid faults information being covered in single space monitoring, so that the relevant variables and the most beneficial information are concentrated in the same block. Then, Kernel principal component analysis is applied in each block to address the challenge that variables may still be high-dimensional and nonlinear. After that, the monitoring result is given based on the proposed SVDD with a restructured radius index, which is more sensitive to the fault. As demonstrated from experimental results on the Tennessee Eastman process, this method is effective and outperforms counterparts with higher mean fault detection rate. <i>Note to Practitioners</i>&#x2014;Recently, multivariate statistical process monitoring (MSPM) has attracted much attention. In general, MSPM incorporates all variables for the large-scale process. However, only a small number of variables are fault-dependent. Namely, the sample value imbalance problem is encountered in application. In this scenario, the monitoring performance degrades and the online computational complexity increases. To this end, a SVDD-based fault detection method, which considers the fault-related variables, is proposed for process monitoring. The proposed method is verified by the Tennessee Eastman process and it is more sensitive to the concerned fault.

Monitoring autocorrelated compositional data vectors using an enhanced residuals Hotelling [formula omitted] control chart

The most prevalent use of multivariate statistical process monitoring (SPM) is in the production industries. In SPM, data are usually assumed to be uncorrelated in both univariate and multivariate control charts. But, in the real industrial process, the data are often autocorrelated. Specifically, while using compositional data (CoDa), autocorrelation (AC) cannot be neglected, as the CoDa vectors have values close to one another. Hence the presence of AC in CoDa vectors is a very common phenomenon. The AC affects the control chart’s signal interpretation and decreases the detection capability. This paper uses a time series autoregressive moving average (ARMA) model on constrained CoDa based on an algebraic system in the simplex sample space (SD) without transformation. For the theoretical modeling foundation, the mathematical concepts in SD are presented. The AC and the partial AC functions of CoDa are defined to fit the CoDa ARMA model. The ordinary least square estimation is used to estimate the parameters of the CoDa ARMA model. After fitting the best CoDa ARMA model, a residual-based Hotelling T2 control chart for the CoDa ARMA model is used. The average run length of the proposed chart has been evaluated using the Monte Carlo simulations. Different levels of AC and sizes of the shift are considered. Two illustrative examples of sources of energy consumption in China and Japan are presented. The Hotelling T2 based on residuals of the CoDa ARMA model is better than the Hotelling T2 based on isometric log-ratio transformed original data to detect the shift in the mean vector.

A Holistic Evaluation of Multivariate Statistical Process Monitoring in a Biological and Membrane Treatment System.

Unsupervised process monitoring for fault detection and data cleaning is underdeveloped for municipal wastewater treatment plants (WWTPs) due to the complexity and volume of data produced by sensors, equipment, and control systems. The goal of this work is to extensively test and tune an unsupervised process monitoring method that can promptly identify faults in a full-scale decentralized WWTP prior to significant system changes. Adaptive dynamic principal component analysis (AD-PCA) is a dimension reduction method modified to address autocorrelation and nonstationarity in multivariate processes and is evaluated in this work for its ability to continuously detect drift, shift, and spike faults. For spike faults, univariate drift faults, and multivariate shift faults, implementing AD-PCA on data that are subset by treatment processes and operating states with significant differences in covariates and whose model parameters use week-long training windows, moderate cumulative variance, and a high threshold for detection was found to detect faults prior to existing operational thresholds. To improve the consistency with which the AD-PCA method detects out-of-control conditions in real time, additional work is needed to remove outliers prior to model fitting and to detect multivariate drift faults in which the covariates change slowly.

A complex-valued slow independent component analysis based incipient fault detection and diagnosis method with applications to wastewater treatment processes

Multivariate statistical process monitoring are the essential approaches to achieve better prognostics and health management (PHM) of process industries. However, incipient faults and complex behaviors (such as nonlinearity and dynamics) always render the traditional multivariate statistical process monitoring approaches inadequate. Thus, a complex-valued slow independent component analysis (CSICA) is proposed, which is able to extract optimized features from a complex-valued matrix containing both of raw data and their changing rates by resorting to a complex-valued independent component analysis operation and a batch of phase shifts. These features, named slow independent components (SICs), not only guarantee the statistical independence but also capture slowly-changing patterns, thus refining both dynamic and non-Gaussian information mostly related with incipient faults. The proposed algorithm together with novel statistics, Is2, If2 and SPE, as well as their control limits can sequentially detect incipient faults effectively. Then, together with the novel differential mapping reconstructed contribution plot (DM-RCP) and Granger causality analysis, the proposed method can accurately locate rooting causes of incipient faults. Finally, the proposed framework of process monitoring is validated through two data sets from a simulation platform and an oxidation-ditch-based wastewater treatment plant, respectively. The results demonstrate that the proposed method can achieve more accurate and efficient performances than conventional methods.

Survey on recursive statistical process monitoring methods

AbstractRecursive statistical process monitoring (RSPM) methods have superior monitoring performance for industrial processes, especially those with time‐varying characteristics, and have recently been studied by many researchers. However, there is no survey paper yet that summarizes and analyzes the existing RSPM methods. In this survey, approximately 60 papers related to RSPM methods are reviewed and categorized from different aspects. Existing RSPM methods are categorized using the classification criteria proposed in this study based on the ways of recursively collecting data and the stages of recursive updating. Three core algorithms that are most commonly used in RSPM methods are uniformly summarized and analyzed specifically for studying recursive computation problems in RSPM methods. Furthermore, different types of multivariate statistical process monitoring (MSPM) methods that combine recursive computation are reviewed.

Low-rank reconstruction-based autoencoder for robust fault detection

Autoencoder (AE) has been widely used in multivariate statistical process monitoring (MSPM) and various AE-based methods have been applied in fault detection. Process data is often contaminated by outliers and measurement noise which may lead to the overfitting problem for AE-based methods. In this paper, a novel feature extraction method called low-rank reconstruction-based autoencoder (LRAE) is proposed for robust fault detection. LRAE decomposes the input into a combination of a low-rank data matrix and a noise matrix. By penalizing the rank of the data matrix, LRAE separates the low-rank clean data from the contaminated process data. Instead of directly reconstructing the loss between the input data and the output data, we design a low-rank reconstruction strategy, i.e. reconstruct the loss between the low-rank clean data and the output of the AE. The proposed LRAE can be trained end-to-end by jointly optimizing an AE and a low-rank approximation. LRAE is a nonlinear method which can tackle the complicated process data better than the linear methods such as principal component analysis (PCA). Moreover, the optimization of the low-rank approximation provides the robustness of LRAE to reconstruct the clean data in the output layer when the input process data is contaminated. After training, the features of the hidden layer can be computed for further fault detection. Extensive experiments demonstrate that LRAE outperforms traditional fault detection methods, including PCA, robust principal component analysis (RPCA), kernel principal component analysis (KPCA), AE, and denoising autoencoder (DAE). Especially, LRAE provides more robust results when the process data suffer from outliers and measurement noise.

Fault Detection of Non-Gaussian and Nonlinear Processes Based on Independent Slow Feature Analysis.

Independent component analysis (ICA) is an excellent latent variables (LVs) extraction method that can maximize the non-Gaussianity between LVs to extract statistically independent latent variables and which has been widely used in multivariate statistical process monitoring (MSPM). The underlying assumption of ICA is that the observation data are composed of linear combinations of LVs that are statistically independent. However, the assumption is invalid because the observation data are always derived from the nonlinear mixture of LVs due to the nonlinear characteristic in industrial processes. Under this circumstance, the ICA-based fault detection is unable to provide accurate detection for specific faults of industrial processes. Since the observation data come from the nonlinear mixing of LVs, this makes the observation data change faster than the intrinsic LVs on the time scale. The temporal slowness can be regarded as an additional criterion in the extraction of LVs. The slow feature analysis (SFA) derived from the temporal slowness has received extensive attention and application in MSPM in recent years. Simultaneously, the temporal slowness is expected to make up for the problem that the LVs extracted by ICA have difficulty accurately describing the characteristics of the process. To solve the above problems, this work proposes to monitor non-Gaussian and nonlinear processes using the independent slow feature analysis (ISFA) that combines statistical independence and temporal slowness in extracting the LVs. When the observation data are composed of a nonlinear mixture of LVs, the extracted LVs of ISFA can describe the characteristics of the processes better than ICA, thereby improving the accuracy of fault detection for the non-Gaussian and nonlinear processes. The superiority of the proposed method is verified by a numerical example design and the Tennessee–Eastman process.

Abnormality monitoring and causality analysis based on KF-PDC and IACE in blast furnace ironmaking process

ABSTRACT Blast furnace (BF) ironmaking is a highly complicated process with multi-variable nonlinear coupling and multi-mode characteristics. In this article, a developed kernel function partial derivative contribution (KF-PDC) is proposed for abnormality location, which makes up for the deficiency of linear multivariate statistical process monitoring (MSPM) and de-redundancies the variable candidate set of causality analysis. Then, to eliminate the influence of multi-mode, an online interval adaptive causation entropy (IACE) is established to analyse the cause–effect relationships of candidate abnormal variables, which contributes to distinguishing the direct and indirect causality, and the Haar wavelet based on the sliding window (HWSW) is constructed for the segmentation of different modes online. Finally, a case study using actual industrial BF ironmaking data illustrates that the monitoring method can better capture the abnormal furnace conditions and effectively obtain the root cause and propagation path.

A Multiblock Kernel Dynamic Latent Variable Model for Large-Scale Industrial Process Monitoring

For industrial processes, the multivariate statistical process monitoring (MSPM) technique is an effective way to perform data-driven process modeling and fault detection. Compared to static multivariate statistical models (SMSMs), dynamic multivariate statistical models (DMSMs) can effectively model the autocorrelation and intercorrelation of data and improve the performance of fault detection. However, the performance of using DMSMs degrades when one uses data collected from large-scale processes, as it cannot effectively model the local behavior of large-scale processes. In contrast, multi-block methods can reflect these local behaviors well. Meanwhile, considering that kernel mapping is a proven technique for revealing nonlinear relationships in large-scale data. In this paper, a multivariate statistical model based on the multi-block kernel dynamic latent variable (MBKDLV) is proposed to monitor large-scale industrial processes. It divides high-dimensional process variables into blocks and monitors the local behavior by analyzing these blocks’ dynamic and static relationships. MBKDLV-based process data modeling is composed of in-block and inter-block algorithms, which also consider in-block and inter-block nonlinearity and correlations. Three statistics are proposed for monitoring the local behavior of large-scale processes. The effectiveness of the proposed approach is illustrated by a case study of a hot strip mill process (HSMP).

Multivariate statistical process monitoring and control of machining process using principal component-based Hotelling T&lt;SUP align="right"&gt;2&lt;/SUP&gt; charts: a machine vision approach

Multivariate statistical process monitoring and control of machining process using principal component-based Hotelling T&lt;SUP align="right"&gt;2&lt;/SUP&gt; charts: a machine vision approach

Multivariate statistical process monitoring based on principal discriminative component analysis

A novel analytical algorithm called Principal Discriminative Component Analysis (PDCA) is proposed to implement just-in-time feature extraction, so that the deviation between the online monitored data and the normal operating dataset can be timely uncovered. Instead of extracting representative signature inherited in the dataset given from the normal operating condition, the PDCA algorithm can always force the extracted latent features to be extremely discriminative to the inconsistency between the online sampled data and the given dataset. Therefore, the application of the PDCA algorithm in Multi-variate Statistical Process Monitoring (MSPM) can consistently guarantee its salient efficiency in contrast to the counterparts. The superiority and effectiveness of the PDCA-based MSPM approach are demonstrated through comparisons in monitoring both static and dynamic processes.

Statistical process monitoring based on just-in-time feature analysis

Different from traditional analytical algorithms which usually extract representative signature only to characterize the variation in normal operating dataset, the proposed just-in-time feature analysis (JITFA) algorithm is derived to implement latent feature analysis in a just-in-time manner for every single online sampled data, so that the deviation between the online sampled data and the normal operating dataset could be timely uncovered. On the basis of the JITFA algorithm, the corresponding implementation for multivariate statistical process monitoring (MSPM) can timely extract a specific latent feature which exposes the inconsistency covered in the online sampled data to the maximal extent. The superiority and effectiveness of the JITFA-based MSPM approach are demonstrated through comparisons with a total of 11 counterparts including both static and dynamic MSPM methods. In addition to the salient monitoring performance, the proposed method does not involve any other pre-determined model parameters, which would further make itself to be more easily applicable in contrast to the classical and state-of-the-art approaches.

Statistical monitoring of the covariance matrix in multivariate processes: A literature review

Monitoring several correlated quality characteristics of a process is common in modern manufacturing and service industries. Although a lot of attention has been paid to monitoring the multivariate process mean, not many control charts are available for monitoring the covariance matrix. This paper presents a comprehensive overview of the literature on control charts for monitoring the covariance matrix in a multivariate statistical process monitoring (MSPM) framework. It classifies the research that has previously appeared in the literature. We highlight the challenging areas for research and provide some directions for future research.

Dynamic Process Monitoring Based on Variational Bayesian Canonical Variate Analysis

Fault detection and fault identification are consecutive steps of multivariate statistical process monitoring. In recent years, increasing attention has been paid to process dynamics. For dynamic process modeling, canonical variate analysis (CVA) extracts process dynamics effectively. However, process noises are not well analyzed in traditional CVA and corresponding fault identification methods are less studied. To solve these issues, a variational Bayesian CVA (VBCVA) model is proposed for dynamic process monitoring. Through a probabilistic perspective, the inevitable noises in realistic industrial processes can be captured in the new model. Moreover, the proposed model is further extended in the variational Bayesian framework to overcome the common problems in probabilistic methods. Besides, an improved fault identification approach based on fault relevance is introduced, which avoids the smearing effect caused by data reconstruction. Finally, the feasibility of the proposed process monitoring scheme is verified on the TE benchmark and a real wastewater treatment process.

Quality-Related Process Monitoring Based on Improved Kernel Principal Component Regression

To date, quality-related multivariate statistical methods are extensively used in process monitoring and have achieved admirable effects. However, most of them contain recursive processes, which result in higher time complexity and are not suitable for increasingly complex industrial processes. Therefore, this paper embeds singular value decomposition (SVD) into the kernel principal component regression (KPCR) to accomplish Quality-related process monitoring with a lower computational cost. Specifically, the kernel technique is devoted to map the original input into the higher dimensional space to boost the nonlinear ability of the principal component regression (PCR), and then the KPCR is employed to capture the correlation between the input kernel matrix and the output matrix. At the same time, the kernelized input space is decomposed into two orthogonal quality-related and quality-unrelated spaces by SVD, and the statistics of the two spaces are calculated to detect the faults respectively. Compared with other multivariate statistical methods, it has the following advantages: 1) A quality-related kernel principal component analysis (QR-KPCR) algorithm is proposed. 2) Compared with partial least squares method, the recursive process is omitted and the training time is shortened. 3) The model is more concise and the fault detection process is faster. 4) By contrast with other multivariate statistical process monitoring, it has a higher fault detection rate. Experimental results on a widespread example and an industry benchmark verify the effectiveness and reliability of the proposed method.

Detection of intermittent faults based on an optimally weighted moving average [formula omitted] control chart with stationary observations

The moving average (MA)-type scheme, also known as the smoothing method, has been well established within the multivariate statistical process monitoring (MSPM) framework since the 1990s. However, its theoretical basis is still limited to smoothing independent data, and the optimality of its equally or exponentially weighted scheme remains unproven. This paper aims to weaken the independence assumption in the existing MA method, and then extend it to a broader area of dealing with autocorrelated weakly stationary processes. With the discovery of the non-optimality of the equally and exponentially weighted schemes used for fault detection when data have autocorrelation, the essence that they do not effectively utilize the correlation information of samples is revealed, giving birth to an optimally weighted moving average (OWMA) theory. The OWMA method is combined with the Hotelling’s T2 statistic to form an OWMA T2 control chart (OWMA-TCC), in order to detect a more challenging type of fault, i.e., intermittent fault (IF). Different from the MA scheme that puts an equal weight on samples within a time window, OWMA-TCC uses correlation (autocorrelation and cross-correlation) information to find an optimal weight vector (OWV) for the purpose of IF detection (IFD). In order to achieve a best IFD performance, the concept of IF detectability is defined and corresponding detectability conditions are provided, which further serve as selection criteria of the OWV. Then, the OWV is given in the form of a solution to nonlinear equations, whose existence is proven with the aid of the Brouwer fixed-point theory. Moreover, symmetrical structure of the OWV is revealed, and the optimality of the MA scheme for any IF directions when data exhibit no autocorrelation is proven. Finally, simulations on a numerical example and a continuous stirred tank reactor process are carried out to give a comprehensive comparison among OWMA-TCC and several existing static and dynamic MSPM methods. The results show a superior IFD performance of the developed methods.

A Holistic Probabilistic Framework for Monitoring Nonstationary Dynamic Industrial Processes

Multivariate statistical process monitoring (MSPM) methods provide sensitive indicators of process conditions by harnessing the value of massive process data. Large-scale industrial processes are subject to wide-range time-varying operating conditions such that some variables inevitably exhibit nonstationary behavior, which poses significant challenges for the design of MSPM schemes. In this brief, a novel nonstationary probabilistic slow feature analysis algorithm is developed to comprehensively describe both nonstationary and stationary variations that underlie process measurements during routine operations. For efficient parameter estimation, the expectation-maximization algorithm is employed. By modeling nonstationarity and stationarity as the random walk and stable autoregressive processes, interpretable monitoring statistics are constructed to detect abnormality in nonstationary dynamics, stationary dynamics, and stationary steady conditions. This forms a holistic and pragmatic monitoring framework for industrial processes, which is beneficial for reducing false alarms and providing meaningful operational information for industrial practitioners. The efficacy of the proposed monitoring framework is validated via two case studies.