Multivariate Process Monitoring Research Articles

A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes

Detecting shifts in the mean vector of a multivariate statistical process control is crucial, and equally important is identifying the source of such a signal. This study introduces a novel approach that combines independent components analysis with support vector machines to address the challenge of multivariate process monitoring. In this hybrid independent components analysis-support vector machines method, statistical metrics $I^2$ derived from the independent components extracted through independent components analysis from observed data serve as input variables for the support vector machines. The probabilistic outputs generated by the support vector machines model are utilized as monitoring statistics for the proposed control chart, referred to as $I^2-\text{PoC}$. Simulation results validate the effectiveness of the independent components analysis with support vector machines approach in both detecting and identifying shifts in multivariate control processes, whether they follow a normal or non-normal distribution. Furthermore, the results demonstrate the robustness of this method in handling various challenges, including complex relationships between process variables, shifts of varying sizes, and different distribution shapes, when compared to existing approaches in the literature.

Variable contribution analysis in multivariate process monitoring using permutation entropy

Multivariate statistical process monitoring is widely used in industrial processes for performance monitoring and fault detection. Once a fault has been detected, fault diagnoses must be performed to identify the variables that are responsible for the fault or the deviation from normal operation. In this paper, we present a new data-driven method for variable contribution analysis to a fault or abnormal behaviour in a process. The method is based on permutation entropy (PE), which measures the order of fluctuations within a time series. The PE method for variable contribution analysis is independent of fault detection statistics, and is used to evaluate which variables experienced the greatest change in their expected behaviour over time, immediately after the fault has been identified. We show how PE can be used to identify and interpret the variables responsible for, or affected by faults in an industrial process. The well-known Tennessee Eastman Process (TEP) is used to illustrate the application of PE for variable contribution analysis. The results show that PE is an efficient analysis tool for specifying variable contributions to faults.

A General Framework for Robust Monitoring of Multivariate Correlated Processes

Statistical process control (SPC) charts provide an important analytic tool for online monitoring of sequential processes. Conventional SPC charts are designed for cases when in-control (IC) process observations are independent and identically distributed at different observation times and the IC process distribution belongs to a parametric (e.g., normal) family. In practice, however, these model assumptions are rarely valid. To address this issue, there have been some existing discussions in the SPC literature for handling cases when the IC process distribution cannot be described well by a parametric form, and some nonparametric SPC charts have been developed based on data ranking and/or data categorization. However, both data ranking and data categorization would lose information in the original process observations. Consequently, the effectiveness of the nonparametric SPC charts would be compromised. In this article, we make another research effort to handle this problem by developing a general process monitoring framework that is robust to the IC process distribution and short-ranged serial correlation. The new method tries to preserve as much information in the original process observations as possible. Instead of using data ranking and/or data categorization, it is based on intensive data pre-processing, including data decorrelation, data transformation, and data integration. Because the distribution of the pre-processed data can be approximated well by a parametric distribution, the design and implementation of the new method is relatively simple. Numerical studies show that it is indeed robust to the IC process distribution and effective for online monitoring of multivariate processes with short-ranged serial correlation.

Industrial Process Monitoring Based on Parallel Global-Local Preserving Projection with Mutual Information

This paper proposes a parallel monitoring method for plant-wide processes by integrating mutual information and Bayesian inference into a global-local preserving projections (GLPP)-based multi-block framework. Unlike traditional multivariate statistic process monitoring (MSPM) methods, the proposed MI-PGLPP method transforms plant-wide monitoring into several sub-block monitoringtasks by fully taking advantage of a parallel distributed framework. First, the original datasets of the process are divided into a group of data blocks by quantifying the mutual information of process variables. The block indexes of new data are generated automatically. Second, each data block is modeled by the GLPP method. The variable information and local structure are well preserved during the whole projection. Third, Bayesian inference is introduced to generate final statistics of the process by the probability framework. To illustrate the algorithm performance, a detailed case study is performed on the Tennessee Eastman process. Compared with the principle component analysis and GLPP-based method, the proposed MI-PGLPP provides higher FDRs and superior performance for plant-wide process monitoring.

Robust monitoring of multivariate processes with short‐ranged serial data correlation

AbstractControl charts are commonly used in practice for detecting distributional shifts of sequential processes. Traditional statistical process control (SPC) charts are based on the assumptions that process observations are independent and identically distributed and follow a parametric distribution when the process is in‐control (IC). In practice, these assumptions are rarely valid, and it has been well demonstrated that these traditional control charts are unreliable to use when their model assumptions are invalid. To overcome this limitation, nonparametric SPC has become an active research area, and some nonparametric control charts have been developed. But, most existing nonparametric control charts are based on data ordering and/or data categorization of the original process observations, which would result in information loss in the observed data and consequently reduce the effectiveness of the related control charts. In this paper, we suggest a new multivariate online monitoring scheme, in which process observations are first sequentially decorrelated, the decorrelated data of each quality variable are then transformed using their estimated IC distribution so that the IC distribution of the transformed data would be roughly N(0, 1), and finally the conventional multivariate exponentially weighted moving average (MEWMA) chart is applied to the transformed data of all quality variables for online process monitoring. This chart is self‐starting in the sense that estimates of all related IC quantities are updated recursively over time. It can well accommodate stationary short‐range serial data correlation, and its design is relatively simple since its control limit can be determined in advance by a Monte Carlo simulation. Because information loss due to data ordering and/or data categorization is avoided in this approach, numerical studies show that it is reliable to use and effective for process monitoring in various cases considered.

Monitoring multistage multivariate therapeutic processes using risk‐adjusted model‐based group multivariate EWMA control chart

AbstractThere exist multitude of therapeutic processes and the results are commonly observed during various dependent steps. For studying such processes that are referred to as multistage therapeutic processes, two concepts are of particular importance; risk adjustment and cascade property. To monitor such processes, a variety of control charts including model‐based control charts are used. In order to design model‐based control charts, analysts must first recognize a suitable model to identify the multistage processes by considering process risks and cascade property. Based on the identified model, the control charts can be proposed. In this study, a risk‐adjusted time‐variant linear state space model is introduced. Afterward, the model order and its parameters are estimated based on Hankel singular value decomposition (HSVD) and prediction error minimization (PEM) methods. Then, the group multivariate exponentially weighted moving average (GMEWMA) control chart is used to monitor a multistage multivariate therapeutic process. To evaluate the performance of the model‐based control chart, a simulation study as well as a two‐stage thyroid cancer surgery was used. Results show that the proposed control chart performs well for predicting and monitoring of multistage multivariate therapeutic processes in real world.

A novel multivariate statistical process monitoring algorithm: Orthonormal subspace analysis

Partial least squares (PLS) and canonical correlation analysis (CCA) are two most popular key performance indicators (KPI) monitoring algorithms, which have shortcomings in dealing with the KPI-related information leakage problem, the model identification problem, the component selection problem, and the process hypothesis problem. To overcome these shortcomings, this study proposes a new multivariate statistics-based process monitoring (MSPM) method called the orthonormal subspace analysis (OSA) method. OSA divides process data and KPI data into three orthonormal subspaces through an analytic solution. Hence, OSA not only can detect a fault but also can judge whether the fault is KPI-related or KPI-unrelated. OSA is always effective irrespective of the availability of the KPI variables in the online monitoring stage. In OSA, the cumulative percent variance (CPV) method is adopted for principal component selection. In brief, OSA is a more effective method than PLS and CCA, and it achieves superior performance in fault detection and classification.

Minimalist module analysis for fault detection and localization

Traditional multivariate statistical-based process monitoring (MSPM) methods are effective data-driven approaches for monitoring large-scale industrial processes, but have a shortcoming in handling the redundant correlations between process variables. To address this shortcoming, this study proposes a new MSPM method called minimalist module analysis (MMA). MMA divides process data into several different minimalist modules and one more independent module. All variables in the minimalist module are strongly correlated, and no redundant variables exist; therefore, the extracted feature components in one minimalist module will not be disturbed by noise from the other modules. This study also proposes new monitoring indices and a fault localization strategy for MMA, and simulation tests demonstrate that MMA achieves superior performance in fault detection and localization.

Fusing NIR and Process Sensors Data for Polymer Production Monitoring.

Process analytical technology and multivariate process monitoring are nowadays the most effective approaches to achieve real-time quality monitoring/control in production. However, their use is not yet a common practice, and industries benefit much less than they could from the outcome of the hundreds of sensors that constantly monitor production in industrial plants. The huge amount of sensor data collected are still mostly used to produce univariate control charts, monitoring one compartment at a time, and the product quality variables are generally used to monitor production, despite their low frequency (offline measurements at analytical laboratory), which is not suitable for real-time monitoring. On the contrary, it would be extremely advantageous to benefit from predictive models that, based on online sensors, will be able to return quality parameters in real time. As a matter of fact, the plant setup influences the product quality, and process sensors (flow meters, thermocouples, etc.) implicitly register process variability, correlation trends, drift, etc. When the available spectroscopic sensors, reflecting chemical composition and structure, consent to monitor the intermediate products, coupling process, and spectroscopic sensor and extracting/fusing information by multivariate analysis from this data would enhance the evaluation of the produced material features allowing production quality to be estimated at a very early stage. The present work, at a pilot plant scale, applied multivariate statistical process control (MSPC) charts, obtained by data fusion of process sensor data and near-infrared (NIR) probes, on a continuous styrene-acrylonitrile (SAN) production process. Furthermore, PLS regression was used for real-time prediction of the Melt Flow Index and percentage of bounded acrylonitrile (%AN). The results show that the MSPC model was able to detect deviations from normal operative conditions, indicating the variables responsible for the deviation, be they spectral or process. Moreover, predictive regression models obtained using the fused data showed better results than models computed using single datasets in terms of both errors of prediction and R 2. Thus, the fusion of spectra and process data improved the real-time monitoring, allowing an easier visualization of the process ongoing, a faster understanding of possible faults, and real-time assessment of the final product quality.

Investigation of weight loss in mozzarella cheese using NIR predicted chemical composition and multivariate analysis

The objective of the present study was to develop multivariate process monitoring models for weight loss of high-moisture mozzarella cheese during shelf-life. Eighty-two production batches from an industrial cheese factory were sampled in four non-consecutive days. Chemical composition of curd and mozzarella was analyzed using near infrared spectroscopy. Weight loss of mozzarella balls at 10 and 21 days after production were evaluated as measure of process efficiency. Principal component regression and linear regression coupled with variable selection were tested for their accuracy in predicting weight loss. Analysis of variance highlighted that shelf-life and sampling day had the strongest effect on mozzarella cheese chemical composition. Weight loss was predicted from curd and mozzarella composition with coefficient of determination ranging from 0.49 to 0.54. Batches with unsatisfactory performances were determined with accuracy ranging between 0.81 and 0.84. Possible applications of proposed models were evaluated based on their performances as well on their usability in dairies. Overall, findings suggest that the most suitable method for routine process control is linear regression algorithm on selected variables.

Convolutional neural nets in chemical engineering: Foundations, computations, and applications

AbstractIn this article, we review the mathematical foundations of convolutional neural nets (CNNs) with the goals of: (i) highlighting connections with techniques from statistics, signal processing, linear algebra, differential equations, and optimization, (ii) demystifying underlying computations, and (iii) identifying new types of applications. CNNs are powerful machine learning models that highlight features from grid data to make predictions (regression and classification). The grid data object can be represented as vectors (in 1D), matrices (in 2D), or tensors (in 3D or higher dimensions) and can incorporate multiple channels (thus providing high flexibility in the input data representation). CNNs highlight features from the grid data by performing convolution operations with different types of operators. The operators highlight different types of features (e.g., patterns, gradients, geometrical features) and are learned by using optimization techniques. In other words, CNNs seek to identify optimal operators that best map the input data to the output data. A common misconception is that CNNs are only capable of processing image or video data but their application scope is much wider; specifically, datasets encountered in diverse applications can be expressed as grid data. Here, we show how to apply CNNs to new types of applications such as optimal control, flow cytometry, multivariate process monitoring, and molecular simulations.

Out-of-Control Multivariate Patterns Recognition Using D2 and SVM: A Study Case for GMAW

Industrial processes seek to improve their quality control, including new technologies and satisfying requirements for globalised markets. In this paper, we present an innovative method based on Multivariate Pattern Recognition (MVPR) and process monitoring in a real-world study case. By identifying a distinctive out-of-control multivariate pattern using the Support Vector Machines (SVM) and the Mahalanobis Distance D2 it is possible to infer the variables that disturbed the process; hence, possible faults can be predicted knowing the state of the process. The method is based on our previous work, and in this paper we present the method application for an automated process, namely, the robotic Gas Metal Arc Welding (GMAW). Results from the application indicate an overall accuracy up to 88.8%, which demonstrates the effectiveness of the method, which can also be used in other MVPR tasks.

Dynamic neural orthogonal mapping for fault detection

Dynamic principal component analysis (DPCA) and its nonlinear extension, dynamic kernel principal component analysis (DKPCA), are widely used in the monitoring of dynamic multivariate processes. In traditional DPCA and DKPCA, extended vectors through concatenating current process data point and a certain number of previous process data points are utilized for feature extraction. The dynamic relations among different variables are fixed in the extended vectors, i.e. the adoption of the dynamic information is not adaptively learned from raw process data. Although DKPCA utilizes a kernel function to handle dynamic and (or) nonlinear information, the prefixed kernel function and the associated parameters cannot be most effective for characterizing the dynamic relations among different process variables. To address these problems, this paper proposes a novel nonlinear dynamic method, called dynamic neural orthogonal mapping (DNOM), which consists of data dynamic extension, a nonlinear feedforward neural network, and an orthogonal mapping matrix. Through backpropagation and Eigen decomposition (ED) technique, DNOM can be optimized to extract key low-dimensional features from original high-dimensional data. The advantages of DNOM are demonstrated by both theoretical analysis and extensive experimental results on the Tennessee Eastman (TE) benchmark process. The results on the TE benchmark process show the superiority of DNOM in terms of missed detection rate and false alarm rate. The source codes of DNOM can be found in https://github.com/htz-ecust/DNOM .

Reducing response delay in multivariate process monitoring by a stacked long-short term memory network and real-time contrasts

Response delay is a critical performance measure for detecting the process shift. Reducing response delay in multivariate process monitoring, especially under big streaming data, is one of many data analysis challenges on smart manufacturing. In the literature, researchers introduced the real-time contrast control chart (RTC) to fast detect the process shift. However, most of the RTCs are underperformed in the big data environment. In this research, a new RTC combined with a stacked long short-term memory network named LSTM-RTC was proposed to monitor big streaming data with short response delay. The LSTM-RTC consists of two LSTM layers, a dropout layer between the LSTM layers, and a dense layer to extend the capability to catch the streaming data's hierarchical representation. A variety of synthesized multivariate normal and bivariate gamma datasets were used to evaluate the proposed LSTM-RTC with other benchmark methods. The experiment results show that the proposed LSTM-RTC outperforms the other benchmark RTC methods with the lowest response delay. The evaluation on two real-world cases also highlights the advantage of the proposed LSTM-RTC on real-world process monitoring applications.

Nonparametric Phase-II control charts for monitoring high-dimensional processes with unknown parameters

Monitoring multivariate and high-dimensional data streams is often an essential requirement for quality management in manufacturing and service sectors in the Industry 4.0 era. Identifying a suitable distribution for a multivariate data set, especially when the number of variables is much larger than the sample size, is often challenging. Consequently, in a high-dimensional set-up, that is, when the number of variables under investigation exceeds sample size, parametric methods are generally not reliable in practice. There are various nonparametric schemes based on data depth for multivariate process monitoring, which are applicable only when the sample size is reasonably larger than the number of variables in the process but not in a high-dimensional set-up. We discuss that most of these charts are not robust when the true process parameters are unknown. There are, however, some nonparametric schemes for a high-dimensional process, when true process parameters are known. Nevertheless, when process parameters are unknown, a highly robust nonparametric scheme for monitoring high-dimensional processes is not yet available. In this paper, we propose some Shewhart-type nonparametric monitoring schemes based on specific distance metrics for surveillance of multivariate as well as high-dimensional processes. Our proposed charts are easy to implement, interpret and also advantageous in simultaneous monitoring of multiple aspects. We discuss the design and implementation issues in details. We carry out a performance study using Monte Carlo simulations and illustrate the proposed methods using a dataset related to industrial production.

On the distribution of the [formula omitted] statistic, used in statistical process monitoring, for high-dimensional data

A modification to the asymptotic distribution of the T2-statistic used in multivariate process monitoring is provided when the dimension of the vectors may exceed the sample size. Under certain mild condition, a unified limit distribution is obtained that is applicable for both Phase I and II charts. Further the limit holds for charts based on individual observations as well as subgroup means. The limit is easily applicable and does not need any data preprocessing or dimension reduction. Simulations are used to demonstrate the accuracy of the proposed limit.

Fault detection of wind turbines via multivariate process monitoring based on vine copulas

This study seeks to develop a scheme for fault detection, under the framework of multivariate process monitoring, employing the d-vine copula-based dependence measure (D-VCDM). This measure adapts to nonlinear and non-Gaussian dependence among high-dimensional variables. We improve upon the previous procedures by incorporating the quantile regression neural network and kernel density estimation. Additionally, we modify the conventional generalized local probability through a variable scale trick (VSGLP), to describe the evolution process of anomalies in the most fault-prone region more intuitively. The scheme is termed as D-VCDM-VSGLP, and its feasibility is verified through a numerical experiment and a real-world application on supervisory control and data acquisition system data collected from a wind turbine. We find that it outperforms the initial C-vine copula-based dependence description and multilinear principal component analysis in terms of accuracy. Besides, the adjustable parameters, including the number of intervals and the coefficient of scale variability in VSGLP, are designed to be convenient for practical use.

On the Efficient Monitoring of Multivariate Processes with Unknown Parameters

Control charts are commonly used tools that deal with monitoring of process parameters in an efficient manner. Multivariate control charts are more practical and are of greater importance for timely detection of assignable causes in multiple quality characteristics. This study deals with multivariate memory control charts to address smaller shifts in process mean vector. By adopting a new homogeneous weighting scheme, we have designed an efficient structure for multivariate process monitoring. We have also investigated the effect of an estimated variance covariance matrix on the proposed chart by considering different numbers and sizes of subgroups. We have evaluated the performance of the newly proposed multivariate chart under different numbers of quality characteristics and varying sample sizes. The performance measures used in this study include average run length, standard deviation run length, extra quadratic loss, and relative average run length. The performance analysis revealed that the proposed control chart outperforms the usual scheme under both known and estimated parameters. An application of the study proposal is also presented using a data set related to Olympic archery, for the monitoring of the location of arrows over the concentric rings on the archery board.

Self-starting single control charts for multivariate processes: a comparison of methods

Abstract Paper aims Based on challenges faced in real SPC application, this paper considers implementation and performance of self-starting methodology in multivariate process monitoring. Originality Traditional omnibus charts depend on in-control process parameters while parameters are generally known. However, in real settings, this information may not exist. This paper proposes and compares novel methods to overcome this difficulty. Research method This paper introduces, evaluates the performance and implements multivariate self-starting charts (SSMEC, SSMELR, and SSMME) for multivariate process monitoring. Main findings Proposed SSMME chart is the best choice in real application because it proves better performance in response to various simulation scenarios and gives diagnostic tools for further analysis. Implications for theory and practice The main contributions are the comparison of different self-starting approaches and introducing a novel multivariate self-starting chart that are suitable in real process monitoring and illustrate the benefit of the selected SPC chart with hypertension monitoring.

The efficiency of run rules schemes for the multivariate coefficient of variation in short runs process

In real industries, multivariate process monitoring is crucial as there are many instances that involve at least two quality variables to be monitored simultaneously. The short runs process is commonly seen in production after the industry moved toward flexible manufacturing. Monitoring the coefficient of variation (CV) is useful in a wide variety of scientific areas. In a view of the importance in monitoring the CV and the fact that most real-life data in process monitoring are multivariate in nature, this paper proposes to monitor the multivariate CV in short runs process by means of run rules (RR) control charts. A Markov chain model is established for designing the proposed charts. The statistical performances of the RR multivariate CV (MCV) and Shewhart MCV (SH MCV) charts are compared in terms of the truncated average run length and the expected truncated average run length. The results show that the proposed charts surpass the SH MCV chart for detecting small and moderate multivariate CV shifts. The implementation of the RR MCV chart in the short runs process is illustrated with an example using a real dataset.