Multivariate Statistical Process Research Articles

Multiple exponentially weighted moving average control schemes for monitoring and diagnostics of correlated quality variables of different types with individual observations

In the era of big data analytics, more data, in terms of quantity and variety, are being collected for quality control purposes, thus leading to new challenges, especially for multivariate statistical process control (MSPC). One such challenge, which has received very little attention, is multivariate control chart (MCC) for monitoring correlated quality variables of different types. A recent study by Huang et al. (2023), which develops a Shewhart type MCC, is the first attempt to tackle such a challenge. The proposed chart of Huang et al. (2023), which utilizes the step-down multiple testing procedure of Holm (1979), not only can monitor correlated variables of different types, but also can provide instantaneous diagnostics of which parameters are out of control when the chart signals. In this study, we adapt and extend the methodology of Huang et al. (2023) to develop an exponentially weighted moving average (EWMA) chart specifically for the case when the sample size is one. The proposed chart is shown to be effective in detecting parameter changes as well as diagnosing which parameters are out of control when the chart signals.

Real-time multivariate statistical monitoring of biopharmaceutical processes with no prior product-specific history

Biologics drug substance manufacturing normally comprises several processing steps including cell culture, purification, etc. Real-time Multivariate Statistical Process Monitoring (RT-MSPM) is an integral need in order to achieve early fault detection and prevention to guarantee robust and efficient process performance. As a part of RT-MSPM framework, Batch Evolution Models (BEMs), which take into account the transient phase of a batch prior to reaching full maturity (steady state), are the standard methods for real-time early fault detection in batch processes. Prior product-specific knowledge and training dataset are critical in order to model time (batch maturity) dependencies. Unlike traditional settings, modern biopharmaceutical manufacturing and clinical facilities use modular designs to enable simultaneous production of multiple medicines and quickly pivot from producing one medicine to another where no prior product-specific historical data might be available. In this article, first, a framework is proposed to detect the steady state operation of a manufacturing scale cell culture bioreactor. This is critical to achieve the ultimate goal to demonstrate feasibility of developing a continuous statistical process monitoring framework to monitor the steady state operation of a batch with no prior product specific history. Steady state time series data from previous step are processed following a systematic approach. The requirements for statistical properties of the processed time series in order to leverage data from different products to monitor a new product are outlined. The new framework establishes a guideline to develop models using datasets from other products with different recipes to monitor new products with no prior product-specific history. A novel software product is developed to test the framework. The framework is successfully applied for real-time monitoring of fast-rate sensors of a manufacturing cell culture bioreactor.

Online nonparametric process monitoring for IoT systems using edge computing

Facilitated by modern sensing and integrated circuit technology, Internet of Things (IoT) systems now have the ability to intelligently and automatically collect data, perform computational tasks on an edge, and efficiently make decisions locally or remotely. However, existing multivariate statistical process control methods commonly assume that sensors directly transmit the raw measurements to the central server, and thus the edge computing capability of IoT systems is not effectively harnessed. To fill this literature gap, we propose a monitoring scheme tailored for IoT systems by leveraging their edge computational power. In particular, inspired by the inherent hierarchical structure of the IoT system, we construct the monitoring statistics hierarchically. At each edge device, we first construct a nonparametric stream-level statistic for each data stream and aggregate them to derive the device-level statistics. To eliminate the inconsistency in device-level statistics across different edge devices, the device-level statistics are transformed into time-related statistics, which are then sent to the central server. At the central server, a system-level monitoring statistic is constructed, and the most informative edge devices that should transmit statistics are intelligently determined. Theoretical analyses are carried out to justify the effectiveness of the data transmission strategy. Both simulations and a case study, which is based on an IoT system we built, are conducted to evaluate the performance and validate the superiority of the proposed monitoring scheme.

Copula deep learning control chart for multivariate zero inflated count response variables

With the increasing popularity of big data analysis, research on zero-inflated count data with the copula method has garnered significant attention because zero-inflated count data do not follow a normal distribution and have high correlation among variables. Within the domain of quality control, there has been limited emphasis on multivariate statistical process control (SPC) techniques that specifically address the challenge of multicollinearity within regression models for multivariate zero-inflated count responses. In this paper, we explain a computational challenge in handling big data with parametric generalized linear models, such as the zero-inflated Poisson model. This challenge motivates us to introduce a copula-based deep learning and neural network model involving multivariate zero-inflated count response variables with highly correlated explanatory variables. This approach builds upon existing deep learning and neural network models designed for univariate count responses, expanding them to encompass multivariate count response models through the integration of a copula regression framework. To evaluate the performance of our proposed methodology, we conduct a comparative analysis of accuracies using the zero-inflated Poisson model, univariate deep learning and neural network models, multivariate count response deep learning and neural network models, and our proposed copula deep learning and neural network models. This assessment involves employing metrics such as root mean square error (RMSE), weighted mean absolute percentage error (WMAPE), and mean absolute deviation (MAD). We include both copula-based asymmetrical zero-inflated simulated data and real-world data. We also propose a temporal dependence control chart for assessing the temporal dependence between bivariate zero-inflated count response variables. Our proposed copula deep learning and neural network temporal control charts can check time-varying dependence and outliers by leveraging the Shewhart statistical process control chart and t-copula ARMA-GARCH dynamic control correlation.

Portable spectroscopy, digital imaging colorimetry and multivariate statistical tools in contaminant identification: A case study of mint (Mentha) and basil (Ocimum basilicum)

The advent of portable Fourier-Transform Infrared (FTIR) and Raman spectrometers has revolutionized analysis capabilities, presenting the possibility of on-site contaminant identification without the need for specialized laboratory settings. Compared to laboratory instrumentation, portable spectroscopy is more prone to noise, and appropriate spectral processing procedures need to be established. This paper introduces a comprehensive methodology that integrates acquisition techniques, spectral analysis, and mathematical tools necessary for utilizing handheld spectrometers to diagnose plant contamination. It focuses on determining the efficacy of handheld FTIR, Raman spectroscopy, and digital imaging for detecting contaminants in two food plants, Basil (Ocimum basilicum) and Mint (Mentha). The study examines the impact of three pollutants: iron (II) sulphate (FeSO4), zinc (II) sulphate (ZnSO4), and copper (II) sulphate (CuSO4), on these plants, but also the necessary amount of measurements to spot the pollutants' effects. Measurements were conducted at the start, after 24 hours, and after 48 hours of exposure, on both fresh and dried plant leaves, as well as in solution. Spectral effects of each of the pollutants were identified with the use of multivariate statistical process control techniques. With the help of the developed methodologies, researchers can identify in-situ contaminant effects, exposure times and run diagnostics directly on the leaf both in alive and dried plants.

A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes

Detecting shifts in the mean vector of a multivariate statistical process control is crucial, and equally important is identifying the source of such a signal. This study introduces a novel approach that combines independent components analysis with support vector machines to address the challenge of multivariate process monitoring. In this hybrid independent components analysis-support vector machines method, statistical metrics $I^2$ derived from the independent components extracted through independent components analysis from observed data serve as input variables for the support vector machines. The probabilistic outputs generated by the support vector machines model are utilized as monitoring statistics for the proposed control chart, referred to as $I^2-\text{PoC}$. Simulation results validate the effectiveness of the independent components analysis with support vector machines approach in both detecting and identifying shifts in multivariate control processes, whether they follow a normal or non-normal distribution. Furthermore, the results demonstrate the robustness of this method in handling various challenges, including complex relationships between process variables, shifts of varying sizes, and different distribution shapes, when compared to existing approaches in the literature.

Variable contribution analysis in multivariate process monitoring using permutation entropy

Multivariate statistical process monitoring is widely used in industrial processes for performance monitoring and fault detection. Once a fault has been detected, fault diagnoses must be performed to identify the variables that are responsible for the fault or the deviation from normal operation. In this paper, we present a new data-driven method for variable contribution analysis to a fault or abnormal behaviour in a process. The method is based on permutation entropy (PE), which measures the order of fluctuations within a time series. The PE method for variable contribution analysis is independent of fault detection statistics, and is used to evaluate which variables experienced the greatest change in their expected behaviour over time, immediately after the fault has been identified. We show how PE can be used to identify and interpret the variables responsible for, or affected by faults in an industrial process. The well-known Tennessee Eastman Process (TEP) is used to illustrate the application of PE for variable contribution analysis. The results show that PE is an efficient analysis tool for specifying variable contributions to faults.

Data fusion of Raman spectra in MSPC for fault detection and diagnosis in pharmaceutical manufacturing

This study investigates the use of Raman spectroscopy fused with other types of data (e.g., pH, temperature and turbidity) for multivariate statistical process control of two pharmaceutical case studies: one simulated industrial-scale fed-batch process for the production of penicillin and one real lab-scale crystallization process. The monitoring schemes are built on local principal component analysis models and hyper-parameters are tuned with regards to highest accuracy in fault detection. Accuracies above 90% are obtained for all types of data and level of DF. Furthermore, for the first case study the model built solely on spectra achieves higher fault detection rates, when only considering faults that also result in off-specification quality. This is supported by the fact that the fault is not necessarily detected when it occurs, but rather when it starts to affect quality variables as measured by the spectra.

Improved Fault Detection Method based on HSMM

This paper proposes a fault detection method for multivariate statistical process control. The proposed method combines the Forward-Backward Hidden Semi-Markov Model (HSMM) and Principal Component Analysis (PCA). A stochastic automaton was used for multi-mode detection with many observation sequences. We used agglomerative clusters to find the initial parameters of HSMM. We allocated an adaptive threshold and a fixed threshold in each mode for fault detection with PCA, including Hotelling T2 statistic and squared predictive error (Q statistic). We simulated this method on the Tennessee Eastman Process (TEP). Some faults were designed with various runs and times of occurrence. The experimental results were compared with the Mixture Bayesian PCA, Hidden Markov Model (HMM), and HSMM methods. The results are robust with an efficient detection rate. This activity recommends ways to find action plans for multi-mode process monitoring in chemical plants.

Process monitoring and fault diagnosis method combining bagging DPCA‐ICA with moving window Kolmogorov–Smirnov test

AbstractProcess monitoring and fault diagnosis systems are essential for ensuring the quality and safety of industrial processes. Real industrial process data always has dynamic features, autocorrelation features, and non‐Gaussian distributions, making many multivariate statistical process monitoring (MSPM) methods, such as principal component analysis (PCA), not work well. To solve the problem, a new process monitoring method (named DPCA‐ICA‐moving window Kolmogorov–Smirnov test [MWKS]) is proposed in the paper, which is a combination of DPCA and independent component analysis (ICA) with adaptive weight parameters and introduces a moving window technique and two‐sample K‐S test to obtain the K‐S distance of the commonly used Hotelling's and square predictive error as new monitoring parameters. Experimental results on the Tennessee Eastman chemical process demonstrate the effectiveness and accuracy of the proposed method, which shows it has better performance than existing commonly used methods. The average fault detection rate (FDR) is 93.131%, while the average false alarm rate (FAR) of the proposed method is only 0.1%.

Comments on the Effects of Variables’ Magnitude of Process Changes and Standard Deviations in “Adaptive Multivariate Statistical Process Control for Monitoring Time-Varying Processes”

Comments on the Effects of Variables’ Magnitude of Process Changes and Standard Deviations in “Adaptive Multivariate Statistical Process Control for Monitoring Time-Varying Processes”

A distribution-free phase II control chart for multivariate individual data with simple post signal diagnostics

Multivariate statistical process control (MSPC) charts are particularly useful when there is a need to simultaneously monitor several quality characteristics of a process. Most of the control charts in MSPC assume that the quality characteristics follow some parametric multivariate distribution, such as the normal. This assumption is almost impossible to be justified in practice. Distribution-free MSPC charts are attractive, as they can overcome this hurdle by guaranteeing a stable (or in-control (IC)) performance of the control chart without the assumption of a parametric multivariate process distribution. Utilizing an existing distribution-free multivariate tolerance interval, we propose a Phase II Shewhart-type distribution-free MSPC chart for individual observations, with control limits based on Phase I order statistics. In addition to being easy to interpret, the proposed chart preserves the original scale of measurements and can easily identify out-of-control variables after a signal. The exact in-control performance based on the conditional and unconditional perspectives is presented and examined along with the control limits determination. The out-of-control performance of the chart is studied by simulation for data from a number of multivariate distributions. Illustrative examples are provided for chart implementation, using both real and simulated data, along with a summary and conclusions.

Online nonparametric monitoring for asynchronous processes with serial correlation

Existing multivariate statistical process control methods commonly require all data streams have the same sampling interval. In practice, this assumption may not be valid, as different sensors can have different sampling intervals. In this article, we first propose a generic nonparametric monitoring scheme to online monitor the asynchronous data streams without considering serial correlation. Then the proposed scheme is extended such that it can handle serially correlated data streams. Specifically, we construct a nonparametric local statistic for each data stream, which is sensitive to mean shifts. To eliminate the influence of different sampling intervals, our innovative idea is to transform the local statistics into time-related statistics according to the sampling intervals. A global monitoring scheme is then constructed based on the sum of top- r time-related statistics. To extend the proposed method for serially correlated data streams, we further propose a novel estimation method for the pairwise covariance functions and the data streams can be decorrelated accordingly. Numerical simulations and a case study are conducted, showing the effectiveness of the proposed method in handling asynchronous data streams with serial correlation.

Use of ultrasound and NIRs as tools for monitoring ice formation in superchilled meat

This study aimed to gain a deeper understanding of the ice crystal formation during the superchilling process (SC) of meat by using non-destructive methods. Ultrasound inspection was performed considering the ultrasonic pulse velocity (UPV), while Near Infrared Spectroscopy (NIR) measurements were carried out with a spectral range of 950–1650 nm. Signals from both methodologies were periodically taken during the controlled freezing of 8 beef cubic samples of (6 cm3). Obtained results revealed that both methods have the ability to detect the ice crystal formation during the SC process, and particularly during the period of thermal arrest. Hence, these results evidence that ultrasound inspection and NIR, combined with Multivariate Statistical Process Control charts, could be implemented as quality control procedures during the SC process of muscle foods, allowing a more accurate control of the process and thus, minimizing the potential deleterious effects of ice crystals formation in meat.

Novel predictive approaches for drug-induced convulsions in non-human primates using machine learning and heart rate variability analysis.

Drug-induced convulsions are a major challenge to drug development because of the lack of reliable biomarkers. Using machine learning, our previous research indicated the potential use of an index derived from heart rate variability (HRV) analysis in non-human primates as a biomarker for convulsions induced by GABAA receptor antagonists. The present study aimed to explore the application of this methodology to other convulsants and evaluate its specificity by testing non-convulsants that affect the autonomic nervous system. Telemetry-implanted males were administered various convulsants (4-aminopyridine, bupropion, kainic acid, and ranolazine) at different doses. Electrocardiogram data gathered during the pre-dose period were employed as training data, and the convulsive potential was evaluated using HRV and multivariate statistical process control. Our findings show that the Q-statistic-derived convulsive index for 4-aminopyridine increased at doses lower than that of the convulsive dose. Increases were also observed for kainic acid and ranolazine at convulsive doses, whereas bupropion did not change the index up to the highest dose (1/3 of the convulsive dose). When the same analysis was applied to non-convulsants (atropine, atenolol, and clonidine), an increase in the index was noted. Thus, the index elevation appeared to correlate with or even predict alterations in autonomic nerve activity indices, implying that this method might be regarded as a sensitive index to fluctuations within the autonomic nervous system. Despite potential false positives, this methodology offers valuable insights into predicting drug-induced convulsions when the pharmacological profile is used to carefully choose a compound.

Measuring the quality of the learning process in basic courses using the multivariate statistical process control method

The quality of learning is the key to being able to compete in the world of education. This research aims to measure the quality of the learning process in basic courses at the University of Logistics and International Business. The method used is Multivariate Statistical Control Process with subgroups whose size is not constant. In this method, multivariate control of the target process is carried out using the T2 Hotelling control diagram. Meanwhile, control of multivariate process variability is carried out using a generalized variance control chart. The sample data used in this research was taken from grades in English courses and introductory logistics courses from 20 classes with a total of 498 students. Data collection was carried out randomly from 2019-2022. The results obtained were that in the Multivariate Process Target (MPT) calculation using the T2 Hotelling control diagram it was found that class 1 was out of control. Meanwhile, in calculating Multivariate Process Variability (MPV) using a Generalized Variance (GV) based control diagram, it was found that classes 5 and 20 were out of control. Investigating the cause of the class being out of control is a part that must be explored by study program and university leaders in an effort to improve the teaching and learning process.

The Impact of Postoperative Pulmonary Complications on Perioperative Outcomes in Patients Undergoing Pneumonectomy: A Multicenter Retrospective Cohort Study of the German Thorax Registry.

Postoperative pulmonary complications have a deleterious impact in regards to thoracic surgery. Pneumonectomy is associated with the highest perioperative risk in elective thoracic surgery. The data from 152 patients undergoing pneumonectomy in this multicenter retrospective study were extracted from the German Thorax Registry database and presented after univariate and multivariate statistical processing. This retrospective study investigated the incidence of postoperative pulmonary complications (PPCs) and their impact on perioperative morbidity and mortality. Patient-specific, preoperative, procedural, and postoperative risk factors for PPCs and in-hospital mortality were analyzed. A total of 32 (21%) patients exhibited one or more PPCs, and 11 (7%) died during the hospital stay. Multivariate stepwise logistic regression identified a preoperative FEV1 < 50% (OR 9.1, 95% CI 1.9-67), the presence of medical complications (OR 7.4, 95% CI 2.7-16.2), and an ICU stay of more than 2 days (OR 14, 95% CI 3.9-59) as independent factors associated with PPCs. PPCs (OR 13, 95% CI 3.2-52), a preoperative FEV1 < 60% in patients with previous pulmonary infection (OR 21, 95% CI 3.2-52), and continued postoperative mechanical ventilation (OR 8.4, 95% CI 2-34) were independent factors for in-hospital mortality. Our data emphasizes that PPCs are a significant risk factor for morbidity and mortality after pneumonectomy. Intensified perioperative care targeting the underlying risk factors and effects of PPCs, postoperative ventilation, and preoperative respiratory infections, especially in patients with reduced pulmonary reserve, could improve patient outcomes.

A New Index to Detect Process Deviations Using IR Spectroscopy and Chemometrics Process Tools

AbstractProcess analytical technologies (PATs) have transformed the beverage production management by providing real-time monitoring and control of critical process parameters through non-destructive measurements, such as those obtained with infrared (IR) spectroscopy and enabling process readjustment if necessary. New requirements in the analysis of beverages call for new methods, so in this article, we propose a method based on the construction of multivariate statistical process control (MSPC) charts from a new dissimilarity index (the evolving window dissimilarity index, EWDI) to monitor fermentation processes. The EWDI was applied to monitor wine alcoholic fermentation, the biochemical transformation of sugars into ethanol. Small-scale fermentations were carried out and analyzed using a portable mid-infrared spectrometer. In some of them, process deviations due to nitrogen deficiency or temperature changes were intentionally promoted to evaluate the performance of the EWDI. The MSPC charts build by using the fermentations carried out under normal operating conditions allowed identifying deviations of the fermentation in its early stages. Furthermore, the shape of the EWDI curve over time provides insights about the specific type of deviation occurring. These results show the potential of this new approach to improve the monitoring and control of key process stages in biochemical processes in the food industry, which allows maximizing quality and minimizing losses. Graphical Abstract

Detecting mean changes in experience sampling data in real time: A comparison of univariate and multivariate statistical process control methods.

Detecting early warning signals of developing mood disorders in continuously collected affective experience sampling (ESM) data would pave the way for timely intervention and prevention of a mood disorder from occurring or to mitigate its severity. However, there is an urgent need for online statistical methods tailored to the specifics of ESM data. Statistical process control (SPC) procedures, originally developed for monitoring industrial processes, seem promising tools. However, affective ESM data violate major assumptions of the SPC procedures: The observations are not independent across time, often skewed distributed, and characterized by missingness. Therefore, evaluating SPC performance on simulated data with typical ESM features is a crucial step. In this article, we didactically introduce six univariate and multivariate SPC procedures: Shewhart, Hotelling's T², EWMA, MEWMA, CUSUM and MCUSUM. Their behavior is illustrated on publicly available affective ESM data of a patient that relapsed into depression. To deal with the missingness, autocorrelation, and skewness in these data, we compute and monitor the day averages rather than the individual measurement occasions. Moreover, we apply all procedures on simulated data with typical affective ESM features, and evaluate their performance at detecting small to moderate mean changes. The simulation results indicate that the (M)EWMA and (M)CUSUM procedures clearly outperform the Shewhart and Hotelling's T² procedures and support using day averages rather than the original data. Based on these results, we provide some recommendations for optimizing SPC performance when monitoring ESM data as well as a wide range of directions for future research. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A change-point–based control chart for detecting sparse mean changes in high-dimensional heteroscedastic data

Because of the “curse of dimensionality,” high-dimensional processes present challenges to traditional multivariate statistical process monitoring (SPM) techniques. In addition, the unknown underlying distribution of and complicated dependency among variables such as heteroscedasticity increase the uncertainty of estimated parameters and decrease the effectiveness of control charts. In addition, the requirement of sufficient reference samples limits the application of traditional charts in high-dimension, low-sample-size scenarios (small n, large p). More difficulties appear when detecting and diagnosing abnormal behaviors caused by a small set of variables (i.e., sparse changes). In this article, we propose two change-point–based control charts to detect sparse shifts in the mean vector of high-dimensional heteroscedastic processes. Our proposed methods can start monitoring when the number of observations is a lot smaller than the dimensionality. The simulation results show that the proposed methods are robust to nonnormality and heteroscedasticity. Two real data examples are used to illustrate the effectiveness of the proposed control charts in high-dimensional applications. The R codes are provided online.