High-dimensional Processing Research Articles

Some quasi-distribution-free schemes for Phase-I analysis of multivariate industrial processes

This paper introduces distance-based quasi-distribution-free approaches for Phase-I analysis of multivariate and high-dimensional processes. Phase-I analysis involves a retrospective study of the collected data to learn the features and stability of the process. At this stage, knowledge about the underlying distribution is limited and nonparametric statistical process monitoring (NSPM) plays a vital role. A wide range of NSPM tools exist for Phase-I analysis of univariate data, but similar schemes for multivariate and high-dimensional data are relatively limited. In particular, the multivariate NSPM literature related to monitoring the location and scale aspects in tandem in Phase I is scarce, but it is crucial in evaluating and establishing process stability. This paper aims to bridge the gap by introducing alternative multivariate schemes that are capable of detecting changes in the multivariate location vector, scale matrix, or both without knowledge of the process distribution. A thorough Monte Carlo experiment is conducted for comparative performance evaluation, and the results are encouraging in favour of the proposed schemes. Guidelines on using the proposed schemes are also offered. Finally, two industrial case studies are reported.

Farthest-Nearest Distance Neighborhood and Locality Projections Integrated With Bootstrap for Industrial Process Fault Diagnosis

It has become a big challenge and a hot topic of research to capture the most relevant features from high-dimensional process data for enhancing fault diagnosis. To effectively extract discriminative features from high-dimensional data, a novel dimensionality reduction (DR) approach named neighborhood and locality projections with the farthest and nearest distance (FNDNLP) is first proposed for industrial process fault feature acquisition and diagnosis. By constructing intra-class weights and inter-class weights, FNDNLP takes both the intra-class distance and the inter-class distance into consideration in its objective function, improving the diagnostic ability of extracted features through maximizing the inter-class distance and minimizing the intra-class distance. In addition, bootstrap based FNDNLP (BFNDNLP) is further proposed to handle the matrix decomposition problem in FNDNLP. To find the proper order through DR, the Akaike information criterion (AIC) is adopted. Finally, the Naïve Bayes (NB) based classifier is utilized to achieve acceptable fault diagnosis. The simulation results from two complex industrial cases indicate that the proposed methodology can achieve higher diagnosis accuracy than other related methods. What's more, the DR features are further analyzed to show the effectiveness and benefits of the proposed BFNDNLP extraction approach. The main source code of this paper can be obtained on GitHub soon.

Optimal Reaction Coordinates: Variational Characterization and Sparse Computation

Reaction coordinates (RCs) are indicators of hidden, low-dimensional mechanisms that govern the long-term behavior of high-dimensional stochastic processes. We present a novel and general variational characterization of optimal RCs and provide conditions for their existence. Optimal RCs are minimizers of a certain loss function, and reduced models based on them guarantee a good approximation of the statistical long-term properties of the original high-dimensional process. We show that for slow-fast systems, metastable systems, and other systems with known good RCs, the novel theory reproduces previous insight. Remarkably, for reversible systems, the numerical effort required to evaluate the loss function scales only with the variability of the underlying, low-dimensional mechanism, and not with that of the full system. The theory provided lays the foundation for an efficient and data-sparse computation of RCs via modern machine learning techniques.

Tug of War games and PDEs on graphs with applications in image and high dimensional data processing

The aim of this note is to revisit the connections between some stochastic games, namely Tug-of-War games, and a class of nonlocal PDEs on graphs. We consider a general formulation of Tug-of-War games which is shown to be related to many classical PDEs in the continuous setting. We transcribe these equations on graphs using ad hoc differential operators and we show that it covers several nonlocal PDEs on graphs such as infty -Laplacian, game p-Laplacian and the eikonal equation. This unifying mathematical framework allows us to easily design simple algorithms to solve several inverse problems in imaging and data science, with a particular focus on cultural heritage and medical imaging.

Joint Diagnosis of High-Dimensional Process Mean and Covariance Matrix based on Bayesian Model Selection

Apart from the quick detection of abnormal changes in a process, it is also critical to pinpoint faulty variables after an out-of-control signal. The existing diagnostic procedures mainly focus on the diagnosis of changes in the process mean. This article investigates the joint diagnosis of high-dimensional process mean and covariance matrix based on Bayesian model selection with nonlocal priors. The proposed procedure enjoys two promising features. First, in addition to the isolation of shifted components, it can also provide a probability that the identified components are true, which is very useful for elimination of root causes of abnormal changes. Second, it possesses the model consistency property in the sense that the probability of identifying the true components with shifts approaches one as the sample size increases. The performance comparisons favor the proposed procedure. A real example based on the urban waste water treatment process is provided to illustrate the implementation of the proposed method.

Phase I analysis of high-dimensional processes in the presence of outliers

One of the significant challenges in monitoring the quality of products today is the high dimensionality of quality characteristics. In this paper, we address Phase I analysis of high-dimensional processes with individual observations when the available number of samples collected over time is limited. Using a new charting statistic, we propose a robust procedure for parameter estimation in Phase I. This robust procedure is efficient in parameter estimation in the presence of outliers or contamination in the data. A consistent estimator is proposed for parameter estimation and a finite sample correction coefficient is derived and evaluated through simulation. We assess the statistical performance of the proposed method in Phase I. This assessment is carried out in the absence and presence of outliers. We show that, in both cases, the proposed control chart scheme effectively detects various kinds of shifts in the process mean. Besides, we present two real-world examples to illustrate the applicability of our proposed method.

Distribution-free Phase-II monitoring of high-dimensional industrial processes via origin and modified interpoint distance based algorithms

Improvements in measuring devices, the development of sensing technologies, automated record-keeping and cloud storage facilities have offered us a large volume of data streams involving many variables for data-driven decision-making. Consequently, monitoring high-dimensional (HD) processes has become an essential and exciting topic in statistical process monitoring. HD monitoring poses a significant challenge because considerable uncertainty is involved in estimating process parameters when an adequately large reference sample is unavailable before monitoring. Thus, designing an effective, conveniently implementable monitoring scheme for the HD processes remains a challenging task that has yet to be effectively addressed. This paper develops two distribution-free exponentially weighted moving average (EWMA) schemes to monitor multivariate and HD processes based on origin and interpoint distances. Previous works on interpoint distance-based charts are generally not distribution-free unless only one reference point is used as a benchmark to compute the interpoint distances. We offer an improved and general way to achieve conditionally distribution-free properties closely. The proposal also combines the advantage of the nonparametric approaches for jointly monitoring the process location and scale parameters with the better shift detection performance of an EWMA scheme. The new EWMA schemes can be easily implemented even when the data dimension is large with a small number of Phase-I samples. Control limits using the median run length as a metric are evaluated and tabulated for some cases to implement the proposed chart quickly. Extensive comparative studies have been conducted to verify the efficacy of the proposed monitoring schemes. Finally, the proposed schemes are illustrated with a real problem of monitoring high-dimensional quality characteristics from semiconductor manufacturing processes.

Double Sampling Adaptive Thresholding LASSO Variability Chart for Phase II Monitoring of High-Dimensional Data Streams

In many manufacturing or service systems, the product quality and/or process outcome is described in terms of a large number of quality characteristics. Since the use of large sample sizes in such systems imposes significant costs on the manufacturer, the sample size is usually considered smaller than the process dimension. In this condition, which is referred to high-dimensionality, the sample covariance matrix is not a positive semi-definite matrix. Moreover, another issue to establish variability control charts under high-dimensional setting is their speed to detect the process disturbances under both diagonal and off-diagonal shift patterns. To solve these drawbacks, in this study, a double sampling-based adaptive thresholding LASSO control chart called DSATL is developed to monitor the variability matrix of high-dimensional processes under sparsity assumption. The run length distribution of the DSATL chart under seven out-of-control patterns is evaluated taking a different number of quality characteristics into account. The simulation results show that the detectability of the DSATL chart outperforms the classical adaptive thresholding LASSO one under various changes in the process covariance matrix. The results also confirm that the sensitivity of the DSATL chart for real-time detection of covariance matrix anomalies improves as the process dimension increases.

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

Variational AutoEncoders-LSTM based fault detection of time-dependent high dimensional processes

In modern large-scale industrial processes, data are often high dimensional time-dependent due to the frequent sampling, dynamic nature and large number of variables. Appropriate monitoring of such processes allows for efficient decision-making that can improve the baseline of manufacturing companies either through decreasing production costs or enhancing production efficiency. Various latent variable-based control charts have been proposed for addressing high dimensional data; however, many of these methods assume that the data are independent and normally distributed. The violation of these assumptions results in an increased false alarm rate, in addition to the deterioration in the performance of such methods. In this study, we propose a Variational Autoencoder-Long Short Term Memory (VAE-LSTM) deep learning based chart that integrates the unique features of both VAE and LSTM for intelligent fault detection of time-dependent high dimensional processes. The effectiveness and applicability of the proposed model are demonstrated through extensive simulations, an open-source online dataset, and a real case study.

Dynamic transfer soft sensor for concept drift adaptation

Data-driven soft sensor technology has been widely used in process monitoring, quality prediction, etc. However, there are dynamic time-varying and concept drift problems in industrial processes, making the accurate modeling of the soft sensor is still a challenging task. To solve the above problems, this paper proposes a concept drift adaptive dynamic partial least squares method. The method maps high-dimensional process data into a low-dimensional latent variable subspace. Dynamic information is introduced by establishing the dynamic regression relationship between process latent variables and quality latent variables. At the same time, transfer learning is used to align second-order statistics between latent variables of data from different distributions to solve the concept drift problem. The experiments on multiple industrial datasets show that the method proposed in this paper can effectively reduce prediction errors and improve the generalization ability of the model.

Attribute-relevant distributed variational autoencoder integrated with LSTM for dynamic industrial soft sensing

Due to the complicated features of process data in terms of high dimension, nonlinearity and coupling, it tends to be a grand challenge to extract important features from complicated process for predicting quality variables using a reliable soft sensor model. Variational Autoencoder (VAE), one of the unsupervised deep learning methods, has been widely used for Gaussian-restricted feature extraction. Considering that the high-dimensional process variables may have different impacts on the predicted quality variable and process variables effect the prediction accuracy, a novel attribute-relevant distributed variational autoencoder (AR-DVAE) is first proposed to effectively extract features from input variables with different correlations. In AR-DVAE, by performing correlation analysis between the input variables and the quality variable, a distributed structure with two encoder subnets is constructed. In each encoder subnet, the input attributes have the same correlation with the output quality variable. As a consequence, the proposed AR-DVAE can extract features with dual channels. Next, a supervised regressor based on the long short-term memory machine shortened to LSTM is utilized for developing soft sensing between the extracted features and the quality variable. Case studies on two actual industrial processes are carried out to confirm the high accuracy of the presented AR-DVAE integrated with the LSTM soft sensing model.

Almost Qudits in the Prepare-and-Measure Scenario.

Quantum communication is often investigated in scenarios where only the dimension of Hilbert space is known. However, assigning a precise dimension is often an approximation of what is actually a higher-dimensional process. Here, we introduce and investigate quantum information encoded in carriers that nearly, but not entirely, correspond to standard qudits. We demonstrate the relevance of this concept for semi-device-independent quantum information by showing how small higher-dimensional components can significantly compromise the conclusions of established protocols. Then we provide a general method, based on semidefinite relaxations, for bounding the set of almost qudit correlations, and apply it to remedy the demonstrated issues. This method also offers a novel systematic approach to the well-known task of device-independent tests of classical and quantum dimensions with unentangled devices. Finally, we also consider viewing almost qubit systems as a physical resource available to the experimenter and determine the optimal quantum protocol for the well-known random access code.

Association Measure and Compact Prediction for Chemical Process Data from an Information-Theoretic Perspective

Mutual information (MI) has been widely used for association mining in complex chemical processes, but how to precisely estimate MI between variables of different numerical types, discriminate their association relationships with targets and finally achieve compact and interpretable prediction has not been discussed in detail, which may limit MI in more complicated industrial applications. Therefore, this paper first reviews the existing information-based association measures and proposes a general framework, GIEF, to consistently detect associations and independence between different types of variables. Then, the study defines four mutually exclusive association relations of variables from an information-theoretic perspective to guide feature selection and compact prediction in high-dimensional processes. Based on GIEF and conditional mutual information maximization (CMIM), a new algorithm, CMIM-GIEF, is proposed and tested on a fluidized catalytic cracking (FCC) process with 217 variables, one which achieves significantly improved accuracies with fewer variables in predicting the yields of four crucial products. The compact variables identified are also consistent with the results of Shapley Additive exPlanations (SHAP) and industrial experience, proving good adaptivity of the method for chemical process data.

A generalized multiple dependent state sampling chart based on ridge penalized likelihood ratio for high-dimensional covariance matrix monitoring

Online monitoring of high-dimensional processes variability in which the number of variables is larger than the sample size is a challenging issue for quality practitioners because the sample covariance matrix is not invariable. To deal with this challenge, a generalized multiple dependent state sampling (GMDS) chart based on ridge penalized likelihood ratio (RPLR) statistic is developed for Phase II monitoring of multivariate process variability under high-dimensional setting. The developed control chart benefits from three advantages: (1) departing from the conventional covariance matrix charts, it can be efficiently employed for both spars and non-spars covariance matrices; (2) it is able to detect spars shift patterns in which only a few covariance matrix elements are deviated from their nominal values; and (3) it outperforms the detectability of the RPLR chart in terms of average run length (ARL) and standard deviation of run length (SDRL). The performance of RPLR, MDS-RPLR, and GMDS-RPLR charts are compared using extensive simulation studies by considering different diagonal and/or off-diagonal covariance matrix disturbance. Moreover, sensitivity analysis are provided to analyze how the number of process variables and GMDS parameter affect the run length properties of the developed chart.

LSTMED: An uneven dynamic process monitoring method based on LSTM and Autoencoder neural network

Due to the complicated production mechanism in multivariate industrial processes, different dynamic features of variables raise challenges to traditional data-driven process monitoring methods which assume the process data is static or dynamically consistent. To tackle this issue, this paper proposes a novel process monitoring method based on the long short-term memory (LSTM) and Autoencoder neural network (called LSTMED) for multivariate process monitoring with uneven dynamic features. First, the LSTM units are arranged in the encoder–decoder form to construct an end-to-end model. Then, the constructed model is trained in an unsupervised manner to capture long-term time dependency within variables and dominant representation of high dimensional process data. Afterward, the kernel density estimation (KDE) method is performed to determine the control limit only based on the reconstruction error from historical normal data. Finally, effective online monitoring for uneven dynamic process can be achieved. The performance and advantage of the process monitoring method proposed are explained through typical cases, including the numerical simulation and Tennessee Eastman (TE) benchmark process, and comparative experimental analysis with state-of-the-art methods.

Development of a surrogate model for high-fidelity laser powder-bed fusion using tensor train and gaussian process regression

A multi-physics high-fidelity computational model is required to study the melting and grain growth phenomena in a laser powder-bed fusion (LPBF) additive manufacturing process. The major challenge with the high-fidelity model is long computational time, which makes it unsuited for any feasible process parameter optimization study in a high dimensional process design space. To address this challenge, surrogate models are a good option to replace the high-fidelity model, resulting in a significant shortening of the computational time at the expense of an acceptable drop in accuracy. In this study, a tensor train (TT) and Gaussian process regression (GPR) based methodology is proposed to develop a surrogate of the high-fidelity powder-scale model. An in-house developed powder-scale model is used to generate the training data by simulating a microscale model of the powder-bed for different values of laser power. The trained TT-GPR model can predict the thermal history of the powder-bed and melt pool geometry for a specified value of laser power, while the computation time required for prediction of any set of process conditions is less than one second. Here we can achieve an approximate computational speedup of 10^4 with the surrogate model. We provide evidence to claim that the proposed surrogate model provides high computational efficiency without compromising accuracy.

Distributed Robust Dictionary Pair Learning and Its Application to Aluminum Electrolysis Industrial Process

In modern industrial systems, high-dimensional process data provide rich information for process monitoring. To make full use of local information of industrial process, a distributed robust dictionary pair learning (DRDPL) is proposed for refined process monitoring. Firstly, the global system is divided into several sub-blocks based on the reliable prior knowledge of industrial processes, which achieves dimensionality reduction and reduces process complexity. Secondly, a robust dictionary pair learning (RDPL) method is developed to build a local monitoring model for each sub-block. The sparse constraint with l2,1 norm is added to the analytical dictionary, and a low rank constraint is applied to the synthetical dictionary, so as to obtain robust dictionary pairs. Then, Bayesian inference method is introduced to fuse local monitoring information to global anomaly detection, and the block contribution index and variable contribution index are used to realize anomaly isolation. Finally, the effectiveness of the proposed method is verified by a numerical simulation experiment and Tennessee Eastman benchmark tests, and the proposed method is then successfully applied to a real-world aluminum electrolysis process.

Dynamic transfer partial least squares for domain adaptive regression

The traditional soft sensor models are based on the independent and identical distribution assumption, which are difficult to adapt to changes in data distribution under multiple operating conditions, resulting in model performance deterioration. The domain adaptive transfer learning methods learn knowledge in different domains by means of distribution alignment, which can reduce the impact of data distribution differences, and effectively improve the generalization ability of the model. However, most of the existing models established by domain adaptation methods are static models, which cannot reflect the dynamic characteristics of the system, and have limited prediction accuracy when applied to dynamic system modeling under multiple operating conditions. The dynamic system modeling methods can effectively extract the dynamic characteristics of the data, but they cannot deal with the concept drift problem caused by the change of data distribution. This paper proposes a new dynamic transfer partial least squares method, which maps the high-dimensional process data into the low-dimensional latent variable subspace, establishes the dynamic regression relationship between the latent variables and the labels, and realizes the systematic dynamic modeling, at the same time, the model adds regular terms for distribution alignment and structure preservation, which realizes dynamic alignment of data distribution difference. The effectiveness of the proposed method is validated on three publicly available industrial process datasets.

High Dimensional Data Processing Based on Optimized DPC Algorithm

With the rapid growth of science and technology, machine learning and big data analysis have developed more and more difficult. Similarly, data also become difficult to process and classify due to the fact that the data dimension is becoming larger and larger. Furthermore, aiming at the defect due to the amount of data the clustering using speedy examine, search, and discovery of density peaks (also known as DPC) does not adjust to data sets with large dimensions (high-dimensional). Therefore, in this paper, we suggest an optimization procedure, which we pronounce as t-dpc, and is founded on the t-sne dimension lessening technique, and which can also optimize the technique for estimation of the Gaussian kernel function, using unified measurement criteria in solving density. In the simulation based experiments, using two different data sets i.e., the UCI standard data set, and the artificial data set, the proposed DPC procedure is associated with the classical t-dpc algorithm. The empirical evaluation and investigational outcomes illustrate that the proposed method of t-dpc not merely acclimatizes to the high-dimensional data sets, nevertheless it also increases the effectiveness of the classical DPC technique.