High-dimensional Processing Research Articles

A penalized likelihood-based quality monitoring via L2-norm regularization for high-dimensional processes

Technological advances have resulted in the introduction of new products and manufacturing processes that have a large number of characteristics and variables to be monitored to ensure the product quality. Simultaneous monitoring of variables under such a high-dimensional environment is quite challenging. Traditional approaches are less sensitive to the out-of-control signals in high-dimensional processes, especially when only a few variables are responsible for abnormal changes in the process output. Recently, variable selection-based charts are proposed to overcome the drawbacks of the traditional approaches. These approaches adopt diagnosis procedures to identify a small subset of potentially changed variables. However, the detection capability of these charts may be low in cases when the size of the shift is relatively small in a highly correlated data structure, which is critical in modern industry. Moreover, the complexity of computation would dramatically increase as the dimension of the process parameters increases due to the diagnosis procedure, which is inappropriate for online monitoring. This article proposes a new penalized likelihood-based approach via norm regularization, which does not select variables but rather “shrinks” all process mean estimates toward zero. A closed-form solution of the proposed approach along with probability distributions of the monitoring statistic under null and alternative hypotheses are obtained, which makes the proposed chart significantly efficient to monitor high-dimensional processes. In addition, we explore theoretical properties of the approach and present several extensions of the proposed chart and its integration with other existing charts. Finally, we compare the performance of the approach with existing methods and present a case study of a high-speed milling process.

An adaptive thresholding-based process variability monitoring

In high-dimensional processes, monitoring process variability is considerably difficult due to the large number of variables and the limited number of samples. Monitoring changes in the covariance matrix of a multivariate process is often used for monitoring process variability under the assumption that only a few elements in the covariance matrix are changed simultaneously from the in-control values. The existing LASSO-based covariance monitoring charts in the high-dimensional settings provide good performance in detecting some shift patterns depending on the prespecified tuning parameter. In practice, control charts that perform reasonably well over various shift patterns are desired when shift patterns are unknown. In this article, we propose a control chart based on an adaptive LASSO-thresholding for monitoring changes in the covariance matrix. The performance of the proposed chart, which is called the ALT-norm chart, is evaluated for various shift patterns and compared with the existing penalized likelihood-based methods. The results show the effectiveness of the proposed chart. Finally, we illustrate the advantages of the ALT-norm chart through simulated and real data from both the semiconductor industry and a high-dimensional milling process.

Quantum leakage detection using a model-independent dimension witness

Users of quantum computers must be able to confirm they are indeed functioning as intended, even when the devices are remotely accessed. In particular, if the Hilbert space dimension of the components are not as advertised -- for instance if the qubits suffer leakage -- errors can ensue and protocols may be rendered insecure. We refine the method of delayed vectors, adapted from classical chaos theory to quantum systems, and apply it remotely on the IBMQ platform -- a quantum computer composed of transmon qubits. The method witnesses, in a model-independent fashion, dynamical signatures of higher-dimensional processes. We present evidence, under mild assumptions, that the IBMQ transmons suffer state leakage, with a $p$ value no larger than $5{\times}10^{-4}$ under a single qubit operation. We also estimate the number of shots necessary for revealing leakage in a two-qubit system.

Visual assessment of multi-photon interference

Classical machine learning algorithms can provide insights on high-dimensional processes that are hardly accessible with conventional approaches. As a notable example, t-distributed Stochastic Neighbor Embedding (t-SNE) represents the state of the art for visualization of data sets of large dimensionality. An interesting question is then if this algorithm can provide useful information also in quantum experiments with very large Hilbert spaces. Leveraging these considerations, in this work we apply t-SNE to probe the spatial distribution of n-photon events in m-dimensional Hilbert spaces, showing that its findings can be beneficial for validating genuine quantum interference in boson sampling experiments. In particular, we find that nonlinear dimensionality reduction is capable to capture distinctive features in the spatial distribution of data related to multi-photon states with different evolutions. We envisage that this approach will inspire further theoretical investigations, for instance for a reliable assessment of quantum computational advantage.

Control charts for variability monitoring in high-dimensional processes

Monitoring process variability is associated with detecting changes in the covariance matrix of a multivariate normal process. Most monitoring methods estimate the sample covariance matrix and compare it with the in-control covariance matrix that is mostly priori known based on the sufficient historical data. However, when the sample size is smaller than the number of variables, the sample covariance matrix is not applicable to estimate the covariance matrix, since the matrix may not be positive semi-definite. In this paper, we propose a new control chart for monitoring changes in the covariance matrix when the sample size is smaller than the number of variables. The proposed chart is based on the ridge penalized likelihood ratio. It detects general changes, without sparsity assumption, in the covariance matrix efficiently when the sample size is small, while other existing penalized likelihood-based methods are expected to detect only sparse changes in the covariance matrix. The superiority of the proposed chart is demonstrated through an average run length performance to variety in shift patterns. The proposed chart also maintains a low computational complexity. These differentiated properties of the proposed chart were proved through numerous simulation studies and in a real example from the semiconductor industry.

Deep Learning-Based Dependability Assessment Method for Industrial Wireless Network

Techniques on 5G and Internet of things bring a strong potential paradigm shift to wireless communication applications in industrial domain. Hence, there is a strong need for quantitative dependability assessment in these applications. However, with the evergrowing complexity and amount of wireless communication systems, their dependability relevant parameters also increase rapidly. In addition, the deep neural network has advantages on high dimensional data process. Hence, a deep learning-based dependability assessment method is proposed to address the issue, wherein a deep auto-encoder based approach is proposed to reduce data dimension and to obtain the data codes, and DBSCAN is used to cluster these codes. An experimental environment is built for collecting data set on the Multifaces, and a rough classification method is proposed to obtain a superior deep encoder model. Based on the superior model and DBSCAN, the data set are mainly divided into four dependability clusters.

Weighted Local Discriminant Preservation Projection Ensemble Algorithm With Embedded Micro-Noise

High-dimensional data often cause the “curse of dimensionality” in data processing. Dimensionality reduction can effectively solve the curse of dimensionality and has been widely used in high-dimensional data processing. However, the existing dimensionality reduction algorithms neglect the effect of noise injection, failing to account for the datasets of large variance within classes and not effectively considering the stability of dimensionality reduction. To solve the problems, this paper proposes a weighted local discriminant preservation projection algorithm based on an ensemble imbedded mechanism with micro-noise injection (n_w_LPPD). The proposed algorithm aims to overcome the problem of large variance within classes and introduces an ensemble projection matrix via Bayesian fusion mechanism with micro-noise to enhance the antijamming capability of the model. Ten public datasets were used to verify the proposed algorithm. The experimental results demonstrated that the proposed algorithm is significantly effective, especially for the case of small sample datasets with high intraclass variance. The classification accuracy is improved by at least 10% compared to the case without dimensionality reduction. Even compared with some representative dimensionality reduction algorithms, the proposed n_w_LPPD has significantly superior classification performance.

Key Process Protection of High Dimensional Process Data in Complex Production

Key Process Protection of High Dimensional Process Data in Complex Production

Classically high-dimensional correlation: simulation of high-dimensional entanglement.

Classical correlation, akin to the entanglement of quantum states, has been proven to bear a great promise in classical and quantum systems, such as enhancing measurement precision and characterizing quantum channels. Despite numerous successful applications of such correlation, it is strictly limited to two orthogonal bases and fails to further extend its dimensionality in practice. Here, we report a classically high-dimensional correlation, which is mathematically equivalent to its quantum counterparts and can be used to simulate the violations of the high-dimensional Bell inequalities. Moreover, we also show theoretically that quantum channels with high-dimensional quantum states can be characterized robustly using our scheme when a one-sided channel is perturbed by a turbulent atmosphere. This means that our results not only provide new physical insights into the notion of classical correlation, but also show potential applications in high-dimensional quantum information processing.

Switching and information exchange in compressed estimation of coupled high dimensional processes

Compressed Estimation approaches, such as the Generalised Compressed Kalman Filter (GCKF), reduce the computational cost and complexity of high-dimensional and high-frequency data assimilation problems, usually without sacrificing optimality. Configured using adequate cores, such as the Unscented Kalman Filter (UKF), the GCKF could also treat certain high-dimensional non-linear cases. However, the application of a compressed estimation process is limited to a class of problems which inherently allow the estimation process to be divided, at certain intervals of time, into a set of lower-dimensional problems. This limitation prohibits applying the compressing techniques for estimating coupled high-dimensional processes. However, those limitations can be overcome by applying proper techniques. In this paper, the concepts of subsystem switching and information exchange architecture, namely ‘Exploiting Local Statistical Dependency’ (ELSD), have been derived and explored, allowing compressed estimators to mimic optimal full-Gaussian estimators. The performances of the methods have been verified through applications in solving usual types of Stochastic Partial Differential Equations (SPDEs). The computational advantages of using the proposed techniques have also been highlighted with a recommendation for its usage over the full filter when dealing with high-dimensional and high-frequency data assimilation.

Bayesian framework for fault variable identification

In most manufacturing processes, identifying the faulty process variables that may lead to process changes is crucial for quality engineers and practitioners. There are several parametric procedures for identifying faulty variables with the assumption that they follow multivariate normal distributions. However, in practice, the normality assumption restricts the applicability of such procedures in identifying the faulty variables. In addition, conventional procedures for fault identification are often computationally expensive, especially in high-dimensional processes. Therefore, this article proposes a data-driven Bayesian approach for fault identification that addresses the limitations posed by the normality assumption. The proposed approach is computationally efficient for high-dimensional data compared with existing approaches. Experimental results with various simulation studies and real-life data sets demonstrate the effectiveness of the proposed procedure.

Intrusion Detection System Using Multivariate Control Chart Hotelling's T2 Based on PCA

Statistical Process Control (SPC) has been widely used in industry and services. The SPC can be applied not only to monitor manufacture processes but also can be applied to the Intrusion Detection System (IDS). In network monitoring and intrusion detection, SPC can be a powerful tool to ensure system security and stability in a network. Theoretically, Hotelling’s T 2 chart can be used in intrusion detection. However, there are two reasons why the chart is not suitable to be used. First, the intrusion detection data involves large volumes of high-dimensional process data. Second, intrusion detection requires a fast computational process so an intrusion can be detected as soon as possible. To overcome the problems caused by a large number of quality characteristics, Principal Component Analysis (PCA) can be used. The PCA can reduce not only the dimension leading a faster computational, but also can eliminate the multicollinearity (among characteristic variables) problem. This paper is focused on the usage of multivariate control chart T 2 based on PCA for IDS. The KDD99 dataset is used to evaluate the performance of the proposed method. Furthermore, the performance of T 2 based PCA will be compared with conventional T 2 control chart. The empirical results of this research show that the multivariate control chart using Hotelling’s T 2 based on PCA has excellent performance to detect an anomaly in the network. Compared to conventional T 2 control chart, the T 2 based on PCA has similar performance with 97 percent hit rate. It also requires shorter computation time.

Neighborhood preserving neural network for fault detection

A novel statistical feature extraction method, called the neighborhood preserving neural network (NPNN), is proposed in this paper. NPNN can be viewed as a nonlinear data-driven fault detection technique through preserving the local geometrical structure of normal process data. The “local geometrical structure ” means that each sample can be constructed as a linear combination of its neighbors. NPNN is characterized by adaptively training a nonlinear neural network which takes the local geometrical structure of the data into consideration. Moreover, in order to extract uncorrelated and faithful features, NPNN adopts orthogonal constraints in the objective function. Through backpropagation and eigen decomposition (ED) technique, NPNN is optimized to extract low-dimensional features from original high-dimensional process data. After nonlinear feature extraction, Hotelling T2 statistic and the squared prediction error (SPE) statistic are utilized for the fault detection tasks. The advantages of the proposed NPNN method are demonstrated by both theoretical analysis and case studies on the Tennessee Eastman (TE) benchmark process. Extensive experimental results show the superiority of NPNN in terms of missed detection rate (MDR) and false alarm rate (FAR). The source code of NPNN can be found in https://github.com/htzhaoecust/npnn.

Reprint of: Big data approach to batch process monitoring: Simultaneous fault detection and diagnosis using nonlinear support vector machine-based feature selection

This paper presents a novel data-driven framework for process monitoring in batch processes, a critical task in industry to attain a safe operability and minimize loss of productivity and profit. We exploit high dimensional process data with nonlinear Support Vector Machine-based feature selection algorithm, where we aim to retrieve the most informative process measurements for accurate and simultaneous fault detection and diagnosis. The proposed framework is applied to an extensive benchmark data set which includes process data describing 22,200 batches with 15 faults. We train fault and time-specific models on the pre-aligned batch data trajectories via three distinct time horizon approaches: one-step rolling, two-step rolling, and evolving which varies the amount of data incorporation during modeling. The results show that two-step rolling and evolving time horizon approaches perform superior to the other. Regardless of the approach, proposed framework provides a promising decision support tool for online simultaneous fault detection and diagnosis for batch processes.

Detection of unstable periodic orbits in mineralising geological systems.

Worldwide, mineral exploration is suffering from rising capital costs, due to the depletion of readily recoverable reserves and the need to discover and assess more inaccessible or geologically complex deposits. For gold exploration, this problem is particularly acute. We propose an innovative approach to mineral exploration and orebody characterisation, based on the analysis of geological core data as a spatial dynamical system, using the mathematical tools of dynamical system analysis. This approach is highly relevant for orogenic gold deposits, which-in contrast to systems formed at chemical equilibrium-exhibit many features of nonlinear dynamical systems, including episodic fluctuations on various length and time scales. Feedback relationships between thermo-chemical and deformation processes produce recurrent fluid temperatures and pressures and the deposition of vein-filling minerals such as pyrite and gold. We therefore relax the typical assumption of chemical equilibrium and analyse the underlying processes as aseismic, non-adiabatic, and inherent to a hydrothermal, nonlinear dynamical open-flow chemical reactor. These processes are approximated using the Gray-Scott model of reaction-diffusion as a complex toy system, which captures some of the features of the underlying mineralisation processes, including the spatiotemporal Turing patterns of unsteady chemical reactions. By use of this analysis, we demonstrate the capability of recurrence plots, recurrence power spectra, and recurrence time probabilities to detect underlying unstable periodic orbits as one sign of deterministic dynamics and their robustness for the analysis of data contaminated by noise. Recurrence plot based quantification is then applied to three mineral concentrations in the core data from the Sunrise Dam gold deposit in the Yilgarn region of Western Australia. Using a moving window, we reveal the episodic recurring low-dimensional dynamic structures and the period doubling route to instability with depth, embedded in and originating from higher-dimensional processes of the complex mineralisation system.

Monitoring big process data of industrial plants with multiple operating modes based on Hadoop

For modeling and monitoring large-scale plant-wide processes with big data from multiple operating conditions, a novel distributed parallel Gaussian mixture model is proposed based on the Hadoop MapReduce framework. To deal with high-dimensional process variables, a multiblock method is adopted. For big data chunks in each divided block, an analytical procedure is carried out with three key procedures. First, the fundamental data statistics are obtained with the designed distributed and parallel manners for data standardization. Second, conventional Gaussian mixture model learning steps are accommodated in the parallel paradigm of the MapReduce platform. Finally, multilevel fault detection and diagnosis schemes are developed to conduct hierarchical monitoring from plant-wide, unit block, and variable levels. The feasibility and effectiveness of the proposed method are demonstrated on two study cases.

The Research on Denoising of SAR Image Based on Improved K-SVD Algorithm

SAR images often receive noise interference in the process of acquisition and transmission, which can greatly reduce the quality of images and cause great difficulties for image processing. The existing complete DCT dictionary algorithm is fast in processing speed, but its denoising effect is poor. In this paper, the problem of poor denoising, proposed K-SVD (K-means and singular value decomposition) algorithm is applied to the image noise suppression. Firstly, the sparse dictionary structure is introduced in detail. The dictionary has a compact representation and can effectively train the image signal. Then, the sparse dictionary is trained by K-SVD algorithm according to the sparse representation of the dictionary. The algorithm has more advantages in high dimensional data processing. Experimental results show that the proposed algorithm can remove the speckle noise more effectively than the complete DCT dictionary and retain the edge details better.

Big data approach to batch process monitoring: Simultaneous fault detection and diagnosis using nonlinear support vector machine-based feature selection

This paper presents a novel data-driven framework for process monitoring in batch processes, a critical task in industry to attain a safe operability and minimize loss of productivity and profit. We exploit high dimensional process data with nonlinear Support Vector Machine-based feature selection algorithm, where we aim to retrieve the most informative process measurements for accurate and simultaneous fault detection and diagnosis. The proposed framework is applied to an extensive benchmark data set which includes process data describing 22,200 batches with 15 faults. We train fault and time-specific models on the pre-aligned batch data trajectories via three distinct time horizon approaches: one-step rolling, two-step rolling, and evolving which varies the amount of data incorporation during modeling. The results show that two-step rolling and evolving time horizon approaches perform superior to the other. Regardless of the approach, proposed framework provides a promising decision support tool for online simultaneous fault detection and diagnosis for batch processes.

Accelerated and Reliable Analog Circuits Yield Analysis Using SMT Solving Techniques

Existing yield analysis methods are computationally expensive and generally encounter challenges with high-dimensional process parameters space. In this paper, we propose a new method for accelerated and reliable computation of parametric yield that combines the advantages of sparse regression and satisfiability modulo theory (SMT) solving techniques, and avoids issues in both. The key idea is to characterize the failure regions as a collection of hyperrectangles in the parameters space. Toward this goal, the method constructs sparse polynomial models based on adaptive least absolute shrinkage and selection operator to find low degree approximations of the circuit performances. A procedure inspired by statistical model checking is then introduced to assess the model accuracy. Given the constructed models, an SMT-based solving algorithm is employed to locate the failure hyperrectangles in the parameters space. The yield estimation is based on a geometric calculation of probabilistic volumes subtended by the located hyperrectangles. We demonstrate the effectiveness of our method using circuits that require expensive run-time simulation during yield evaluation. They include: an integrated ring oscillator, a 6T static RAM cell and a multistage fully-differential amplifier. Experimental results show that the proposed method is suitable for handling problems with tens of process parameters. Meanwhile, it can provide $5{\times }{-}2000{\times}$ speed-up over Monte Carlo methods, when a high prediction accuracy is required.

High-dimensional information processing through resilient propagation in quaternionic domain

Abstract This paper proposes a fast but effective machine learning technique for the high-dimensional information processing problems. A quaternion is the hyper-complex number in four dimensions which possesses four components in a single body with embedded phase information among their components. Thus, a neural network with quaternion as unit of information flow has ability to learn and generalize magnitude and argument of high-dimensional information simultaneously. The slow convergence and getting stuck into bad minima are the main drawbacks of the back-propagation learning algorithm; therefore learning technique with faster convergence is the demand of quaternionic neurocomputing. The basic issues of quaternionic back-propagation (ℍ-BP) algorithm have been combated by proposed resilient propagation algorithm in quaternionnic domain (ℍ-RPROP). The efficient and intelligent behavior of the proposed learning algorithm has been vindicated by the wide spectrum of benchmark problems. The simulations on different transformations demonstrate its ability to learn 3D motion, which is very desirable in viewing of objects through different orientations in many engineering applications. The critical thing is the training through mapping over straight line having a small number of points and eventually generalization over complicated geometrical structures (Sphere, Cylinder and Torus) possessing huge amount of point cloud data. The 3D and 4D time series prediction as benchmark applications are also taken into the account for the significant contribution of the proposed work. The substantial edge of the proposed algorithm is its quicker convergence with better approximation capability in high-dimensional problems.