High-dimensional Processing Research Articles

Variational approach to learning photonic unitary operators

Structured light, light tailored in its internal degrees of freedom, has become topical in numerous quantum and classical information processing protocols. In this work, we harness the high dimensional nature of structured light modulated in the transverse spatial degree of freedom to realize an adaptable scheme for learning unitary operations. Our approach borrows from concepts in variational quantum computing, where a search or optimization problem is mapped onto the task of finding a minimum ground state energy for a given energy/goal function. We achieve this by a pseudo-random walk procedure over the parameter space of the unitary operation, implemented with optical matrix-vector multiplication enacted on arrays of Gaussian modes by exploiting the partial Fourier transforming capabilities of a cylindrical lens in the transverse degree of freedom for the measurement. We outline the concept theoretically, and experimentally demonstrate that we are able to learn optical unitary matrices for dimensions d = 2, 4, 8, and 16 with average fidelities of >90%. Our work advances high dimensional information processing and can be adapted to both process and quantum state tomography of unknown states and channels.

High-dimensional process monitoring under time-varying operating conditions via covariate-regulated principal component analysis

High-dimensional process monitoring, which aims to raise warnings for faulty events, is a fundamental tool in ensuring the safety of large-scale industrial systems. Over the last few decades, there has been significant interest in Principal Component Analysis (PCA)-based Statistical Process Control (SPC) charts. However, most of them rely on PCA’s optimal properties in homogeneous data, which often fails when dealing with heterogeneous data from industrial processes under time-varying operating conditions. To address this issue, this study proposes a Covariate-regulated PCA model (CrPCA) seamlessly integrated into a process monitoring framework. The main idea is to adjust the principal component directions smoothly according to the time-varying operating conditions, resulting in significantly less information loss in the principal component scores compared to existing methods. This approach further yields a low-dimensional feature vector that follows a distribution with fixed parameters, regardless of covariates, for real-time surveillance. We validate the performance of our methodology through a comprehensive numerical study and two real-world applications: wind turbine condition monitoring and hot rolling process monitoring.

Matrix Separation and Poisson Multi-Bernoulli Mixture Filtering for Extended Multi-Target Tracking with Infrared Images

Multi-target tracking using infrared images is receiving more and more attention. There are many state-of-the-art methods, and the deep learning network and low-rank and sparse matrix separation are two kinds of methods with high accuracy. However, the former suffers from heavy training samples, and the latter requires high-dimensional processing, meaning its computing cost is huge. In this work, a united detection and tracking method with matrix separation and PMBM filtering is proposed. In the detection process, a low-rank and sparse matrix separation algorithm with a differentiable form based on a single image is constructed. In the filtering process, the multi-target state is modeled as a PMBM distribution, which is conjugate in the Bayesian framework. The two processes interact mutually in that the detection provides measurements, and the filtering offers prior information for the next detection to improve accuracy. The computational complexity is given by a theoretical analysis, which shows a significant reduction. The numerical analysis, carried out on a practical dataset, verifies an enhancement in the BSF and SCRG metrics and ROC curves.

A double sampling ridge penalized likelihood ratio control charting method with variable sample size for Phase II monitoring of high-dimensional covariance matrix

Today, manufacturers face two challenges: the financial limitation of sampling and the large number of study quality characteristics. In such situations, the problem dimension is usually larger than the sample size, which is known in the literature of statistical quality control as a high dimensional process. The use of conventional multivariate charts to monitor the covariance matrix of such processes is not possible for two reasons: (1) the non-reversibility of the sample covariance matrix, (2) the possibility of the occurrence of sparse shifts in which only a few elements of the covariance matrix are affected. This paper focuses on Phase II monitoring of the covariance matrix of high-dimensional processes based on integrating the ridge penalized likelihood ratio (RPLR) statistic and the combined DS-VSS sampling method, in which DS and VSS features are used concurrently. The performance of the proposed DS-VSS RPLR control chart is compared with the RPLR one in terms of three metrics of the average run length, standard deviation of the run length, and expected value of sample size under three types of joint diagonal/off-diagonal, diagonal and off-diagonal shift patterns. The results show that the DS-VSS scheme is more sensitive than the RPLR in detecting both sparse and non- sparse changes.

Monitoring of group-structured high-dimensional processes via sparse group LASSO

In a general high-dimensional process, a large number of process parameters or quality characteristics is found to be featured through their dependencies and relevance. The features that have similar characteristics or behaviors in the process operation can be categorized into multiple groups. Thus, when a few quality characteristics in the process change, it is highly probable that the process shift would have occurred in a few relevant groups. Recently, several advanced statistical process control techniques are developed to monitor the changes in high-dimensional processes under sparsity. However, monitoring schemes that utilize the grouped pattern of the quality characteristics are sparse. This paper proposes a new method to monitor the high-dimensional process when the grouped structure of the process data is observed. The proposed method identifies the potentially changed groups and individual variables within the groups based on a modified sparse group LASSO (MSGL) model. Then, a monitoring statistic is obtained using MSGL-based likelihood function to test abnormality of the process. Extensive numerical studies are conducted to demonstrate the effectiveness and efficiency of the proposed method. In addition, a real-life application of a liquefied natural gas process is presented to illustrate the proposed method.

RETROFIT: Real-Time Control of Time-Dependent 3D Point Cloud Profiles

Abstract In modern manufacturing processes, ensuring the precision of 3D profiles of products is crucial. Nonetheless, achieving this accuracy is challenging due to the complex interactions between process inputs and the data structure of the 3D profile data. Our solution, a 3D profile-based control framework, addresses this challenge by actively adapting and controlling the manufacturing process to enhance 3D shape accuracy. 3D profile scans represent the ultimate measure of desired part quality. Therefore, utilizing them as the system responses for control purposes yields the most direct and effective feedback. We leverage recent advancements from Koopman operator theory to create an effective model-based control strategy. Initially, we estimate the process model by exploring the relationship between 3D profiles and heterogeneous process inputs. Then, we formulate an online model predictive control law. Challenges include dealing with unstructured, high-dimensional 3D point cloud data, capturing spatial and temporal structures, and integrating heterogeneous, high-dimensional process input data into the control model. To overcome these challenges, we introduce RETROFIT, a solution designed for the real-time control of time-dependent 3D point cloud profiles. Unlike traditional models, RETROFIT is not bound by linear assumptions and can handle unstructured 3D point cloud data directly. We demonstrate its effectiveness through a wire arc additive manufacturing case study, highlighting its potential to enhance 3D profile accuracy in manufacturing processes.

Control charting methods for monitoring high dimensional data streams: A conceptual classification scheme

There are always challenges in various industrial or non-industrial processes in which the product quality/service is described by a large number of quality characteristics. Thus, statistical process monitoring (SPM) techniques for capturing the quality of high-dimensional processes are becoming increasingly important, and various control charts have been developed to monitor different types of quality characteristics in this area. In general, the construction of conventional control charts to monitor multivariate processes in a high-dimensional setting has some statistical limitations and leads to misleading interactions. As a result, novel control charting techniques have recently been suggested to ameliorate the efficiency of monitoring schemes under high-dimensional data streams. This paper undertakes a conceptual classification structure using content analysis to classify the existing literature in this context for the period 2004–2024. Furthermore, on the basis of 72 selected papers, the research gaps are identified and some directions to stimulate potential for future research are suggested.

Sparse Regularization Based on Orthogonal Tensor Dictionary Learning for Inverse Problems

In seismic data processing, data recovery including reconstruction of the missing trace and removal of noise from the recorded data are the key steps in improving the signal-to-noise ratio (SNR). The reconstruction of seismic data and removal of noise becomes a sparse optimization problem that can be solved by using sparse regularization. Sparse regularization is a key tool in the solution of inverse problems. They are used to introduce prior knowledge and make the approximation of ill-posed inverses feasible. It deals with ill-posedness by replacing an ill-posed inverse problem with a well-posed problem that has a solution close to the true solution. In the last 2 decades, interest has shifted from linear toward nonlinear regularization methods even for linear inverse problems. In inverse problems, regularizations serve as stabilizing the solution of ill-posed inverse problems and give a solution that adequately fits measurements without producing unjustifiably complex artifacts. In this paper, we present a novel sparse regularization based on a tensor-based dictionary method for inverse problems (seismic data interpolation and denoising). This regularization avoids the vectorization step for sparse representation of seismic data during the reconstruction process. The key step in sparsifying signals is the choice of sparsity-promoting dictionary learning. The learning-based approach can adaptively sparsify datasets but has high computational complexity and involves no prior-constraint pattern information for the dataset. Many existing dictionary learning methods would be computationally infeasible for the high dimensional seismic data processing. These methods also destroy the essential information as well as it reduces the discriminability and expressibility of the signal, since they deal with vectorization. In this paper, the orthogonal tensor dictionary learning that learns a dictionary from the input data by employing orthogonality and separability is proposed as sparse regularization for the inverse problems. The performance of the proposed method was validated in seismic data interpolation and denoising individually as well as simultaneously. Numerical examples of synthetic and real seismic datasets demonstrate the validity of the proposed method. The SNR of the recovered data confirms that the proposed method is the most effective method than K-singular value decomposition and orthogonal dictionary learning methods.

Improved Fault Detection in Chemical Engineering Processes via Non-Parametric Kolmogorov–Smirnov-Based Monitoring Strategy

Fault detection is crucial in maintaining reliability, safety, and consistent product quality in chemical engineering processes. Accurate fault detection allows for identifying anomalies, signaling deviations from the system’s nominal behavior, ensuring the system operates within desired performance parameters, and minimizing potential losses. This paper presents a novel semi-supervised data-based monitoring technique for fault detection in multivariate processes. To this end, the proposed approach merges the capabilities of Principal Component Analysis (PCA) for dimensionality reduction and feature extraction with the Kolmogorov–Smirnov (KS)-based scheme for fault detection. The KS indicator is computed between the two distributions in a moving window of fixed length, allowing it to capture sensitive details that enhance the detection of faults. Moreover, no labeling is required when using this fault detection approach, making it flexible in practice. The performance of the proposed PCA–KS strategy is assessed for different sensor faults on benchmark processes, specifically the Plug Flow Reactor (PFR) process and the benchmark Tennessee Eastman (TE) process. Different sensor faults, including bias, intermittent, and aging faults, are considered in this study to evaluate the proposed fault detection scheme. The results demonstrate that the proposed approach surpasses traditional PCA-based methods. Specifically, when applied to PFR data, it achieves a high average detection rate of 98.31% and a low false alarm rate of 0.25%. Similarly, when applied to the TE process, it provides a good average detection rate of 97.27% and a false alarm rate of 6.32%. These results underscore the efficacy of the proposed PCA–KS approach in enhancing the fault detection of high-dimensional processes.

Components and Exit Times of Brownian Motion in Two or More p-Adic Dimensions

The fundamental solution to a pseudo-differential equation for functions defined on the d-fold product of the p-adic numbers, Qp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {Q}_p$$\\end{document}, induces an analogue of the Wiener process in Qpd\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {Q}_p^d$$\\end{document}. As in the real setting, the components are 1-dimensional p-adic Brownian motions with the same diffusion constant and exponent as the original process. Asymptotic analysis of the conditional probabilities shows that the vector components are dependent for all time. Exit time probabilities for the higher dimensional processes reveal a concrete effect of the component dependency.

Review of the current status and the potential of machine learning tools in boiling heat transfer

Boiling is a liquid-to-vapor phase change process that involves complex and high dimensional processes spanning multiple length- and time-scales. Most correlations apply to a narrow range of conditions and the development of mechanistic models has been limited. Machine learning (ML) provides tools to uncover fundamental principles governing such challenging problems. It uses a variety of techniques to analyze large datasets and identify patterns and relationships that are otherwise challenging to establish using the traditional methods. Attempts have already been made to leverage the power of ML to provide new insights and make more accurate predictions in boiling heat transfer. We here report a comprehensive review of such studies, put in perspective their findings with the existing knowledgebase obtained using traditional approaches, and identify the key advantages and challenges of ML in this domain. The review suggests that it is important to pay attention to data collection, feature selection and extraction, choice of algorithm, and performance metrics to improve the accuracy, reliability, and robustness of ML models. While the traditional analytical and empirical models are more interpretable and easier to visualize, ML models are noted to enable more accurate predictions over a wider range of operating conditions. The issues of interpretability can be addressed by reducing the complexity of the data via dimensionality reduction and clustering techniques. Moreover, physics-inspired techniques can be used to align predictions with physical principles and improve generalizability. The ability of ML to swiftly process alternative forms of data such as optical images, thermal maps, and acoustics in real-time opens new frontiers for investigations. Finally, we close the discussion by emphasizing the need for high-quality data collection and standardization for increasing the impact of ML in phase change heat transfer.

A change-point–based control chart for detecting sparse mean changes in high-dimensional heteroscedastic data

Because of the “curse of dimensionality,” high-dimensional processes present challenges to traditional multivariate statistical process monitoring (SPM) techniques. In addition, the unknown underlying distribution of and complicated dependency among variables such as heteroscedasticity increase the uncertainty of estimated parameters and decrease the effectiveness of control charts. In addition, the requirement of sufficient reference samples limits the application of traditional charts in high-dimension, low-sample-size scenarios (small n, large p). More difficulties appear when detecting and diagnosing abnormal behaviors caused by a small set of variables (i.e., sparse changes). In this article, we propose two change-point–based control charts to detect sparse shifts in the mean vector of high-dimensional heteroscedastic processes. Our proposed methods can start monitoring when the number of observations is a lot smaller than the dimensionality. The simulation results show that the proposed methods are robust to nonnormality and heteroscedasticity. Two real data examples are used to illustrate the effectiveness of the proposed control charts in high-dimensional applications. The R codes are provided online.

Data-driven process adjustment policies for quality improvement

Common objectives in machine learning research are to predict the output quality of manufacturing processes, to perform root cause analysis in case of reduced quality, and to propose intervention strategies. The cost of reduced quality must be weighed against the cost of the interventions, which depend on required downtime, personnel costs, and material costs. Furthermore, there is a risk of false negatives, i.e., failure to identify the true root causes, or false positives, i.e., adjustments that further reduce the quality. A policy for process adjustments describes when and where to perform interventions, and we say that a policy is worthwhile if it reduces the expected operational cost. In this paper, we describe a data-driven alarm and root cause analysis framework, that given a predictive and explanatory model trained on high-dimensional process and quality data, can be used to search for a worthwhile adjustment policy. The framework was evaluated on large-scale simulated process and quality data. We find that worthwhile adjustment policies can be derived also for problems with a large number of explanatory variables. Interestingly, the performance of the adjustment policies is almost exclusively driven by the quality of the model fits. Based on these results, we discuss key areas of future research, and how worthwhile adjustment policies can be implemented in real world applications.

Monitoring high-dimensional heteroscedastic processes using rank-based EWMA methods

Monitoring high-dimensional processes is a challenging task, as the underlying dependency structure among variables is often too complicated to estimate accurately. The inherent volatility of dependence, so-called heteroscedasticity, is rarely mentioned nor considered in process monitoring problems. We consider time-dependent heteroscedasticity a common cause variability and propose an integrated scheme for monitoring and diagnosis of changes in the location parameters of high-dimensional processes. Our proposed method consists of rank-based EWMA control charts which are designed to detect mean shifts in a small subset of variables. A bootstrap algorithm determines the control limits by achieving a pre-specified false alarm probability. A post-signal diagnosis strategy is executed to cluster the shifted variables and estimate a time window for the change point. Simulation results show that the proposed methodology is robust to heteroscedasticity and sensitive to small and moderate sparse mean shifts. It can efficiently identify out-of-control variables and the corresponding change points. A real-life example of monitoring online vibration data for predictive maintenance applications illustrates the proposed methodology.

Quantum Bayesian computation

AbstractQuantum Bayesian computation is an emerging field that levers the computational gains available from quantum computers. They promise to provide an exponential speed‐up in Bayesian computation. Our article adds to the literature in three ways. First, we describe how quantum von Neumann measurement provides quantum versions of popular machine learning algorithms such as Markov chain Monte Carlo and deep learning that are fundamental to Bayesian learning. Second, we describe quantum data encoding methods needed to implement quantum machine learning including the counterparts to traditional feature extraction and kernel embeddings methods. Third, we show how quantum algorithms naturally calculate Bayesian quantities of interest such as posterior distributions and marginal likelihoods. Our goal then is to show how quantum algorithms solve statistical machine learning problems. On the theoretical side, we provide quantum versions of high dimensional regression, Gaussian processes and stochastic gradient descent. On the empirical side, we apply a quantum FFT algorithm to Chicago house price data. Finally, we conclude with directions for future research.

An EWMA chart for high dimensional process with multi-class out-of-control information via random forest learning

ABSTRACT Modern manufacturing and quality monitoring involve multi-class out-of-control (OOC) information from the training sample. It is essential to use such information during online monitoring of data streams from complex processes. In this paper, a monitoring framework is designed by combining the random forest technique with the exponentially weighted moving average method for monitoring complex processes with multi-class OOC information. To be specific, a process surveillance technique in the form of a control chart is proposed based on the probability that the online data is classified as an in-control (IC) sample, and the control chart triggers an alarm when the probability is lower than the control limit. Our numerical findings based on the Monte–Carlo simulation show that the proposed control chart performs more effectively than its competitors under various distributions and data types, especially for high-dimensional cases when multi-class OOC information is known in advance. Moreover, the proposed method is illustrated with an application using the data related to the hard disk manufacturing processes.

Deep Cascade Gradient RBF Networks With Output-Relevant Feature Extraction and Adaptation for Nonlinear and Nonstationary Processes.

The main challenge for industrial predictive models is how to effectively deal with big data from high-dimensional processes with nonstationary characteristics. Although deep networks, such as the stacked autoencoder (SAE), can learn useful features from massive data with multilevel architecture, it is difficult to adapt them online to track fast time-varying process dynamics. To integrate feature learning and online adaptation, this article proposes a deep cascade gradient radial basis function (GRBF) network for online modeling and prediction of nonlinear and nonstationary processes. The proposed deep learning method consists of three modules. First, a preliminary prediction result is generated by a GRBF weak predictor, which is further combined with raw input data for feature extraction. By incorporating the prior weak prediction information, deep output-relevant features are extracted using a SAE. Online prediction is finally produced upon the extracted features with a GRBF predictor, whose weights and structure are updated online to capture fast time-varying process characteristics. Three real-world industrial case studies demonstrate that the proposed deep cascade GRBF network outperforms existing state-of-the-art online modeling approaches as well as deep networks, in terms of both online prediction accuracy and computational complexity.

Latent-Enhanced Variational Adversarial Active Learning Assisted Soft Sensor

As the acquisition of variables that measure quality is typically challenging, labeled samples for building a model for soft sensors are often inadequate. Additionally, owing to the installation of redundant sensors, high-dimensional process data with strong correlations are acquired. Therefore, in this study, a sample selection strategy for active learning, referred to as latent-enhanced variational adversarial active learning (LVAAL), is developed to enhance quality prediction performance with limited labeled data. The LVAAL method uses a minimax game to explore the latent representation of the original data. Specifically, a latent-enhanced variational autoencoder is used to deceive the adversarial network into predicting all samples, both labeled and unlabeled, as labeled. However, the adversarial network attempts to indicate the difference between latent representations. Subsequently, Gaussian process regression is adopted as a base model for LVAAL. With the proposed LVAAL selection strategy, informative unlabeled samples are incrementally selected, thereby enhancing the prediction performance. The application results for a numerical case and an industrial example demonstrate the advantages of LVAAL compared with existing active learning strategies.

Dioxin emission modeling using feature selection and simplified DFR with residual error fitting for the grate-based MSWI process

Municipal solid waste incineration (MSWI) with grate technology is a widely applied waste-to-energy process in various cities in China. Meanwhile, dioxins (DXN) are emitted at the stack and are the critical environmental indicator for operation optimization control in the MSWI process. However, constructing a high-precision and fast emission model for DXN emission operation optimization control becomes an immediate difficulty. To address the above problem, this research utilizes a novel DXN emission measurement method using simplified deep forest regression (DFR) with residual error fitting (SDFR-ref). First, the high-dimensional process variables are optimally reduced following the mutual information and significance test. Then, a simplified DFR algorithm is established to infer or predict the nonlinearity between the selected process variables and the DXN emission concentration. Moreover, a gradient enhancement strategy in terms of residual error fitting with a step factor is designed to improve the measurement performance in the layer-by-layer learning process. Finally, an actual DXN dataset from 2009 to 2020 of the MSWI plant in Beijing is utilized to verify the SDFR-ref method. Comparison experiments demonstrate the superiority of the proposed method over other methods in terms of measurement accuracy and time consumption.

A dynamical systems treatment of transcriptomic trajectories in hematopoiesis.

Inspired by Waddington's illustration of an epigenetic landscape, cell-fate transitions have been envisioned as bifurcating dynamical systems, wherein exogenous signaling dynamics couple to the enormously complex signaling and transcriptional machinery of a cell to elicit qualitative transitions in its collective state. Single-cell RNA sequencing (scRNA-seq), which measures the distributions of possible transcriptional states in large populations of differentiating cells, provides an alternate view, in which development is marked by the variations of a myriad of genes. Here, we present a mathematical formalism for rigorously evaluating, from a dynamical systems perspective, whether scRNA-seq trajectories display statistical signatures consistent with bifurcations and, as a case study, pinpoint regions of multistability along the neutrophil branch of hematopoeitic differentiation. Additionally, we leverage the geometric features of linear instability to identify the low-dimensional phase plane in gene expression space within which the multistability unfolds, highlighting novel genetic players that are crucial for neutrophil differentiation. Broadly, we show that a dynamical systems treatment of scRNA-seq data provides mechanistic insights into the high-dimensional processes of cellular differentiation, taking a step toward systematic construction of mathematical models for transcriptomic dynamics.