Principal Subspace Research Articles

KPI-oriented process monitoring based on causal-weighted partial least squares

With the increasing demand for product quality, Key performance indicator (KPI)-oriented process monitoring plays an important role in modern industrial. Partial least squares (PLS)-based methods are widely adopted for KPI-oriented process monitoring, in which the process variable space is decomposed into a principal component subspace that has a strong correlation with KPIs and a residual subspace unrelated to KPIs, and the two spaces are monitored separately. However, if a KPI-unrelated fault occurs in a variable that has no causal relation with any KPI, but has a spurious correlation with some KPIs because of the existence of confounders, PLS based methods may mis-regard the KPI-unrelated fault as a KPI-related fault. This occurs because as a correlation analysis-based method, PLS cannot discriminate whether a variable is a real cause of KPIs or has a spurious correlation with KPIs. Motivated by this, this paper proposes two causal-weighted PLS methods for KPI-oriented process monitoring by combining the LiNGAM-based causal discovery with PLS, which first calculate causal weights based on the modified LiNGAM algorithm and bootstrap strategy, and then employ the causal weights to reweight the weight vectors of PLS to enhance the influence of causal variables in the KPI-related principal subspace and reduce the influence of spurious correlations. Case studies based on a simulated dataset, Tennessee Eastman Process dataset and field data from finishing rolling mill process show that the proposed methods can significantly reduce the false alarm rate for KPI-unrelated faults (i.e., the probability that a KPI-unrelated fault sample is mis-regarded as a KPI-related fault sample) caused by spurious correlation without significantly compromising the fault detection rate of KPI-related faults.

Mode-wise principal subspace pursuit and matrix spiked covariance model

Abstract This paper introduces a novel framework called Mode-wise Principal Subspace Pursuit (MOP-UP) to extract hidden variations in both the row and column dimensions for matrix data. To enhance the understanding of the framework, we introduce a class of matrix-variate spiked covariance models that serve as inspiration for the development of the MOP-UP algorithm. The MOP-UP algorithm consists of two steps: Average Subspace Capture (ASC) and Alternating Projection. These steps are specifically designed to capture the row-wise and column-wise dimension-reduced subspaces which contain the most informative features of the data. ASC utilizes a novel average projection operator as initialization and achieves exact recovery in the noiseless setting. We analyse the convergence and non-asymptotic error bounds of MOP-UP, introducing a blockwise matrix eigenvalue perturbation bound that proves the desired bound, where classic perturbation bounds fail. The effectiveness and practical merits of the proposed framework are demonstrated through experiments on both simulated and real datasets. Lastly, we discuss generalizations of our approach to higher-order data.

Study on advanced partial least squares for quality-related fault detection

The issue of quality-related fault detection in the industrial process has attracted much attention in recent years. The partial least squares (PLS) is considered an efficient tool for predicting and monitoring. The modified partial least squares (MPLS) is an extended algorithm for solving the oblique decomposition of PLS, however, the study indicated that the loss of quality variable information may affect the prediction of quality information in the decomposition process of the MPLS algorithm. Furthermore, the detection rate of traditional statistics and static control limit is low, and the existing dynamic control limit has certain limitations. Therefore, a new PLS space-decomposition algorithm called advanced partial least squares (APLS) is proposed. APLS avoids the loss of quality information by orthogonal decomposition of process variables according to their relationship with quality. APLS has a more accurate prediction of quality when process variables contain more noise; the fault false alarm rates (FAR) of quality-related faults are reduced by using the new statistics and thresholds combined with local information increment technology in the process variable principal component subspace. Finally, the effectiveness of the proposed approach is verified by a numerical example and an industrial benchmark problem.

Development of an Adaptive Artifact Subspace Reconstruction Based on Hebbian/Anti-Hebbian Learning Networks for Enhancing BCI Performance.

Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications.

Online unsupervised monitoring for hob wear based on multi-domain vibration features extraction and improved Q-statistic control chart

Current mainstream wear diagnosis approaches rely on the entire lifecycle wear-labeled datasets, which is costly, delayed, and low-universality. To achieve the monitoring goal of online, real-time, and strong generalization for hob wear and get rid of the dependence on the wear datasets, an online unsupervised monitoring method is explored based on the multi-domain features extraction and improved Q-statistic control chart, which is progressive. First, the monitoring feature matrix of each hob shifting period is transformed into the principal component subspace, the wear subspace, and the interference subspace based on the principal component analysis (PCA). Then, features in the wear subspace of the previous shifting period are used to get the control limit and those of the next shifting period are used to calculate the Q-statistic. The hob wear status is determined by comparing the control limit and the Q-statistic. The gear hobbing experiment verifies the feasibility of the proposed method.

Jet schemes, quantum dilogarithm and Feigin-Stoyanovsky's principal subspaces

We analyze the structure of Feigin-Stoyanovsky's principal subspaces of affine Lie algebra from the jet algebra viewpoint. For type A level one principal subspaces, we show that their shifted multi-graded Hilbert series can be expressed either using the quantum dilogarithm or as certain generating functions “counting” finite-dimensional representations of A-type quivers. This notably results in novel fermionic character formulas for these principal subspaces. Moreover, our result implies that all level one principal subspaces of type A are “classically free” as vertex algebras.We also analyze infinite jet algebras associated to principal subspaces of affine vertex algebras L1(so5), L1(so8) and L1(g2). We derive a new character formula for the principal subspace of L1(so5), proving that it is classically free, and present evidence that the principal subspaces of L1(so8) and of L1(g2) are also classically free.

Supervised Principal Component Regression for Functional Responses with High Dimensional Predictors

We propose a supervised principal component regression method for relating functional responses with high-dimensional predictors. Unlike the conventional principal component analysis, the proposed method builds on a newly defined expected integrated residual sum of squares, which directly makes use of the association between the functional response and the predictors. Minimizing the integrated residual sum of squares gives the supervised principal components, which is equivalent to solving a sequence of nonconvex generalized Rayleigh quotient optimization problems. We reformulate the nonconvex optimization problems into a simultaneous linear regression with a sparse penalty to deal with high dimensional predictors. Theoretically, we show that the reformulated regression problem can recover the same supervised principal subspace under certain conditions. Statistically, we establish nonasymptotic error bounds for the proposed estimators when the covariate covariance is bandable. We demonstrate the advantages of the proposed method through numerical experiments and an application to the Human Connectome Project fMRI data. Supplementary materials for this article are available online.

Learning Feature-Sparse Principal Subspace.

The principal subspace estimation is directly connected to dimension reduction and is important when there is more than one principal component of interest. In this article, we introduce two new algorithms to solve the feature-sparsity constrained PCA problem (FSPCA) for the principal subspace estimation task, which performs feature selection and PCA simultaneously. Existing optimization methods for FSPCA require data distribution assumptions and are lack of global convergence guarantee. Though the general FSPCA problem is NP-hard, we show that, for a low-rank covariance, FSPCA can be solved globally (Algorithm 1). Then, we propose another strategy (Algorithm 2) to solve FSPCA for the general covariance by iteratively building a carefully designed proxy. We prove (data-dependent) approximation bound and regular stationary convergence guarantees for the new algorithms. For the spectrum of covariance with exponential/Zipf's distribution, we provide exponential/posynomial approximation bounds. Constructive examples and numerical results are provided to demonstrate the tightness of our results. Experimental results show the promising performance and efficiency of the new algorithms compared with the state-of-the-arts on both synthetic and real-world datasets.

On the optimality of the Oja’s algorithm for online PCA

In this paper we analyze the behavior of the Oja’s algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.

Sampling theorem and reconstruction formula for the space of translates on the Heisenberg group

The paper deals with the necessary and sufficient conditions for obtaining reconstruction formulae and sampling theorems for every function belonging to the principal shift invariant subspace of $ L^2(\mathbb{H}^n) $, both in the time domain and a transform domain, where $ \mathbb{H}^n $ denotes the Heisenberg group.

Bayesian Sparse Spiked Covariance Model with a Continuous Matrix Shrinkage Prior

We propose a Bayesian methodology for estimating spiked covariance matrices with a jointly sparse structure in high dimensions. The spiked covariance matrix is reparameterized in terms of the latent factor model, where the loading matrix is equipped with a novel matrix spike-and-slab LASSO prior, which is a continuous shrinkage prior for modeling jointly sparse matrices. We establish the rate-optimal posterior contraction for the covariance matrix with respect to the spectral norm as well as that for the principal subspace with respect to the projection spectral norm loss. We also study the posterior contraction rate of the principal subspace with respect to the two-to-infinity norm loss, a novel loss function measuring the distance between subspaces that is able to capture entrywise eigenvector perturbations. We show that the posterior contraction rate with respect to the two-to-infinity norm loss is tighter than that with respect to the routinely used projection spectral norm loss under certain low-rank and bounded coherence conditions. In addition, a point estimator for the principal subspace is proposed with the rate-optimal risk bound with respect to the projection spectral norm loss. The numerical performance of the proposed methodology is assessed through synthetic examples and the analysis of a real-world face data example.

Vertex algebraic construction of modules for twisted affine Lie algebras of type [formula omitted

Let g˜ be the affine Lie algebra of type A2l(2). The integrable highest weight g˜-module L(kΛ0) called the standard g˜-module is realized by a tensor product of the twisted module VLT for the lattice vertex operator algebra VL. By using such vertex algebraic construction, we construct bases of the standard module, its principal subspace and the parafermionic space. As a consequence, we obtain their character formulas and settle the conjecture for vacuum modules stated in [15].

A Novel Background Modeling Algorithm for Hyperspectral Ground-Based Surveillance and Through-Foliage Detection.

Foliage penetration is an unsolved important part of border surveillance of remote areas between regular border crossing points. Detecting penetrating objects (e.g., persons and cars) through dense foliage in various climate conditions using visual sensors is prone to high fault rates. Through-foliage scenarios contain an unprecedented amount of occlusion—in fact, they often contain fragmented occlusion (for example, looking through the branches of a tree). Current state-of-the-art detectors based on deep learning perform inadequately under moderate-to-heavy fragmented occlusion. The FOLDOUT project builds a system that combines various sensors and technologies to tackle this problem. Consequently, a hyperspectral sensor was investigated due to its extended spectral bandwidth, beyond the range of typical RGB sensors, where vegetation exhibits pronounced reflectance. Due to the poor performance of deep learning approaches in through-foliage scenarios, a novel background modeling-based detection approach was developed, dedicated to the characteristics of the hyperspectral sensor, namely strong correlations between adjacent spectral bands and high redundancy. The algorithm is based on local dimensional reduction, where the principal subspace of each pixel is maintained and adapted individually over time. The successful application of the proposed algorithm is demonstrated in a through-foliage scenario comprised of heavy fragmented occlusion and a highly dynamical background, where state-of-the-art deep learning detectors perform poorly.

Principal Subspace of Dynamic Functional Connectivity for Diagnosis of Autism Spectrum Disorder

The study of functional connectivity (FC) of the brain using resting-state functional magnetic resonance imaging (rs-fMRI) has gained traction for uncovering FC patterns related to autism spectrum disorder (ASD). It is believed that the neurodynamic components of neuroimaging data enhance the measurement of the FC of brain nodes. Hence, methods based on linear correlations of rs-fMRI may not accurately represent the FC patterns of brain nodes in ASD patients. In this study, we proposed a new biomarker for ASD detection based on wavelet coherence and singular value decomposition. In essence, the proposed method provides a novel feature-vector based on extraction of the principal component of the neuronal dynamic FC patterns of rs-fMRI BOLD signals. The method, known as principal wavelet coherence (PWC), is implemented by applying singular value decomposition (SVD) on wavelet coherence (WC) and extracting the first principal component. ASD biomarkers are selected by analyzing the relationship between ASD severity scores and the amplitude of wavelet coherence fluctuation (WCF). The experimental rs-fMRI dataset is obtained from the publicly available Autism Brain Image Data Exchange (ABIDE), and includes 505 ASD patients and 530 normal control subjects. The data are randomly divided into 90% for training and cross-validation and the remaining 10% unseen data used for testing the performance of the trained network. With 95.2% accuracy on the ABIDE database, our ASD classification technique has better performance than previous methods. The results of this study illustrate the potential of PWC in representing FC dynamics between brain nodes and opens up possibilities for its clinical application in diagnosis of other neuropsychiatric disorders.

Linear Inverse Modeling of Large-Scale Atmospheric Flow Using Optimal Mode Decomposition

Abstract Linear inverse modeling or principal oscillation pattern (POP) analysis is a widely applied tool in climate science for extracting from data dominant spatial patterns together with their dynamics as approximated by a linear Markov model. The system is projected onto a principal linear subspace and the system matrix is estimated from data. The eigenmodes of the system matrix are the POPs, with the eigenvalues providing their decay time scales and oscillation frequencies. Usually, the subspace is spanned by the leading principal components (PCs) and empirical orthogonal functions (EOFs). Outside of climate science, this procedure is now more commonly referred to as dynamic mode decomposition (DMD). Here, we use optimal mode decomposition (OMD) to address the full linear inverse modeling problem of simultaneous optimization of the principal subspace and the linear operator. The method is illustrated on two pedagogical examples and then applied to a three-level quasigeostrophic atmospheric model with realistic mean state and variability. The OMD models significantly outperform the EOF/DMD models in predicting the time evolution of the large-scale flow modes. The advantage of the OMD models stems from finding more persistent modes as well as from better capturing the nonnormality of the linear operator and the associated nonmodal growth. The dynamics of the large-scale flow modes turn out to be markedly non-Markovian and the OMD modes are superior to the EOF/DMD modes also in a modeling setting with a higher-order vector autoregressive process. The OMD modes could also be used as basis functions for a nonlinear dynamical model although they are not optimized for that purpose. Potential applications of the OMD method in weather and climate science include ENSO or MJO prediction, reduced-rank data assimilation, and generation of initial perturbations for ensemble prediction.

High-dimensional, slow-time-varying process monitoring technique based on adaptive eigen subspace extraction method

In this paper, in order to monitor the slow-time-varying industrial process, an adaptive method is proposed based on the neural network model and fault reconstruction method. Firstly, a unified neural network algorithm is introduced to extract the principal and minor eigen subspace with low computational complexity, and the whole eigenspace is divided into three partitions to further reduce the complexity of high-dimensional data computation. Then, the process is monitored based on a combined statistic index and the corresponding adaptive threshold. Moreover, the eigen subspace can still be updated even when in a faulty case. Finally, computer simulation confirms the capacity of the proposed method for high-dimensional, slow-time-varying process monitoring.

Building clustering for regional seismic response and damage analysis

This article presents a framework to cluster buildings into typologically similar groups and select indicator buildings for regional seismic response and damage analysis. The framework requires a robust database of buildings to provide high-level structural and site information of buildings. Here, a database of 234 reinforced concrete buildings with five or more above-ground stories in the central business district of Wellington, New Zealand, has been selected as the case study of this research. First, key structural and site parameters that contribute to the seismic demand, response, and damage of each building are extracted from the database. Extracted parameters comprise three numerical and five categorical attributes of each building, including the year of construction, height, period, lateral load resisting system, floor system, site subsoil class, importance level, and strong motion station. Next, two prominent unsupervised machine learning clustering approaches are utilized to cluster the mixed categorical and numerical building database: k-prototype on the mixed numerical and categorical database and k-means on principal components numerical subspace adopted from factor analysis of mixed data (FAMD). A novel autoencoder deep learning neural network is also designed and trained to convert the mixed data into a low-dimensional subspace called latent space and feed this into k-means for clustering. The proposed autoencoder method is demonstrated to be more effective at clustering buildings into useful typological clusters for seismic response and damage analysis based on multiple criteria from both data-science and engineering perspectives. The details of selected indicator buildings for each similar seismic vulnerability cluster are also represented.

Fault Diagnosis Method for Rolling Bearing Based on Sparse Principal Subspace Discriminant Analysis

Rolling bearings are omnipresent parts in industrial fields. To comprehensively reflect the status of rolling bearing and improve the classification accuracy, fusion information is widely used in various studies, which may result in high dimensionality, redundancy information of dataset, and time consumption. Thus, it is of crucial significance in extracting optimal features from high-dimensional and redundant feature space for classification. In this study, a fault diagnosis of rolling bearings model based on sparse principal subspace discriminant analysis is proposed. It extracts sparse discrimination information, meanwhile preserving the main energy of original dataset, and the sparse regularization term and sparse error term constrained by l2,1-norm are introduced to improve the performance of feature extraction and the robustness to noise and outliers. The multi-domain feature space involved a time domain, frequency domain, and time-frequency domain is first derived from the original vibration signals. Then, the intrinsic geometric features extracted by sparse principal subspace discriminant analysis are fed into a support vector machine classifier to recognize different operating conditions of bearings. The experimental results demonstrated that the feasibility and effectiveness of the proposed fault diagnosis model based on a sparse principal subspace discriminant analysis algorithm can achieve higher recognition accuracy than fisher discriminant analysis and its extensions, and it is relatively insensitive to the impact of noise and outliers owing to the sparse property.

Communication-Efficient Decentralized Subspace Estimation

Machine learning techniques have been widely used in communication systems because of their superior performance. Among them, the classical machine learning algorithm Principal Component Analysis (PCA) aims to estimate the principal subspace of the received signals, and thus is also known as subspace estimation. It is often used in Direction of Arrival (DoA) tasks. Recently, the widespread deployment of networks has motivated researchers to solve the problem of subspace estimation in decentralized settings. An important concern in the design of decentralized algorithms is the communication complexity, because communication consumes much energy, while the sensors in the network are generally energy-limited. Specifically, communication complexity refers to the minimum amount of transmitted variables to obtain an estimate with an error smaller than <inline-formula><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula>. For existing algorithms, this complexity is <inline-formula><tex-math notation="LaTeX">$O(\log ^{2}(1/\varepsilon))$</tex-math></inline-formula>. In this paper, we improve this complexity to be <inline-formula><tex-math notation="LaTeX">$O(\log (1/\varepsilon))$</tex-math></inline-formula> by designing a new algorithm named Decentralized Subspace Estimation (DSE). Such improvement is achieved by using the gradient tracking technique. We theoretically analyze the influence of the network connectivity and the eigen-gap of the data on communication complexity. In addition, we use DSE to solve the DoA estimation problem. Experiments verify the effectiveness of the algorithm and the correctness of the theoretical result.

High dimensional reliability analysis based on combinations of adaptive Kriging and dimension reduction technique

AbstractHigh dimensional reliability analysis is unavoidable if a structural system involving stochastic process because Karhunen–Loève (K‐L) expansion is commonly used. Reliability analysis for structural systems with computationally intensive numerical models and high dimensions is challenging. In this study, an effective high dimensional reliability analysis method is proposed based on principal components and active subspace (PCAS) and active Kriging, and is termed as PCAS‐AK. The proposed method can address two shortcomings existing in PCAS: the latter select training samples randomly and cannot deal with parallel processing. The proposed method, however, selects training samples adaptively, subsequently, the training samples that have high contribution to probability of failure are selected, which can improve computational efficiency. To reduce overall computational time and allow parallelization, K‐weighted‐means combined with learning function are used to select multiple samples at each iteration. After the dimension reduction of both input and output spaces, the samples selected by adaptive method are used to construct Kriging model in the latent low‐dimensional space. Three examples are used to demonstrate the applicability and accuracy of the proposed method. Results show that the proposed method can deal with parallel processing and is, generally, more effective than PCAS for high dimensional reliability problems.