Dynamic Principal Component Analysis Research Articles

Background subtraction in dynamic scenes using the dynamic principal component analysis

This study presents a foreground detection method capable of robustly estimating the background under the presence of dynamic effects. The key contribution of this study is the use of the dynamic principal component analysis to model the serial correlation between successive frames and construct a robust pixel-based background model. The frames are normalised in hue, saturation and value colour space to reduce the effect of illumination changes. To restrict the background model, kernel density estimation is used to identify the distribution of the background time-lagged data matrix and then confidence interval limits are used to determine the corresponding detection thresholds. The foreground is detected using background subtraction. This method is tested on several common sequences such as CDnet 2014, ETSI 2014 and MULTIVISION 2013. The authors also hold comparisons based on quantitative metrics with several state-of-the-art methods. Experimental results show that their method outperforms some state-of-the-art methods and has comparable performance with some depth-based methods.

Detection of sensor precision degradation by monitoring second-order statistics

For industrial processes, there are usually a number of measurement sensors equipped for monitoring and control purposes. In practice, sensors may suffer from the precision degradation phenomenon due to several aspects such as aging and ambient interference. This phenomenon may lead to imprecise or even incorrect control commands and indications, so the corresponding fault detection task is of vital importance. In this paper, inspired by the fact that precision degradation of a sensor can result in the increase of the variable’s variance, an algorithm based on second-order statistics analysis is proposed to accomplish the detection task for sensor precision degradation faults. By employing the sliding window technique, second-order statistics of process variables are first extracted. Then, conventional principal component analysis (PCA) is used as a dissimilarity quantification tool, with detection statistics and corresponding control limits established, to perform fault detection. Finally, simulations on a numerical example and the continuous stirred tank reactor (CSTR) benchmark process are performed to illustrate the effectiveness and advantages of the proposed method, in comparison with some existing methods such as PCA, dynamic PCA, and dissimilarity (DISSIM).

Fault Detection in the Tennessee Eastman Benchmark Process Using Principal Component Difference Based on K-Nearest Neighbors

Industrial data usually have nonlinear or multimodal characteristics which do not meet the data assumptions of statistics in principal component analysis (PCA). Therefore, PCA has a lower fault detection rate in industrial processes. Aiming at the above limitations of PCA, a fault detection method using principal component difference based on k-nearest neighbors (Diff-PCA) is proposed in this paper. First, find the k nearest neighbors set of each sample in the training data set and calculate its mean vector. Second, build an augmented vector using each sample and its corresponding mean vector. Third, calculate the loading matrix and score matrix using PCA. Next, calculate the estimated scores using the mean vector of each sample and missing data imputation technique for PCA. At last, build two new statistics using the difference between the real scores and estimated scores to detect faults. In addition, the fault diagnosis method based on contribution plots of monitored variables is also proposed in this paper. In Diff-PCA, the difference skill can eliminate the impact of the nonlinear and multimodal structure on fault detection. Meanwhile, the monitored subspaces by the two new statistics are different from that by T2 and SPE in PCA. The efficiency of the proposed strategy is implemented in two numerical cases (nonlinear and multimode) and the Tennessee Eastman (TE) processes. The fault detection results indicate that Diff-PCA outperforms the conventional PCA, Kernel PCA, dynamic PCA, principal component-based k nearest neighbor rule and k nearest neighbor rule.

Gearbox Incipient Fault Detection Based on Deep Recursive Dynamic Principal Component Analysis

As a part of the energy transmission chain, gearboxes are considered as important components in rotating machines, and the gearbox failure results in costly economic losses. Therefore, it is necessary to detect the appearance of incipient gearbox faults by implementing an appropriate detected model. The incipient failure characteristics of the gearbox are weak and hidden in a set of time-varying series signals the vibration signals, which is difficult to effectively extract under the background of strong noise. The PCA method is not effective in detecting weak fault features in time-varying signals, so this paper proposes a method based on Deep Recursive Dynamic Principal Component Analysis (Deep RDPCA) to detect incipient faults in gearboxes. The proposed approach is modeled via both the deep decomposed theorems and time-varying dynamic model based on traditional PCA to extract characteristic of time-varying and weak fault information under the background of strong noise. The proposed method could get a better real-time reflection for changed system by introducing “Moving Window” technologies, so that the incipient fault of gearbox could be detected accurately, too. Finally, the effect of Deep RDPCA-based fault diagnosis is compared with the results of PCA, DPCA, RDPCA, Deep PCA, and Deep DPCA methods. It is concluded that the proposed method can effectively capture the time-varying relationship of process variables and accurately extract the weak fault characteristics in the vibration signal, which effectively improves the fault detection performance.

Abnormal node detection method for time-sharing heating energy consumption in multi-storey buildings based on drosophila algorithm

In order to solve the problem of high energy consumption of time-sharing heating in multi-storey buildings caused by abnormal node, it is necessary to study the detection method of abnormal node of heating energy consumption. Based on fruit flies algorithm of multi-storey building time-sharing heating energy consumption of the abnormal node detection method, using synchronous sequential PLD multi-storey building heating system of heating time-sharing data, put optimisation algorithm is introduced into EEMD algorithm, noise by flies optimisation calculation the corresponding threshold, remove the noise, the IMF component complete data denoising processing. Based on the recursive dynamic principal component analysis method, the detection model of abnormal heating energy consumption nodes is constructed to realise the detection of abnormal heating energy consumption nodes in multi-storey buildings. Experimental results show that the proposed method has good denoising effect, low false alarm rate and high detection rate.

Adaptive-learning model predictive control for complex physiological systems: Automated insulin delivery in diabetes

An adaptive-learning model predictive control (AL-MPC) framework is proposed for incorporating disturbance prediction, model uncertainty quantification, pattern learning, and recursive subspace identification for use in controlling complex dynamic systems with periodically recurring large random disturbances. The AL-MPC integrates online learning from historical data to predict the future evolution of the model output over a specified horizon and proactively mitigate significant disturbances. This goal is accomplished using dynamic regularized latent variable regression (DrLVR) approach to quantify disturbances from the past data and forecast their future progression time series. An enveloped path for the future behavior of the model output is extracted to further enhance the robustness of the closed-loop system. The controller set-point, penalty weights of the objective function, and constraints criteria can be modified in advance for the expected periods of the disturbance effects. The proposed AL-MPC is used to regulate glucose concentration in people with Type 1 diabetes by an automated insulin delivery system. Simulation results demonstrate the effectiveness of the proposed technique by improving the performance indices of the closed-loop system. The MPC algorithm integrated with DrLVR disturbance predictor has compared to MPC reinforced with dynamic principal component analysis linked with K-nearest neighbors and hyper-spherical clustering (k-means) technique. The simulation results illustrate that the AL-MPC can regulate the glucose concentrations of people with Type 1 diabetes to stay in the desired range (70–180) mg/dL 84.4% of the time without causing any hypoglycemia and hyperglycemia events.

A multi-feature extraction technique based on principal component analysis for nonlinear dynamic process monitoring

Principal component analysis (PCA) and its modified methods have been widely applied in industrial process monitoring. In practice, industrial processes are with disparate characteristics, the process monitoring system should consider as many process characteristics as possible, such as dynamic and nonlinear characteristics. In this paper, a multi-feature extraction technique based on PCA is proposed for nonlinear dynamic process monitoring. The proposed method integrates dynamic inner PCA (DiPCA), PCA and kernel PCA (KPCA) methods through a serial structure to extract the dynamic, linear and nonlinear features among the process data. Along with the proposed method, the original data space is decomposed into several orthogonal subspaces, in which abnormal variations of different features can be monitored. For real-time process monitoring, a combined Hotelling’s T2 statistic based on the extracted multi-feature and a squared prediction error (SPE or Q) statistic are established. Case studies on a numerical example and the Tennessee Eastman process are carried out to demonstrate the superior process monitoring performance of the proposed method compared with other relevant methods.

Dynamic principal component analysis with missing values

Dynamic principal component analysis (DPCA), also known as frequency domain principal component analysis, has been developed by Brillinger [Time Series: Data Analysis and Theory, Vol. 36, SIAM, 1981] to decompose multivariate time-series data into a few principal component series. A primary advantage of DPCA is its capability of extracting essential components from the data by reflecting the serial dependence of them. It is also used to estimate the common component in a dynamic factor model, which is frequently used in econometrics. However, its beneficial property cannot be utilized when missing values are present, which should not be simply ignored when estimating the spectral density matrix in the DPCA procedure. Based on a novel combination of conventional DPCA and self-consistency concept, we propose a DPCA method when missing values are present. We demonstrate the advantage of the proposed method over some existing imputation methods through the Monte Carlo experiments and real data analysis.

A Novel Dynamic Weight Principal Component Analysis Method and Hierarchical Monitoring Strategy for Process Fault Detection and Diagnosis

Traditional monitoring algorithms use the normal data for modeling, which are universal for different types of faults. However, these algorithms may perform poorly sometimes because of the lack of fault information. In order to further increase the fault detection rate while preserving the universality of the algorithm, a novel dynamic weight principal component analysis (DWPCA) algorithm and a hierarchical monitoring strategy are proposed. In the first layer, the dynamic PCA is used for fault detection and diagnosis, if no fault is detected, the following DWPCA-based second layer monitoring will be triggered. In the second layer, the principal components (PCs) are weighted according to its ability in distinguishing between the normal and fault conditions, then the PCs which own larger weight are selected to construct the monitoring model. Compared to the DPCA method, the proposed DWPCA algorithm establishes the monitoring model by combining the information of fault. Afterward, the DWPCA-based variable relative contribution and a novel control limit for the variable relative contribution are presented for the fault diagnosis. Finally, the superiority of the proposed method is demonstrated by a numerical case and the Tennessee Eastman process.

Remaining useful life prediction of lubricating oil with dynamic principal component analysis and proportional hazards model

Lubricating oil contains a lot of tribological information of the machine and plays an important role in machine health. Oil degrades with serving time and causes severe wear afterwards, which is a complex dynamic process, and difficult to be accurately described by a single property. Therefore, the main purpose of deterioration prediction is to estimate the remaining useful life that the oil can still fulfill its functions by analyzing oil condition monitoring data. With a large amount of oil condition monitoring data collected, a vector autoregressive model is applied to the original oil data to describe the dynamic deterioration process. Then dynamic principal component analysis, an effective dimensionality reduction method, is employed to obtain the principal components capturing the most information of the oil data. The proportional hazards model is then built to calculate the failure risk of the lubricating oil based on the condition monitoring information, where its baseline function represents the aging process assuming to follow the Weibull distribution and its positive link function represents the influence of covariates (the principal components) on the failure risk. Finally, the remaining useful life prediction of lubricating oil can be obtained by explicit formulas of the characteristics such as the conditional reliability function and the mean residual life function. This work provides an approach to assess the health of lubricating oil, and a guidance for oil maintenance strategy.

Pattern recognition in tornado-like vortices at different swirl ratios

A new paper examines the dynamic structure of tornado-like vortices at different swirl ratios using a novel dynamic principal component analysis technique.

DYNAMIC PRINCIPAL COMPONENT REGRESSION: APPLICATION TO AGE-SPECIFIC MORTALITY FORECASTING

AbstractIn areas of application, including actuarial science and demography, it is increasingly common to consider a time series of curves; an example of this is age-specific mortality rates observed over a period of years. Given that age can be treated as a discrete or continuous variable, a dimension reduction technique, such as principal component analysis (PCA), is often implemented. However, in the presence of moderate-to-strong temporal dependence, static PCA commonly used for analyzing independent and identically distributed data may not be adequate. As an alternative, we consider a dynamic principal component approach to model temporal dependence in a time series of curves. Inspired by Brillinger’s (1974, Time Series: Data Analysis and Theory. New York: Holt, Rinehart and Winston) theory of dynamic principal components, we introduce a dynamic PCA, which is based on eigen decomposition of estimated long-run covariance. Through a series of empirical applications, we demonstrate the potential improvement of 1-year-ahead point and interval forecast accuracies that the dynamic principal component regression entails when compared with the static counterpart.

Provable Dynamic Robust PCA or Robust Subspace Tracking

Dynamic robust PCA refers to the dynamic (time-varying) extension of robust PCA (RPCA). It assumes that the true (uncorrupted) data lies in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. We develop and study a novel algorithm, that we call simple-ReProCS, based on the recently introduced Recursive Projected Compressive Sensing (ReProCS) framework. Our work provides the first guarantee for dynamic RPCA that holds under weakened versions of standard RPCA assumptions, slow subspace change and a lower bound assumption on most outlier magnitudes. Our result is significant because (i) it removes the strong assumptions needed by the two previous complete guarantees for ReProCS-based algorithms; (ii) it shows that it is possible to achieve significantly improved outlier tolerance, compared with all existing RPCA or dynamic RPCA solutions by exploiting the above two simple extra assumptions; and (iii) it proves that simple-ReProCS is online (after initialization), fast, and, has near-optimal memory complexity.

Performance Analysis of Dynamic PCA for Closed-Loop Process Monitoring and Its Improvement by Output Oversampling Scheme

The performance of dynamic principal component analysis (DPCA)-based fault detection and diagnosis in a closed-loop system is studied and its improvement by the output oversampling scheme is proposed in this paper. By the subspace decomposition technique, DPCA with the closed-loop data for fault detection does not perform better than DPCA with the open-loop data. Moreover, using fault reconstruction based on DPCA to determine the root cause would also become invalid in the closed loop. To eliminate the adverse effect of feedback control on the performance of the DPCA model, a new algorithm that directly constructs DPCA based on the closed-loop data is investigated using the output oversampled data without excitations in the reference signals. The associated enhanced characteristics of the sampled data in the output oversampling scheme are analyzed. A simulated continuous stirred tank heater illustrates that the proposed algorithm can significantly improve the DPCA performance of process monitoring and fault reconstruction in closed-loop systems.

Degradation Data-Driven Time-To-Failure Prognostics Approach for Rolling Element Bearings in Electrical Machines

Time-to-failure (TTF) prognostic plays a crucial role in predicting remaining lifetime of electrical machines for improving machinery health management. This paper presents a novel three-step degradation data-driven TTF prognostics approach for rolling element bearings (REBs) in electrical machines. In the degradation feature extraction step, multiple degradation features, including statistical features, intrinsic energy features, and fault frequency features, are extracted to detect the degradation phenomenon of REBs using complete ensemble empirical mode decomposition with adaptive noise and Hilbert–Huang transform methods. In degradation feature reduction step, the degradation features, which are monotonic, robust, and correlative to the fault evolution of the REBs, are selected and fused into a principal component Mahalanobis distance health index using dynamic principal component analysis and Mahalanobis distance methods. In TTF prediction step, the degradation process and local TTF of the REBs are observed by an exponential regression-based local degradation model, and the global TTF is predicted by an empirical Bayesian algorithm with a continuous update. A practical case study involving run-to-failure experiments of REBs on PRONOSTIA platform is provided to validate the effectiveness of the proposed approach and to show a more accurate prediction of TTF than the existing major approaches.

Classification of Profit-Based Operating Regions for the Tennessee Eastman Process using Deep Learning Methods

Abstract The focus of this work is on the classification of an input-design space into different regions of input conditions that result in different corresponding ranges of productivity costs in the Tennessee Eastman Process (TEP). Although similar classification tasks had been previously carried out using linear multivariate statistical data analysis methods, these are of limited efficacy when dealing with highly non-linear dynamics. In this work, we present two Deep Learning Tools for classification: using either supervised learning or un-supervised learning. For classification with supervised learning a Recurrent Neural Network (RNN) known as Long Short-Term Memory (LSTM) is trained on normalized training data. Since deep learning networks generally involve a large number of nodes and parameters an algorithm named Sequential Layer-wise Relevance Propagation (SLRPFP) is proposed for selecting the relevant inputs and for pruning the LSTM network such that the test accuracy at each step is maintained or even improved. For classification with unsupervised learning, main features from the input dataset are first extracted using an Autoencoder. Then a Multi-dimensional Support Vector Machines (MSVM) model is applied to the features identified by the autoencoder. The performance of the proposed supervised and unsupervised deep learning approaches are compared to an approach that combines linear Dynamic Principal Component Analysis (DPCA) and a MSVM based classification and conclusions are drawn on the relative advantages of the deep learning methods.

Time-Varying General Dynamic Factor Models and the Measurement of Financial Connectedness

We propose a new time-varying Generalized Dynamic Factor Model for high-dimensional, locally stationary time series. Estimation is based on dynamic principal component analysis jointly with singular VAR estimation, and extends to the locally stationary case the one-sided estimation method proposed by Forni et al. (2017) for stationary data. We prove consistency of our estimators of time-varying impulse response functions as both the sample size T and the dimension n of the time series grow to infinity. This approach is used in an empirical application in order to construct a time-varying measure of financial connectedness for a large panel of adjusted intra-day log ranges of stocks. We show that large increases in long-run connectedness are associated with the main financial turmoils. Moreover, we provide evidence of a significant heterogeneity in the dynamic responses to common shocks in time and over different scales, as well as across industrial sectors.

Fault detection method based on principal component difference associated with DPCA

AbstractAiming at the fault detection in a dynamic process with nonlinear or multimodal features, a new fault detection strategy based on principal component difference associated with dynamic PCA (Diff‐DPCA) is proposed in this paper. First, augment the training data set using the time lag, and then obtain an augmented training data set. Second, find the K‐nearest neighbor set of each sample in the augmented training set and calculate the mean of the K nearest neighbor set. Third, calculate the loading and score matrices of the augmented training data set using PCA, and then calculate the scores of the proposed mean above using the loading matrix. Next, calculate the difference between the scores of a sample and its corresponding mean. Finally, 2 new statistics are built in difference subspace and residual subspace respectively. Principal component difference associated with dynamic PCA, which inherit the ability of DPCA monitoring a dynamic process, can improve the fault detection rate in the process with multimodal or nonlinear characteristics; meanwhile, the new statistics present low autocorrelation levels. The proposed method in this paper and other traditional methods are applied to detect faults in 2 cases and the Tennessee Eastman process, such as PCA, DPCA, and FD‐KNN. The experimental results indicate that the proposed method outperforms the conventional PCA, DPCA, and FD‐KNN.

Identification of Errors-in-Variables Models Using Dynamic Iterative Principal Component Analysis

Identification of linear dynamic systems from input–output data has been a subject of study for several decades. A broad class of problems in this field pertain to what is widely known as the errors-in-variables (EIV) class, where both input and output are known with errors, in contrast to the traditional scenario where only outputs are assumed to be corrupted with errors. In this work, we present a novel and systematic approach to the identification of dynamic models for the EIV case in the principal component analysis (PCA) framework. The key contribution of this work is a dynamic iterative PCA (DIPCA) algorithm that has the ability to determine the correct order of the process, handle unequal measurement noise variances (the heteroskedastic case), and provide accurate estimates of the model together with the noise covariance matrix under large sample conditions. The work is presented within the scope of single-input, single-output (SISO) deterministic systems corrupted by white-noise errors. Simulation...

Static and Dynamic Robust PCA and Matrix Completion: A Review

Principal Components Analysis (PCA) is one of the most widely used dimension reduction techniques. Robust PCA (RPCA) refers to the problem of PCA when the data may be corrupted by outliers. Recent work by Cand{\`e}s, Wright, Li, and Ma defined RPCA as a problem of decomposing a given data matrix into the sum of a low-rank matrix (true data) and a sparse matrix (outliers). The column space of the low-rank matrix then gives the PCA solution. This simple definition has lead to a large amount of interesting new work on provably correct, fast, and practical solutions to RPCA. More recently, the dynamic (time-varying) version of the RPCA problem has been studied and a series of provably correct, fast, and memory efficient tracking solutions have been proposed. Dynamic RPCA (or robust subspace tracking) is the problem of tracking data lying in a (slowly) changing subspace while being robust to sparse outliers. This article provides an exhaustive review of the last decade of literature on RPCA and its dynamic counterpart (robust subspace tracking), along with describing their theoretical guarantees, discussing the pros and cons of various approaches, and providing empirical comparisons of performance and speed. A brief overview of the (low-rank) matrix completion literature is also provided (the focus is on works not discussed in other recent reviews). This refers to the problem of completing a low-rank matrix when only a subset of its entries are observed. It can be interpreted as a simpler special case of RPCA in which the indices of the outlier corrupted entries are known.