Static Principal Component Analysis Research Articles

Dynamic Weighted Canonical Correlation Analysis for Auto-Regressive Modeling

Canonical correlation analysis (CCA) is widely used as a supervised learning method to extract correlations between process and quality datasets. When used to extract relations between current data and historical data, CCA can also be regarded as an auto-regressive modeling method to capture dynamics. Various dynamic CCA algorithms were developed in the literature. However, these algorithms do not consider strong dependence existing in adjacent samples, which may lead to unnecessarily large time lags and inaccurate estimation of current values from historical data. In this paper, a dynamic weighted CCA (DWCCA) algorithm is proposed to address this issue with a series of polynomial basis functions. DWCCA extracts dynamic relations by maximizing correlations between current data and a weighted representation of past data, and the weights rely only on a limited number of polynomial functions, which removes the negative effect caused by strongly collinear neighboring samples. After all the dynamics are exploited, static principal component analysis is then employed to further explore the cross-correlations in the dataset. The Tennessee Eastman process is utilized to demonstrate the effectiveness of the proposed DWCCA method in terms of prediction efficiency and collinearity handling.

Segmentation of Multivariate Industrial Time Series Data Based on Dynamic Latent Variable Predictability

Time series segmentation is an important vehicle of data mining and extensively applied in the areas of machine learning and anomaly detection. In real world tasks, dynamics widely exist in time series but have been little concerned. This paper proposes an algorithm which can partition a multivariate time series into subsequences at points where the dynamics change. In process industries, the change of dynamics often relates to operation regime change, working condition shifting or faults. Therefore, to segment time series according to change of dynamics can be useful in data preprocessing and getting deep insight of the process in various industrial process monitoring tasks. The proposed algorithm recursively merges neighborhood subsequences through a heuristic bottom-up procedure. The cost of merging is defined as the mutual predictability of the subsequence models which are constructed using the dynamic-inner principal component analysis algorithm. Then, the goal becomes finding the segmentation scheme which minimizes the averaged dynamic prediction errors of each subsequence model. The proposed algorithm is evaluated on both simulated data and the time series data collected from an industrial processing plant. The results show that it outperforms the static principal component analysis based methods.

Dynamic principal component regression for forecasting functional time series in a group structure

ABSTRACTWhen generating social policies and pricing annuity at national and subnational levels, it is essential both to forecast mortality accurately and ensure that forecasts at the subnational level add up to the forecasts at the national level. This has motivated recent developments in forecasting functional time series in a group structure, where static principal component analysis is used. In the presence of moderate to strong temporal dependence, static principal component analysis designed for independent and identically distributed functional data may be inadequate. Thus, through using the dynamic functional principal component analysis, we consider a functional time series forecasting method with static and dynamic principal component regression to forecast each series in a group structure. Through using the regional age-specific mortality rates in Japan obtained from the Japanese Mortality Database [(2019). National Institute of Population and Social Security Research. Available at http://www.ipss.go.jp/p-toukei/JMD/index-en.asp (data downloaded on 14 August 2018)], we investigate the point and interval forecast accuracies of our proposed extension, and subsequently make recommendations.

DYNAMIC PRINCIPAL COMPONENT REGRESSION: APPLICATION TO AGE-SPECIFIC MORTALITY FORECASTING

AbstractIn areas of application, including actuarial science and demography, it is increasingly common to consider a time series of curves; an example of this is age-specific mortality rates observed over a period of years. Given that age can be treated as a discrete or continuous variable, a dimension reduction technique, such as principal component analysis (PCA), is often implemented. However, in the presence of moderate-to-strong temporal dependence, static PCA commonly used for analyzing independent and identically distributed data may not be adequate. As an alternative, we consider a dynamic principal component approach to model temporal dependence in a time series of curves. Inspired by Brillinger’s (1974, Time Series: Data Analysis and Theory. New York: Holt, Rinehart and Winston) theory of dynamic principal components, we introduce a dynamic PCA, which is based on eigen decomposition of estimated long-run covariance. Through a series of empirical applications, we demonstrate the potential improvement of 1-year-ahead point and interval forecast accuracies that the dynamic principal component regression entails when compared with the static counterpart.

Advance warning for loss of separation in an industrial distillation column

AbstractThis paper describes the development of a data‐driven advance warning system for the onset of loss of separation in an industrial distillation column. The system would enable preventive actions to avoid several hours of bad operation and subsequent recovery of the process. Data of more than 2 years of process operation were used to identify and validate various monitoring systems based on both static principal component analysis (PCA) and dynamic PCA. Despite the presence of autocorrelation in the data, only minor differences in advance warning were observed between PCA and dynamic PCA.The developed system provides warnings for 35% to 45% of the observed periods of bad column operation, with respective advance warning times of 16 and 6 minutes. It proves a valuable additional tool to monitor the operation of the distillation column and avoid losses of product, with the potential of reducing bad operation (and the associated costs) by up to 45% and substantially improving overall process reliability.

Detection of Nucleation during Cooling Crystallization through Moving Window PCA Applied to in Situ Infrared Data

The method here proposed based on the moving window principal component analysis can be employed in time-varying processes tracked through in situ spectroscopy. In the case under investigation, it aimed to detect the nucleation during crystallization processes. For this purpose, statistical indexes were employed, and the contribution plot helped identifying the spectral variables that were changing due to nucleation. Isonicotinamide was here considered as a model active pharmaceutical ingredient, and its cooling crystallization was monitored by means of in situ infrared spectroscopy. The procedure allowed to overcome issues that may be encountered with static principal component analysis, since it could distinguish the slow-varying changes due to external perturbations (temperature) from abnormal events such as the sudden concentration decrease related to the crystallization. The proposed method was demonstrated to correctly detect nucleation without any a priori knowledge of the peaks involved in the pro...

A novel dynamic PCA algorithm for dynamic data modeling and process monitoring

Principal component analysis (PCA) has been widely applied for data modeling and process monitoring. However, it is not appropriate to directly apply PCA to data from a dynamic process, since PCA focuses on variance maximization only and pays no attention to whether the components contain dynamics or not. In this paper, a novel dynamic PCA (DiPCA) algorithm is proposed to extract explicitly a set of dynamic latent variables with which to capture the most dynamic variations in the data. After the dynamic variations are extracted, the residuals are essentially uncorrelated in time and static PCA can be applied. The new models generate a subspace of principal time series that are most predictable from their past data. Geometric properties are explored to give insight into the new dynamic model structure. For the purpose of process monitoring, fault detection indices based on DiPCA are developed based on the proposed model. Case studies on simulation data, data from an industrial boiler process, and the Tennessee Eastman process are presented to illustrate the effectiveness of the proposed dynamic models and fault detection methods.

A systematic comparison of PCA‐based Statistical Process Monitoring methods for high‐dimensional, time‐dependent Processes

High‐dimensional and time‐dependent data pose significant challenges to Statistical Process Monitoring. Most of the high‐dimensional methodologies to cope with these challenges rely on some form of Principal Component Analysis (PCA) model, usually classified as nonadaptive and adaptive. Nonadaptive methods include the static PCA approach and Dynamic Principal Component Analysis (DPCA) for data with autocorrelation. Methods, such as DPCA with Decorrelated Residuals, extend DPCA to further reduce the effects of autocorrelation and cross‐correlation on the monitoring statistics. Recursive Principal Component Analysis and Moving Window Principal Component Analysis, developed for nonstationary data, are adaptive. These fundamental methods will be systematically compared on high‐dimensional, time‐dependent processes (including the Tennessee Eastman benchmark process) to provide practitioners with guidelines for appropriate monitoring strategies and a sense of how they can be expected to perform. The selection of parameter values for the different methods is also discussed. Finally, the relevant challenges of modeling time‐dependent data are discussed, and areas of possible further research are highlighted. © 2016 American Institute of Chemical Engineers AIChE J, 62: 1478–1493, 2016

Loss-of-Main Monitoring and Detection for Distributed Generations Using Dynamic Principal Component Analysis

In this paper, our previous work on Principal Component Analysis (PCA) based fault detection method is extended to the dynamic monitoring and detection of loss-of-main in power systems using wide-area synchrophasor measurements. In the previous work, a static PCA model was built and verified to be capable of detecting and extracting system faulty events; however the false alarm rate is high. To address this problem, this paper uses a well-known ‘time lag shift’ method to include dynamic behavior of the PCA model based on the synchronized measurements from Phasor Measurement Units (PMU), which is named as the Dynamic Principal Component Analysis (DPCA). Compared with the static PCA approach as well as the traditional passive mechanisms of loss-of-main detection, the proposed DPCA procedure describes how the synchrophasors are linearly auto- and cross-correlated, based on conducting the singular value decomposition on the augmented time lagged synchrophasor matrix. Similar to the static PCA method, two statistics, namely T2 and Q with confidence limits are calculated to form intuitive charts for engineers or operators to monitor the loss-of-main situation in real time. The effectiveness of the proposed methodology is evaluated on the loss-of-main monitoring of a real system, where the historic data are recorded from PMUs installed in several locations in the UK/Ireland power system.

Forecasting the industrial production using alternative factor models and business survey data

This paper compares the forecasting performance of three alternative factor models based on business survey data for the industrial production in Italy. The first model uses static principal component analysis, while the other two apply dynamic principal component analysis in frequency domain and subspace algorithms for state-space representation, respectively. Once the factors are extracted from the business survey data, then they are included into a single equation to predict the industrial production index. The forecast results show that the three factor models have a better performance than that of a simple autoregressive benchmark model regardless of the specification and estimation methods. Furthermore, the state-space model yields superior forecasts amongst the factor models.

Factor Based Composite Indicators for the Italian Economy

A factor based approach is often used to build Composite Indicators (CI) from qualitative data stemming from Business and Consumers Survey (BCS). Bruno and Malgarini (2002) and Gayer and Genet (2006) have used factor analysis to synthesize the information contained in the balances of the various surveys Harmonized by the EC (industry, consumers, retail, building and services). However, Marcellino (2006) pointed out that the use of aggregate balance series could imply missing relevant information contained in the surveys. For this reason, in this paper we consider additional information stemming from the percentage of equal answers; moreover, we also use more disaggregate data at the branch level (considering socio-economics characteristics of the respondents for the consumers survey). More specifically, we consider Main Industrial Groupings for the industry survey; small and large multiple shops for the retail survey; building and civil engineering for the construction survey; households and business services for the service survey. Variables to be included in the analysis are pre selected prior to factor extraction on the basis of their contemporaneous or leading/lagging correlation with sector-specific target series. Three methods are then used to extract Composite Indicators, namely Static Principal Component Analysis and Static and Dynamic Factor Analysis (Forni, Hallin, Lippi, Reichlin, 2000, 2001). The various Composite Indicators obtained from the factor based approach are then investigated against the traditional Confidence Indicators in terms of performance with respect to the reference series. As alternative evaluation criteria we use: a) the cross-correlation between the CI and the reference series; b) the directional coherence of movement with the targets; c) turning points analysis (determined applying the Bry-Boschan method). Finally, from the whole set of data stemming from ISAE business and consumers survey we extract aggregate Composite Indicators for the whole Italian economy using the same methods and evaluation criteria outlined above. Indicators calculated with Static Factor Analysis on aggregate balances show the best performance in tracking the reference cycle, i.e. the rate of growth of Italian GDP.

Monitoring bioreactors using principal component analysis: production of penicillin G acylase as a case study

The complexity of biological processes often makes impractical the development of detailed, structured phenomenological models of the cultivation of microorganisms in bioreactors. In this context, data pre-treatment techniques are useful for bioprocess control and fault detection. Among them, principal component analysis (PCA) plays an important role. This work presents a case study of the application of this technique during real experiments, where the enzyme penicillin G acylase (PGA) was produced by Bacillus megaterium ATCC 14945. PGA hydrolyzes penicillin G to yield 6-aminopenicilanic acid (6-APA) and phenyl acetic acid. 6-APA is used to produce semi-synthetic beta-lactam antibiotics. A static PCA algorithm was implemented for on-line detection of deviations from the desired process behavior. The experiments were carried out in a 2-L bioreactor. Hotteling's T(2) was the discrimination criterion employed in this multivariable problem and the method showed a high sensibility for fault detection in all real cases that were studied.

Statistical monitoring of fed-batch process using dynamic multiway neighborhood preserving embedding

A multivariate statistical process control (MSPC) method using dynamic multiway neighborhood preserving embedding (DMNPE) is proposed for fed-batch process monitoring. Different from principal component analysis (PCA) which aims at preserving the global Euclidean structure of the data set, neighborhood preserving embedding aims to preserve the local neighborhood structure of the data set. The neighborhood preserving property enables NPE to find more meaningful intrinsic information hidden in the high-dimensional observations compared with PCA. Moreover, the robustness of NPE is better than that of PCA. On the other hand, a dynamic monitoring approach based on moving window technique is employed to deal with the time-variant property of the dynamic processes. An industrial cephalosporin fed-batch fermentation process is used to demonstrate the performance of the DMNPE. The results show the advantages of DMNPE over those methods such as dynamic multiway PCA (DMPCA), static multiway NPE (SMNPE) and static multiway PCA (SMPCA) in fed-batch process monitoring. Finally, the robustness of the DMNPE monitoring is tested by adding noises to the original data sets.

INTRODUCTION OF DYNAMICS TO AN APPROACH FOR BATCH PROCESS MONITORING USING INDEPENDENT COMPONENT ANALYSIS

Use of independent component analysis (ICA) in developing statistical monitoring charts for batch processes has been reported previously. This article extends the previous work by introducing time lag shifts to include process dynamics in the ICA model. Comparison of the dynamic ICA-based method with other batch process monitoring approaches based on static ICA, static principal component analysis (PCA), and dynamic PCA is made for an industrial batch polymerization reactor and a simulated fed-batch penicillin fermentation process. For both case studies, it was found that the dynamic ICA approach detected faults earlier than other approaches, with less ambiguity, and was the only approach that detected all the faults.

Fault reconstruction in linear dynamic systems using multivariate statistics

Treasure et al. (2004) recently proposed a new subspace-monitoring technique, based on the N4SID algorithm, within the multivariate statistical process control framework. This dynamic-monitoring method requires considerably fewer variables to be analysed when compared with dynamic principal component analysis (PCA). The contribution charts and variable reconstruction, traditionally employed for static PCA, are analysed in a dynamic context. The contribution charts and variable reconstruction may be affected by the ratio of the number of retained components to the total number of analysed variables. Particular problems arise if this ratio is large and a new reconstruction chart is introduced to overcome these. The utility of such a dynamic contribution chart and variable reconstruction is shown in a simulation and by application to industrial data from a distillation unit.

Fault Diagnosis in the Feedback-Invariant Subspace of Closed-Loop Systems

In this paper we demonstrate that feedback control will obscure the apparent source of a fault and make fault isolation difficult. A new method is proposed whereby fault detection and diagnosis is performed in the feedback-invariant subspace of the closed-loop process output. The feedback-invariant signals are achieved by a simple orthogonal projection without the knowledge of the process or controller models. The only requirement is that the process or controller must have at least one time delay, which is generally satisfied. The feedback-invariant signals remove autocorrelation due to closed-loop dynamics, making static principal component analysis applicable for fault detection and diagnosis. A simulated example is provided to illustrate the procedure.

ToF‐SIMS for the characterization of hyperbranched aliphatic polyesters: probing their molecular weight on surfaces based on principal component analysis (PCA)

AbstractA series of 2,2‐bis(hydroxymethyl)propionic acid (Bis‐MPA) hyperbranched aliphatic polyesters with different molecular weights (generations) is analysed for the first time by time‐of‐flight secondary ion mass spectrometry (ToF‐SIMS). The main negative and positive low‐mass fragments are identified in the fingerprint part of the spectra (m/z < 400) and are principally assigned to fragmentation of the Bis‐MPA repeating units. In addition, it is shown that the fragmentation pattern is highly affected by the functional end‐groups. This is illustrated for a phthalic acid end‐capped hyperbranched polymer and for an acetonide‐terminated dendrimer analog. Also, typical fragments assigned to the ethoxylated pentaerythritol core molecule are detected. These ions show decreasing intensities with increasing molecular weight. This intensity dependency on the generation is used to calibrate the molecular weight of hyperbranched polyesters on the surface. To obtain quantitative information, a principal component analysis (PCA) multivariate statistical method is applied to the ToF‐SIMS data. The influence of different normalization procedures prior to PCA calculation is tested, e.g. normalization to the total intensity, to the intensities of ions assigned to the Bis‐MPA repeating unit or to intensities of fragments due to the core molecule. It is shown that only one principal component (PC1) is needed to describe most of the variance between the samples. In addition, PC1 takes into account the generation effect. However, different relationships between the PC1 scores and the hyperbranched mass average molecular weights are observed depending on the normalization procedure used. Normalization of data set ion intensities by ion intensities from the core molecule allows linearization of the SIMS intensities versus the molecular weight and allows the hyperbranched polymers to be discriminated up to the highest generations. In addition, PCA applied to ToF‐SIMS data provides an extended interpretation of the spectra leading to further identification of the correlated mass peaks, such as those of the Bis‐MPA repeating unit (terminal, dendritic and linear) and those of the core molecule. Finally, the work presented demonstrates the extreme potential of the static ToF‐SIMS and PCA techniques in the analysis of dendritic molecules on solid surfaces. Copyright © 2003 John Wiley & Sons, Ltd.

Decentralized PCA Charts for Performance Assessment of Plant-Wide Control Structures

Static principal component analysis (PCA) statistical charts are used to assess and compare the performance of two plant-wide control strategies of the Tennessee Eastman process. Both centralized and decentralized charts are used as well as charts relating to the energy and material balance of the process. In most cases these charts reveal the origin of the disturbance and the effectiveness of the control structure to compensate for the disturbance. Differences and similarities between the two control structures are also examined.

Disturbance detection and isolation by dynamic principal component analysis

In this paper we extend previous work by ourselves and other researchers in the use of principal component analysis (PCA) for statistical process control in chemical processes. PCA has been used by several authors to develop techniques to monitor chemical processes and detect the presence of disturbances [1–5]. In past work, we have developed methods which not only detect disturbances, but isolate the sources of the disturbances [4]. The approach was based on static PCA models, T 2 and Q charts [6], and a model bank of possible disturbances. In this paper we use a well-known ‘time lag shift’ method to include dynamic behavior in the PCA model. The proposed dynamic PCA model development procedure is desirable due to its simplicity of construction, and is not meant to replace the many well-known and more elegant procedures used in model identification. While dynamic linear model identification, and time lag shift are well known methods in model building, this is the first application we are aware of in the area of statistical process monitoring. Extensive testing on the Tennessee Eastman process simulation [7] demonstrates the effectiveness of the proposed methodology.