Traditional Principal Component Analysis Research Articles

Joint Lp-Norm and L2,1-Norm Constrained Graph Laplacian PCA for Robust Tumor Sample Clustering and Gene Network Module Discovery.

The dimensionality reduction method accompanied by different norm constraints plays an important role in mining useful information from large-scale gene expression data. In this article, a novel method named Lp-norm and L2,1-norm constrained graph Laplacian principal component analysis (PL21GPCA) based on traditional principal component analysis (PCA) is proposed for robust tumor sample clustering and gene network module discovery. Three aspects are highlighted in the PL21GPCA method. First, to degrade the high sensitivity to outliers and noise, the non-convex proximal Lp-norm (0 < p < 1)constraint is applied on the loss function. Second, to enhance the sparsity of gene expression in cancer samples, the L2,1-norm constraint is used on one of the regularization terms. Third, to retain the geometric structure of the data, we introduce the graph Laplacian regularization item to the PL21GPCA optimization model. Extensive experiments on five gene expression datasets, including one benchmark dataset, two single-cancer datasets from The Cancer Genome Atlas (TCGA), and two integrated datasets of multiple cancers from TCGA, are performed to validate the effectiveness of our method. The experimental results demonstrate that the PL21GPCA method performs better than many other methods in terms of tumor sample clustering. Additionally, this method is used to discover the gene network modules for the purpose of finding key genes that may be associated with some cancers.

Soil fertility quality assessment based on geographically weighted principal component analysis (GWPCA) in large-scale areas

Principal component analysis (PCA) has been widely used in the integrated soil fertility quality index (SFQI) (SFQIPCA) assessment. However, the traditional PCA, only established in variable space, does not consider the spatially varying relationships among soil fertility indicators in large-scale areas, and thus cannot appropriately determine the spatially varying relative importance (i.e., weight calculated based on communality) of soil indicators in the SFQI assessment. Moreover, uncertainty inevitably exists in the spatial distribution pattern of SFQI due to the limited sample points, which is critical to the precision management of soil fertility. To address these limitations, this study first proposed a novel assessment method for SFQI based on geographically weighted principal component analysis (GWPCA) (SFQIGWPCA). Secondly, SFQIGWPCA was assessed in Shayang County, China, and then compared with the traditional SFQIPCA. Finally, the spatial uncertainty of SFQIGWPCA was assessed based on the 1000 realizations generated by sequential Gaussian simulation (SGS). Results showed that (i) spatially varying relationships among soil indicators were revealed by Monte Carlo test and GWPCA outputs (i.e., the winning variable and the local percentage of total variation), while traditionally-used PCA was conducted in the assumption of spatially stationary relationships among soil indicators; (ii) in SFQI assessment, the spatially varying indicator weights were determined by GWPCA, but could not be determined by PCA; (iii) the areas with higher threshold-exceeding probability (≥ 0.95) mainly located in the northwest of this county, and the areas with lower threshold-exceeding probability (≤ 0.05) mainly located in the east of this county. It is concluded that GWPCA adequately considers the spatially varying relationships among soil indicators, and SFQIGWPCA is an effective tool in the SFQI assessment in large-scale areas.

Dependently Coupled Principal Component Analysis for Bivariate Inversion Problems.

Principal Component Analysis (PCA) is a widely used technique for dimensionality reduction in various problem domains, including data compression, image processing, visualization, exploratory data analysis, pattern recognition, time-series prediction, and machine learning. Often, data is presented in a correlated paired manner such that there exist observable and correlated unobservable measurements. Unfortunately, traditional PCA techniques generally fail to optimally capture the leverageable correlations between such paired data as it does not yield a maximally correlated basis between the observable and unobservable counterparts. This instead is the objective of Canonical Correlation Analysis (and the more general Partial Least Squares methods); however, such techniques are still symmetric in maximizing correlation (covariance for PLSR) over all choices of the basis for both datasets without differentiating between observable and unobservable variables (except for the regression phase of PLSR). Further, these methods deviate from PCA's formulation objective to minimize approximation error, seeking instead to maximize correlation or covariance. While these are sensible optimization objectives, they are not equivalent to error minimization. We therefore introduce a new method of leveraging PCA between paired datasets in a dependently coupled manner, which is optimal with respect to approximation error during training. We generate a dependently coupled paired basis for which we relax orthogonality constraints in decomposing unreliable unobservable measurements. In doing so, this allows us to optimally capture the variations of the observable data while conditionally minimizing the expected prediction error for the unobservable component. We show preliminary results that demonstrate improved learning of our proposed method compared to that of traditional techniques.

Fault Monitoring of Chemical Process Based on Sliding Window Wavelet DenoisingGLPP

In industrial process fault monitoring, it is very important to collect accurate data, but in the actual process, there are often various noises that are difficult to eliminate in the collected data due to sensor accuracy, measurement errors, or human factors. Existing statistical process monitoring methods often ignore the problem of data noise. To solve this problem, a sliding window wavelet denoising-global local preserving projections (SWWD-GLPP) process monitoring method is proposed. In the offline stage, the wavelet denoising method is used to denoise the offline data, and then, the GLPP method is used for offline modeling, and then, the control limit is obtained by the kernel density estimation method. In the online phase, the sliding window wavelet denoising method is used to denoise the online data in real time. Then, use the model of the GLPP method to find the statistics, compare them with the control limit, judge the fault situation, and finally, use the contribution graph method to determine the variable that caused the fault, so as to diagnose the fault. This article uses a numerical case to illustrate the effectiveness of the algorithm, using the Tennessee Eastman (TE) process to compare the traditional principal component analysis (PCA) and GLPP methods to further prove the effectiveness and superiority of the method.

Fault Detection of Wind Turbines by Subspace Reconstruction-Based Robust Kernel Principal Component Analysis

Data collected from the supervisory control and data acquisition (SCADA) system are widely used in wind farms to detect wind turbines’ (WTs) faults. However, it is challenging to extract wind turbines’ performance trends and provide accurate alarms based on SCADA data with large operating conditions fluctuations. This article presents a fault detection frame of subspace reconstruction-based robust kernel principal component analysis (SR-RKPCA) model for wind turbines SCADA data to extract nonlinear features under discontinuous interference. First, to improve the stability of the fault detection model of wind turbines, an RKPCA method is developed to decompose the original signal, decomposed into principal component subspace and residual subspace. Second, the permutation entropy is adopted to measure the residual matrix components’ disturbance level and remove trendless noise components. Then, a combined index is developed for fault detection with the aid of the residual space matrix with trend information. Ultimately, the proposed SR-RKPCA method for wind turbine detection is verified by investigating wind turbine faults cases. The proposed method has outstanding robustness and a more vital ability to extract nonlinear features for fault detection than traditional principal component analysis (PCA)- or KPCA-based methods.

Measuring sustainability of Russian largest companies

Prolonged and uncertain sustainable development routes provoke an urgent need for companies, investors and counterparties for assessment of efficiency of pursued business strategies, opportunity costs and long run benefits. However a unified algorithm is lacking and it makes compatible and impartial assessment of business performance rather complicated. The study develops and applies an automated adaptive methodology for calculating the integral indicator of company sustainability based on the generalized modified principal component analysis. The study is based on data from 23 Russian companies with maximum disclosure of sustainability indicators. The companies’ integral indicators generate sustainable development rating. It can be updated automatically with a new wave of the companies’ financial and non-financial reports. The proposed algorithm exceeds alternative methods of comparison of business models based on principles of sustainable development because it does not assume any expert assessments of companies’ activities and provides distinct ranking of companies. In contrast to the traditional principal component analysis the method developed in the article avoids any loss of variation of the initial data. Decomposition of the integral indicator identifies financial, socio-economic and environmental determinants for companies’ sustainability. The study examines the relationship between the ownership and management structure of the company and its achievements on sustainable development path.

Enhanced-PCA based Dimensionality Reduction and Feature Selection for Real-Time Network Threat Detection

With the rise of the data amount being collected and exchanged over networks, the threat of cyber-attacks has also increased significantly. Timely and accurate detection of any intrusion activity in networks has become a crucial task in order to safeguard data and other valuable assets. While manual moderation and programmed logic have been used for this purpose, the use of machine learning algorithms for superior pattern mapping is desired. The system logs in a network tend to include many parameters, and not all of them provide indications of an impending network threat. The selection of the right features is thus important for achieving better results. There is a need for accurate mapping of high dimension features to low dimension intermediate representations while retaining crucial information. In this paper, an approach for feature reduction and selection when working on the task of network threat detection is proposed. This approach modifies the traditional Principal Component Analysis (PCA) algorithm by working on its shortcomings and by minimizing the false detection rates. Specifically, work has been done upon the calculation of symmetric uncertainty and subsequent sorting of features. The performance of the proposed approach is evaluated on four standard-sized datasets that are collected using the Microsoft SYSMON real-time log collection tool. The proposed method is found to be better than the standard PCA and FAST methods for data reduction. The proposed approach makes a strong case as a dimensionality reduction and feature selection technique for minimizing false detection rates when operating on real-time data.

The instrument fault detection and identification based on kernel principal component analysis and coupling analysis in process industry

This paper investigates the fault detection problem of instruments in process industry. Considering the difficulty of fault identification and the problems of multivariable and large computation complexity based on traditional kernel principal component analysis (KPCA), this paper presents a new method for fault detection and identification, which combines the coupling analysis with kernel principal component for multivariable fault detection and employed the local outlier factor (LOF) for multivariable fault identification. The new method consists of three parts. Firstly, according to nonlinear correlation of multivariable, coupling analysis and module division of variables based on detrended cross-correlation analysis (DCCA) are considered to reduce false alarm rate (FAR) and missed detection rate (MDR) in fault detection and identification. Secondly, KPCA is employed to detect fault in each sub-module of variables. Finally, for the sub-module which has the fault detected in second step, the LOF is adopted to calculate abnormal contribution of each variable in sub-modules to realize fault identification. To prove that the new method has the better capability of processing multivariable fault detection and the more accuracy rate on fault detection and identification than the conventional methods of KPCA, a case study on Tennessee process is carried out at the end.

Modal Parameters Identification Based on Equalization Principal Component Analysis

Aiming at the magnitude influence of different discrete points in modal parameters identifications with traditional principal component analysis (PCA), a method of modal parameters identifications by equalization PCA is proposed in this paper. The basic idea is using the mean of time domain response data before identifying modal parameters with PCA. On the premise of not losing the main information of the original data, the influence of magnitude on the data is weakened. Taking a doubly-clamped beam as an example, modal parameters of the beam are identified by traditional PCA and equalization PCA with the same constraints of principal component cumulative contribution rate. The results show that compared with the traditional PCA, equalization PCA can effectively identify more orders of modal parameters. The errors of vibration modal shapes and natural frequencies identified by equalization PCA are allowable.

Fault Diagnosis of Multi-level Principal Component Space Based on High-Dimensional Feature Representation

Due to the influence of space equivalent representation and pattern composition, traditional principal component analysis (PCA) is faced with challenges in identifying small faults. This paper proposes a fault diagnosis method is proposed based on multi-levels principal component space based on high-dimensional feature representation. Firstly, based on the original PCA, the dimensions are expanded first and then reduced, so that the information that cannot be expressed in the original space is fully expressed. This method further divides the principal element space based on the eigenvalues, and projects the occurrence of the event and the size of the identification abnormality according to the projection to different sub-spaces. Compared with the usual fault detection methods, simulations verify that this algorithm can effectively measure the level of faults and improve the correct detection rate of minor faults.

On the use of PCA for Diagnostics via Novelty Detection: interpretation, practical application notes and recommendation for use

The Principal Component Analysis (PCA) is the simplest eigenvector-based multivariate data analysis tool and dates back to 1901 when Karl Pearson proposed it as a way for finding the best fitting d-1 hyperplane of a system of points in a d-dimensional (Euclidean) space.&#x0D; Over the time, the PCA evolved in different fields with several different names and with different scopes, but, in its essence, it is always an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.&#x0D; Generalizing Pearson’s purpose, the knowledge derived by such an analysis is mostly used to find a subspace which effectively and efficiently summarizes the original system of points by losing a minimum amount of information.&#x0D; In the field of Diagnostics, the fundamental task of detecting damage is basically a binary classification problem which is in many cases tackled via Novelty Detection: an observation is classified as novel if it differs significantly from other observations. Novelty can, in principle, be assessed directly in the original space, but the effectiveness of the estimated novelty can be improved by taking advantage of the PCA.&#x0D; In this work, the traditional PCA will be compared to a robust modification that is commonly used in the field of diagnostics to face the issue of confounding influences which could affect the novelty-damage correspondence. Comparisons will be made to shed light on the main misleading aspects of PCA, and finally, define a unique, theoretically justified procedure for Diagnostics via Novelty Detection.

Improved PCA model for multiple fault detection, isolation and reconstruction of sensors in nuclear power plant

Safety is the most important indicator of a nuclear power plant. The extensive use of sensors could help operators gain more information about nuclear power plants. However, it also increases the risk of sensor failures. Therefore, it is necessary to study the multiple fault detection, isolation and reconstruction (FDIR) of sensors. For the traditional principal component analysis (PCA) model, this paper proposes two improvements. The first improvement is to propose a Corrected Reconstruction Algorithm (CRA) to improve the accuracy of the reconstruction. The traditional PCA reconstruction has lower accuracy when reconstructing multi-sensor faults. The second improvement is a cyclic PCA (CPCA) monitoring model to detect multi-sensor failures. The purpose of this is to improve the PCA model's ability to detect multiple sensor faults. Finally, the data from an actual nuclear power plant is used for modeling and verification. Simulation tests show that the CPCA model could accurately detect the different kinds of sensor faults and successfully reconstruct fault data.

Unsupervised classification of the intrinsic and morphological properties of quasars through self‐organizing maps

AbstractThe aim of this paper is to unveil, by means of self‐organizing maps (SOMs), the connections between intrinsic properties of radio quasars, namely redshift (z), luminosity (P), continuum slope (α), and parameters characterizing radio morphology of the host galaxy: observed linear size and bending. The strong luminosity–redshift correlation established by a prior traditional principal component analysis (PCA) is confirmed for the same dataset that contains 36 nearby (z &lt; 1.5) and 44 distant (z &gt; 1.5) quasars with five parameters. The SOM visualizations disentangled the luminosity–linear size anti‐correlation, supporting a dimensionality of p = 2 for the quasar parameter space. The SOM also clustered radio quasars into subgroups with common physical properties. We expect this work to provide a new probe, leading to further clustering of nonlabeled data with respect to the evolution of morphological properties.

Comparison of incremental linear dimension reduction methods for streaming data

Traditional linear dimension reduction methods such as Principal component analysis (PCA) and linear discriminant analysis (LDA) have been used in many application areas due to simplicity and high performance. However, in data streams where data instances are generated continuously over time, it is difficult to apply traditional PCA or LDA. Moreover, data streams can have drifting concepts over time. In this paper, we compared several incremental linear dimension reduction algorithms which can be applied for classification in streaming data. Also, the performance comparison for prediction accuracy and time complexity was conducted in various streaming environments such as low dimensional data streams, high dimensional data streams, and data streams with concept drifts. Experimental results showed that incremental least squares formulation (ILS) combined with incremental PCA can be used effectively for classification in streaming data.

Hyperspectral image classification using fuzzy-embedded hyperbolic sigmoid nonlinear principal component and weighted least squares approach

Hyperspectral image (HSI) classification is one of the significant research topics in the remote sensing community. The high dimensionality of the hyperspectral data, the high correlation among pixels, and the availability of fewer numbers of training samples affect the HSI classification accuracy. We propose an approach to extract the best representative bands from the high-dimensional imagery for better classification. Initially, the spectral bands are extracted by re-representing the traditional principal component analysis in terms of Hebbian learning, formulated and solved as a fuzzy optimization problem. Next, a spatial filter is applied to these spectral bands to obtain the smoothed image that preserves the spatial details. Finally, the spectral and spatial features are trained with the nonlinear support vector machine with the radial basis function kernel to obtain the classification map. Performance of the proposed approach is tested by varying different values of the parameters used in our model. The classification accuracy of the proposed approach is compared with the state-of-the-art techniques, which proves the effectiveness of the proposed methodology. The proposed approach can be applied in real-world applications, such as food quality, environment change detection, mineralogy, and pharmaceutical drug design.

Fault detection for centrifugal chillers using a Kernel Entropy Component Analysis (KECA) method

Fault detection is beneficial for chiller routine operation management in building automation systems. Considering the limitations of traditional principal component analysis (PCA) algorithm for chiller fault detection, a so-called kernel entropy component analysis (KECA) method has been developed and the development results are reported in this paper. Unlike traditional PCA, in KECA, the feature extraction or dimensionality reduction is implemented in a new space, called kernel feature space. The new space is nonlinearly related to the input space. The data set in the kernel feature space is projected onto a principal component subspace constructed by the feature space principal axes determined by the maximum Renyi entropy rather than the top eigenvalues. The proposed KECA is more suitable to deal with nonlinear process without Gaussian assumption. Using the available experimental data from ASHRAE RP-1043, seven typical chiller faults were tested by the proposed KECA method, and the results were compared to that of PCA. Two statistics, i.e. T2 and squared prediction error (SPE), were employed for fault detection monitoring. The fault detection results showed that the proposed KECA method had a better performance in terms of a higher detection accuracy in comparison to the traditional PCA. For the seven typical faults, the fault detection ratios were over 55%, even at their corresponding least severity level when using the proposed KECA based chiller fault detection method.

Robust deflated principal component analysis via multiple instance factorings for dimension reduction in remote sensing images

Remote sensing data are often adversely affected by noises due to atmospheric water absorption, transmission errors, sensor sensitivity, and saturation. Therefore, the performance of the classical principal component analysis (PCA) is negatively affected when dealing with such noisy data. To overcome this problem, a robust deflated PCA via multiple instance factorings (RDPCA-MIF) for dimension reduction in remote sensing images is proposed. In contrast to the traditional PCA, a penalty is imposed on the instance space through instance factoring technique to suppress the effect of noise in pursuing projections. To achieve this, we use two strategies, total distance, and cosine similarity metrics. Both strategies discriminate between noise and authentic data points by iteratively learning the relationship between instances and projections. Moreover, a deflation strategy that enables multiple relationships learning between instances and projections is applied to thoroughly evaluate the importance of instances in relation to all projections. Extensive experimental evaluations of the proposed RDPCA-MIF on Salinas-A dataset and shallow and deep features from the NWPU-RESISC45 remote sensing dataset prove its superiority over six comparative methods in dimension reduction and classification tasks.

A subspace type incremental two-dimensional principal component analysis algorithm

Principal component analysis (PCA) has been a powerful tool for high-dimensional data analysis. It is usually redesigned to the incremental PCA algorithm for processing streaming data. In this paper, we propose a subspace type incremental two-dimensional PCA algorithm (SI2DPCA) derived from an incremental updating of the eigenspace to compute several principal eigenvectors at the same time for the online feature extraction. The algorithm overcomes the problem that the approximate eigenvectors extracted from the traditional incremental two-dimensional PCA algorithm (I2DPCA) are not mutually orthogonal, and it presents more efficiently. In numerical experiments, we compare the proposed SI2DPCA with the traditional I2DPCA in terms of the accuracy of computed approximations, orthogonality errors, and execution time based on widely used datasets, such as FERET, Yale, ORL, and so on, to confirm the superiority of SI2DPCA.

Probabilistic predictive principal component analysis for spatially misaligned and high-dimensional air pollution data with missing observations.

Accurate predictions of pollutant concentrations at new locations are often of interest in air pollution studies on fine particulate matters (PM2.5), in which data is usually not measured at all study locations. PM2.5 is also a mixture of many different chemical components. Principal component analysis (PCA) can be incorporated to obtain lower-dimensional representative scores of such multi-pollutant data. Spatial prediction can then be used to estimate these scores at new locations. Recently developed predictive PCA modifies the traditional PCA algorithm to obtain scores with spatial structures that can be well predicted at unmeasured locations. However, these approaches require complete data, whereas multi-pollutant data tends to have complex missing patterns in practice. We propose probabilistic versions of predictive PCA which allow for flexible model-based imputation that can account for spatial information and subsequently improve the overall predictive performance.

Online monitoring scheme using principal component analysis through Kullback-Leibler divergence analysis technique for fault detection

Principal component analysis (PCA) is a common tool in the literature and widely used for process monitoring and fault detection. Traditional PCA is associated with the two well-known control charts, the Hotelling’s T2 and the squared prediction error (SPE), as monitoring statistics. This paper develops the use of new measures based on a distribution dissimilarity technique named Kullback-Leibler divergence (KLD) through PCA by measuring the difference between online estimated and offline reference density functions. For processes with PCA scores following a multivariate Gaussian distribution, KLD is computed on both principal and residual subspaces defined by PCA in a moving window to extract the local disparity information. The potentials of the proposed algorithm are afterwards demonstrated through an application on two well-known processes in chemical industries; the Tennessee Eastman process as a reference benchmark and three tank system as an experimental validation. The monitoring performance was compared to recent results from other multivariate statistical process monitoring (MSPM) techniques. The proposed method showed superior robustness and effectiveness recording the lowest average missed detection rate and false alarm rates in process fault detection.