Principal Component Analysis Scheme Research Articles

Improved privacy-preserving PCA using optimized homomorphic matrix multiplication

Principal Component Analysis (PCA) is a pivotal technique widely utilized in the realms of machine learning and data analysis. It aims to reduce the dimensionality of a dataset while minimizing the loss of information. In recent years, there have been endeavors to utilize homomorphic encryption in privacy-preserving PCA algorithms for secure cloud computing. These approaches commonly employ a PCA routine known as PowerMethod, which takes the covariance matrix as input and generates an approximate eigenvector corresponding to the primary component of the dataset. However, their performance is constrained by the absence of an efficient homomorphic covariance matrix computation circuit and an accurate homomorphic vector normalization strategy in the PowerMethod algorithm. In this study, we propose a novel approach to privacy-preserving PCA that addresses these limitations, resulting in superior efficiency, accuracy, and scalability compared to previous approaches.We attain such efficiency and precision through the following contributions: (i) We implement space and speed optimization techniques for a homomorphic matrix multiplication method, specifically tailored for parallel computing scenarios. (ii) Leveraging the benefits of this optimized matrix multiplication, we devise an efficient homomorphic circuit for computing the covariance matrix homomorphically. (iii) Utilizing the covariance matrix, we develop a novel and efficient homomorphic circuit for the PowerMethod that incorporates a universal homomorphic vector normalization strategy to enhance both its accuracy and practicality.Our privacy-preserving PCA scheme, implemented using our innovative homomorphic PowerMethod circuit, surpasses the state-of-the-art approach with an average speedup of 1.9 times on datasets with size 200×256. Notably, our scheme demonstrates an even more remarkable estimated speedup of 25 times when applied to larger datasets of size 60000×256, along with an R2 accuracy improvement of up to 0.285, showcasing efficiency and accuracy that has not been reported by previous approaches.

TrustGWAS: A full-process workflow for encrypted GWAS using multi-key homomorphic encryption and pseudorandom number perturbation.

The statistical power of genome-wide association studies (GWASs) is affected by the effective sample size. However, the privacy and security concerns associated with individual-level genotype data pose great challenges for cross-institutional cooperation. The full-process cryptographic solutions are in demand but havenot been covered, especially the essential principal-component analysis (PCA). Here, we present TrustGWAS, a complete solution for secure, large-scale GWAS, recapitulating gold standard results against PLINK without compromising privacy and supporting basic PLINK steps including quality control, linkage disequilibrium pruning, PCA, chi-square test, Cochran-Armitage trend test, covariate-supported logistic regression and linear regression, and their sequential combinations. TrustGWAS leverages pseudorandom number perturbations for PCA and multiparty scheme of multi-key homomorphic encryption for all other modules. TrustGWAS can evaluate 100,000 individuals with 1 million variants and complete QC-LD-PCA-regression workflow within 50 h. We further successfully discover gene loci associated with fasting blood glucose, consistent with the findings of the ChinaMAP project.

Anomaly Detection in Wind Tunnel Experiments by Principal Component Analysis

Principal component analysis (PCA) was used to detect anomalies in wind tunnel measurements. Data were compiled from previous measurements for three two-dimensional airfoils in the Virginia Tech Stability Wind Tunnel with similar experimental arrangements. Measurements included the surface pressure distribution about the airfoil, total pressure distribution in the airfoil wake, static pressure in the wake, angle of attack, freestream velocity, flow temperature, and ambient pressure. These data were used to train the PCA scheme through an eigendecomposition of the measurement covariance matrix. A low-order reconstruction of this covariance matrix was then used to assess whether new measurements not included in the training set were anomalous by comparing predictions of the expected result with measured values. Results show that the method is very good at detecting anomalies in data from airfoils included in the training set as well as an additional airfoil outside of this set. Predictions of a particular measurement can be further improved by biasing the data used to construct the covariance matrix with observations that are more similar to the measurement of interest. However, this biasing reduces the ability of the method to predict results and therefore detect anomalies for experimental conditions outside of the range of the input data.

Landslide Hazard Analysis Using a Multilayered Approach Based on Various Input Data Configurations

Landslide is a natural disaster that occurs mostly in hill areas. Landslide hazard mapping is used to classify the prone areas to mitigate the risk of landslide hazards. This paper aims to compare spatial landslide prediction performance using an artificial neural network (ANN) model based on different data input configurations, different numbers of hidden neurons, and two types of normalization techniques on the data set of Penang Island, Malaysia. The data set involves twelve landslide influencing factors in which five factors are in continuous values, while the remaining seven are in categorical/discrete values. These factors are considered in three different configurations, i.e., original (OR), frequency ratio (FR), and mixed-type (MT) data, which act as an input to train the ANN model separately. A significant effect on the final output is the number of hidden neurons in the hidden layer. In addition, three data configurations are processed using two different normalization methods, i.e., mean-standard deviation (Mean-SD) and Min-Max. The landslide causative data often consist of correlated information caused by overlapping of input instances. Therefore, the principal component analysis (PCA) technique is used to eliminate the correlated information. The area under the receiver of characteristics (ROC) curve, i.e., AUC is also applied to verify the produced landslide hazard maps. The best result of AUC for both Mean-SD and Min-Max with PCA schemes are 96.72% and 96.38%, respectively. The results show that Mean-SD with PCA of MT data configuration yields the best validation accuracy, AUC, and lowest AIC at 100 number of hidden neurons. MT data configuration with the Mean-SD normalization and PCA scheme is more robust and stable in the MLP model's training for landslide prediction. 
 Keywords: Landslide; ANN; Hidden Neurons; Normalization; PCA; ROC; Hazard map
 
 Copyright (c) 2021 Geosfera Indonesia and Department of Geography Education, University of Jember
 This work is licensed under a Creative Commons Attribution-Share A like 4.0 International License

A principal component analysis-based feature dimensionality reduction scheme for content-based image retrieval system

In Content-Based Image Retrieval (CBIR) system, one approach of image representation is to employ combination of low-level visual features cascaded together into a flat vector. While this presents more descriptive information, it however poses serious challenges in terms of high dimensionality and high computational cost of feature extraction algorithms to deployment of CBIR on platforms (devices) with limited computational and storage resources. Hence, in this work a feature dimensionality reduction technique based on Principal Component Analysis (PCA) is implemented. Each image in a database is indexed using 174 dimensional feature vector comprising of 54-dimensional Colour Moments (CM54), 32-bin HSV-histogram (HIST32), 48-dimensional Gabor Wavelet (GW48) and 40-dimensional Wavelet Moments (MW40). The PCA scheme was incorporated into a CBIR system that utilized the entire feature vector space. The k-largest Eigenvalues that yielded a not more than 5% degradation in mean precision were retained for dimensionality reduction. Three image databases (DB10, DB20 and DB100) were used for testing. The result obtained showed that with 80% reduction in feature dimensions, tolerable loss of 3.45, 4.39 and 7.40% in mean precision value were achieved on DB10, DB20 and DB100.

Synchrophasor-Based Islanding Detection for Distributed Generation Systems Using Systematic Principal Component Analysis Approaches

Systematic principal component analysis (PCA) methods are presented in this paper for reliable islanding detection for power systems with significant penetration of distributed generations (DGs), where synchrophasors, recorded by phasor measurement units, are used for system monitoring. Existing islanding detection methods, such as rate-of-change-of-frequency and vector shift are fast for processing local information; however, with the growth in installed capacity of DGs, they suffer from several drawbacks. Incumbent genset islanding detection cannot distinguish a system-wide disturbance from an islanding event, leading to maloperation. The problem is even more significant when the grid does not have sufficient inertia to limit frequency divergences in the system fault/stress due to the high penetration of DGs. To tackle such problems, this paper introduces PCA methods for islanding detection. A simple control chart is established for intuitive visualization of the transients. A recursive PCA scheme is proposed as a reliable extension of the PCA method to reduce the false alarms for time-varying process. To further reduce the computational burden, the approximate linear dependence condition errors are calculated to update the associated PCA model. The proposed PCA and RPCA methods are verified by detecting abnormal transients occurring in the U.K. utility network.

Fault detection with Conditional Gaussian Network

The main interest of this paper is to illustrate a new representation of the Principal Component Analysis (PCA) for fault detection under a Conditional Gaussian Network (CGN), a special case of Bayesian networks. PCA and its associated quadratic statistics such as T2 and SPE are integrated under a sole CGN. The proposed framework projects a new observation into an orthogonal space and gives probabilities on the state of the system. It could do so even when some data in the sample test are missing. This paper also gives the probabilities thresholds to use in order to match quadratic statistics decisions. The proposed network is validated and compared to the standard PCA scheme for fault detection on the Tennessee Eastman Process and the Hot Forming Process.

A Fast and Efficient Clutter Rejection Algorithm for SAR Imagery

A clutter rejection scheme is proposed for synthetic aperture radar (SAR) imagery based on two-stage two-dimensional principal component analysis (two-stage 2DPCA) followed by a bipolar eigenspace separation transformation (BEST) and a multilayer perceptron (MLP). For this, we have examined and analyzed four different algorithms. They are based on principal component analysis (PCA) both in one dimension (conventional PCA) and in two dimension with three different forms (2DPCA, alternative 2DPCA and two-stage 2DPCA), followed by BEST and MLP in each case. Feature extraction in different cases is carried out using respective PCA scheme. Each algorithm uses the BEST to further reduce dimensionality and enhance the generalization capability of the classifier. Classification between target chips and clutter chips is finally made through an MLP classifier. Experimental results on MSTAR public release database of SAR imagery are presented. Comparison of all the 2DPCA algorithms with an existing technique shows improvement both in performance and time. Moreover, all the 2DPCA algorithms compute eigenvectors and eigenvalues of image covariance matrix accurately and very efficiently with respect to time, and further reduces the effect of noise better than that in the case of PCA. Comparison reveals the fact that two-stage 2DPCA-based algorithm is the best (both in performance and time) between all algorithms.

Conventional and spatial principal component analysis on near-field head-related transfer functions

Head-related transfer functions (HRTFs) vary with both frequency and source position. The near-field HRTFs with source distance less than 1.0 m are particularly complicated due to their distance dependence. Principal component analysis (PCA), which is conventionally carried out in the frequency or time domain, has been widely applied to reduce the dimensionality of far-field HRTF data with source distance greater than 1.0 m. The present work first extends the conventional PCA, and then proposes a spatial PCA method in the spatial domain rather than in the frequency or time domain to reduce the dimensionality of near-field HRTF data. An illustrative case indicates that near-field HRTF magnitudes at 9 distances with 493 directions for each distance can be approximately represented by the weighted sum of 15 spectral shape basis functions using the conventional PCA or the weighted sum of 15 spatial basis functions using the spatial PCA. Both representations account for more than 98% energy variation of the original data, and reduce the dimensionality of the original data to about a quarter. The proposed method is also applicable to the head-related impulse responses in the time domain. Furthermore, the spatial PCA scheme is potentially applicable to simplify near-field HRTF measurement.

A pure [formula omitted]-norm principal component analysis

The L1 norm has been applied in numerous variations of principal component analysis (PCA). An L1-norm PCA is an attractive alternative to traditional L2-based PCA because it can impart robustness in the presence of outliers and is indicated for models where standard Gaussian assumptions about the noise may not apply. Of all the previously-proposed PCA schemes that recast PCA as an optimization problem involving the L1 norm, none provide globally optimal solutions in polynomial time. This paper proposes an L1-norm PCA procedure based on the efficient calculation of the optimal solution of the L1-norm best-fit hyperplane problem. We present a procedure called L1-PCA∗ based on the application of this idea that fits data to subspaces of successively smaller dimension. The procedure is implemented and tested on a diverse problem suite. Our tests show that L1-PCA∗ is the indicated procedure in the presence of unbalanced outlier contamination.

Robust PCA as Bilinear Decomposition With Outlier-Sparsity Regularization

Principal component analysis (PCA) is widely used for dimensionality reduction, with well-documented merits in various applications involving high-dimensional data, including computer vision, preference measurement, and bioinformatics. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify PCA against outliers. A least-trimmed squares estimator of a low-rank bilinear factor analysis model is shown closely related to that obtained from an $\ell_0$-(pseudo)norm-regularized criterion encouraging sparsity in a matrix explicitly modeling the outliers. This connection suggests robust PCA schemes based on convex relaxation, which lead naturally to a family of robust estimators encompassing Huber's optimal M-class as a special case. Outliers are identified by tuning a regularization parameter, which amounts to controlling sparsity of the outlier matrix along the whole robustification path of (group) least-absolute shrinkage and selection operator (Lasso) solutions. Beyond its neat ties to robust statistics, the developed outlier-aware PCA framework is versatile to accommodate novel and scalable algorithms to: i) track the low-rank signal subspace robustly, as new data are acquired in real time; and ii) determine principal components robustly in (possibly) infinite-dimensional feature spaces. Synthetic and real data tests corroborate the effectiveness of the proposed robust PCA schemes, when used to identify aberrant responses in personality assessment surveys, as well as unveil communities in social networks, and intruders from video surveillance data.

Adaptive Kernel Principal Component Analysis (KPCA) for Monitoring Small Disturbances of Nonlinear Processes

The Tennessee Eastman (TE) process, created by Eastman Chemical Company, is a complex nonlinear process. Many previous studies focus on the detectability of monitoring a multivariate process by using TE process as an example. Principal component analysis (PCA) is a widely used dimension-reduction tool for monitoring multivariate linear process. Recently, the kernel principal component analysis (KPCA) has emerged as an effective method to tackling the problem of nonlinear data. Nevertheless, the conventional KPCA used the sum of squares of latest observations as the monitoring statistics and hence failed to detect small disturbance of the process. To enhance the detectability of the KPCA-based monitoring method, an adaptive KPCAbased monitoring statistic is proposed in this paper. The basic idea of the proposed method is first adopting the multivariate exponentially moving average to predict the process mean shifts and then combining the estimated mean shifts with the extracted components by KPCA to construct the adaptive monitoring statistic. The efficiency of the proposed monitoring scheme is implemented in a simulated nonlinear system and in the TE process. The experimental results indicate that the proposed method outperforms the traditional PCA and KPCA monitoring schemes.

An improved PCA scheme for sensor FDI: Application to an air quality monitoring network

In this paper a sensor fault detection and isolation procedure based on principal component analysis (PCA) is proposed to monitor an air quality monitoring network. The PCA model of the network is optimal with respect to a reconstruction error criterion. The sensor fault detection is carried out in various residual subspaces using a new detection index. For our application, this index improves the performance compared to classical detection index SPE. The reconstruction approach allows, on one hand, to isolate the faulty sensors and, on the other hand, to estimate the fault amplitudes.

Nonnegative color spectrum analysis filters from principal component analysis characteristic spectra.

Nonnegative color analysis filters are obtained by using an invertible linear transformation of characteristic spectra, which are orthogonal vectors from a principal component analysis (PCA) of a representative ensemble of color spectra. These filters maintain the optimal compression properties of the PCA scheme. Linearly constrained nonlinear programming is used to find a transformation that minimizes the noise sensitivity of the filter set. The method is illustrated by computing analysis and synthesis filters for an ensemble of measured Munsell color spectra.