Ordinary Principal Component Analysis Research Articles

Machine learning in viscoelastic fluids via energy-based kernel embedding

The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce energy-compatible families of kernel functions corresponding to a given viscoelastic stress model. Each kernel implicitly embeds flowfield snapshots into a Reproducing Kernel Hilbert Space (RKHS) in which distances and angles are computed and whose squared norm equals the total mechanical energy. Additionally, we present a solution to the preimage problem for these kernels, enabling accurate reconstruction of flowfields from their RKHS representations. Through numerical experiments on an unsteady viscoelastic lid-driven cavity flow, we demonstrate the utility of energy-compatible kernels for extracting energetically-dominant coherent structures in viscoelastic flows across a range of Reynolds and Weissenberg numbers. Specifically, the features extracted by Kernel Principal Component Analysis (KPCA) of flowfield snapshots using energy-compatible kernel functions yield reconstructions with superior accuracy in terms of mechanical energy compared to conventional methods such as ordinary Principal Component Analysis (PCA) with naïvely-defined state vectors or KPCA with ad-hoc choices of kernel functions. Our findings underscore the importance of principled choices of metrics in both scientific and machine learning investigations of complex fluid systems.

Extended Principal Component Analysis for Spatiotemporal Filtering of Incomplete Heterogeneous GNSS Position Time Series

When ordinary principal component analysis (PCA) is employed to analyze the position time series of a regional GNSS station network, the GNSS time series are assumed to be homogeneous, and the missing data in the time series must be restored beforehand. To directly process incomplete and heterogeneous GNSS position time series, we develop the extended PCA (EPCA) and weighted EPCA approaches to solving for the missing values based on the best low-rank approximation in the spatiotemporal domain. The proposed approaches are used to process the real GNSS position time series of 24 stations in North China spanning 2011 to 2019 and successfully extract the common mode errors (CMEs). The proposed approaches are compared with modified PCA (MPCA) and weighted MPCA, in which an additional optimization criterion needs to be introduced in the frequency domain. The results show that EPCA can extract more CMEs than MPCA for both the unweighted and weighted cases. Consequently, EPCA outperforms MPCA in reducing noise and improving the accuracy of site velocity estimates. Repeated simulation experiments show that the CMEs extracted by EPCA are closer to the simulated true values than those extracted by MPCA. When the formal errors of the time series are considered, both weighted EPCA and weighted MPCA outperform their unweighted counterparts, and the former outperforms the latter. In addition, EPCA is computationally more efficient than MPCA since fewer unknowns need to be estimated.

Perturbation theory for cross data matrix-based PCA

Principal component analysis (PCA) has long been a useful and important tool for dimension reduction. However, this method must be used with care under certain circumstances such as high dimension and small sample size. In general, low dimension with large sample size or large signal to noise ratio is vital to guarantee the consistency of the leading eigenvalues and eigenvectors obtained by PCA. Cross data matrix (CDM)-based PCA is another way to estimate PCA components, through splitting data into two subsets and calculating singular value decomposition for the cross product of the corresponding covariance matrices. It has been shown that CDM-based PCA has a broader region of consistency than ordinary PCA for leading eigenvalues and eigenvectors. Although the difference in regions of consistency is well studied, an interesting practical as well as theoretical question is how they differ in eigenvalues and eigenvectors estimation, especially for the case where both fall in a common region of consistency. In this article, we derive the finite sample approximation results as well as the asymptotic behavior for CDM-based PCA via matrix perturbation. Furthermore, we also derive a comparison measure for CDM-based PCA vs. ordinary PCA. This measure only depends on the data dimension, noise correlations and the noise-to-signal ratio (NSR). Using this measure, we develop an algorithm, which selects good partitions and integrates results from these good partitions to form a final estimate for CDM-based PCA. Numerical and real data examples are presented for illustration.

Application of self-organizing map (SOM) and K-means clustering algorithms for portraying geochemical anomaly patterns in Moalleman district, NE Iran

In this paper, in order to reveal the regional geochemical patterns of regularly sampled stream sediment data, we have employed the K-means and self-organizing map (SOM) as clustering methods in the Moalleman district, northeast Iran. Initially, a set of analyzed elements of geochemical data was subjected to isometric log-ratio (ilr) transformation to address the closure problem related to geochemical data, then, ordinary principal component analysis (PCA) was utilized for recognizing the internal relations between selected elements (As, Au, Cu, Pb, Sb and Zn). Subsequently, the K-means and SOM as unsupervised clustering methods were applied based on PC1 (Cu-Pb-Zn aggregation) and PC2 (Au-As-Sb aggregation) to distinguish different populations of multi-element geochemical indicators. In this regard, Silhouette Width (SW) was implemented for computing the optimal cluster number in K-means clustering method. In the next step, due to the presence of numerous copper mineral deposits/occurrences in the study area, we opted to implement the supervised SOM on ilr-transformed values of Cu-Pb-Zn elements for delineating high anomalous zones. For this purpose, a confusion matrix based on training and out-of-bag (OOB) data was developed for the supervised SOM model and the results indicated the accuracy of 96.27% and 94.26%, respectively. Moreover, success-rate curves were used for assessing the overall performance of K-means and SOM (unsupervised and supervised) models. Experimental outcomes represented the superiority of SOM models (especially the supervised SOM) over K-means in delineating the geochemical anomaly targets which can be used as an effective and powerful tool for discovering the complex patterns among variables in exploratory geochemical data.

Constrained principal component analysis with stochastically ordered scores for high-dimensional mass spectrometry data

In this paper, we consider a constrained principal component analysis (PCA) for the projection of high-dimensional samples from different groups to a lower-dimensional space for which the principal scores are stochastically ordered over the groups. We express the problem as the minimization of a constrained biconvex problem and develop an iterative algorithm to solve it. We numerically show that the solution to our constrained PCA problem approximately rotates the principal coordinates of the ordinary PCA to achieve ordered scores. Consequently, our approach significantly improves the scores and the corresponding loading matrix compared to the original PCA if their true values are ordered over groups. We finally apply our method to two data examples: (i) the direct infusion multiple reaction monitoring mass spectrometry (DI-MRM-MS) data of white rice to verify the authenticity of adulterated Japonica rice and (ii) high-dimensional lipidomics data from ultra-performance liquid chromatography-mass spectrometry/mass spectrometry (UPLC-MS/MS) analysis of patients with liver diseases.

Field Theoretical Approach for Signal Detection in Nearly Continuous Positive Spectra II: Tensorial Data.

The tensorial principal component analysis is a generalization of ordinary principal component analysis focusing on data which are suitably described by tensors rather than matrices. This paper aims at giving the nonperturbative renormalization group formalism, based on a slight generalization of the covariance matrix, to investigate signal detection for the difficult issue of nearly continuous spectra. Renormalization group allows constructing an effective description keeping only relevant features in the low “energy” (i.e., large eigenvalues) limit and thus providing universal descriptions allowing to associate the presence of the signal with objectives and computable quantities. Among them, in this paper, we focus on the vacuum expectation value. We exhibit experimental evidence in favor of a connection between symmetry breaking and the existence of an intrinsic detection threshold, in agreement with our conclusions for matrices, providing a new step in the direction of a universal statement.

On the asymptotic normality and efficiency of Kronecker envelope principal component analysis

Dimension reduction methods for matrix or tensor data have been an active research field in recent years. Li et al. (2010) introduced the notion of the Kronecker envelope and proposed dimension folding estimators for supervised dimension reduction. In a data analysis of cryogenic electron microscopy (cryo-EM) images (Chen et al., 2014), Kronecker envelope principal component analysis (PCA) was used to reduce the dimension of cryo-EM images. Kronecker envelope PCA is a two-step procedure, which consists of projecting data onto a multilinear envelope subspace as the first step, followed by ordinary PCA on the projected core tensor. The multilinear envelope subspace preserves the natural Kronecker product structure of observations when searching for the leading principal subspace. The main advantage of preserving the Kronecker product structure is the parsimonious usage of parameters in specifying the leading principal subspace, which mitigates the adverse influence of high-dimensionality. The method of PCA will convert possibly correlated variables to uncorrelated ones and further reduce the dimension of the projected core tensor. In this article we derive the asymptotic normality of Kronecker envelope PCA and compare it with ordinary PCA. Utilizing majorization theory, we show that Kronecker envelope PCA is asymptotically more efficient than ordinary PCA in the sense that the asymptotic total stochastic variation of Kronecker envelope PCA is smaller than that of ordinary PCA. A motivating real data example of cryo-EM image clustering and simulation studies are presented to show the merits of Kronecker envelope PCA.

Optimal sparse eigenspace and low-rank density matrix estimation for quantum systems

Quantum state tomography, which aims to estimate quantum states that are described by density matrices, plays an important role in quantum science and quantum technology. This paper examines the eigenspace estimation and the reconstruction of large low-rank density matrix based on Pauli measurements. Both ordinary principal component analysis (PCA) and iterative thresholding sparse PCA (ITSPCA) estimators of the eigenspace are studied, and their respective convergence rates are established. In particular, we show that the ITSPCA estimator is rate-optimal. We present the reconstruction of the large low-rank density matrix and obtain its optimal convergence rate by using the ITSPCA estimator. A numerical study is carried out to investigate the finite sample performance of the proposed estimators.

Dimensionality reduction for binary data through the projection of natural parameters

Principal component analysis (PCA) for binary data, known as logistic PCA, has become a popular alternative to dimensionality reduction of binary data. It is motivated as an extension of ordinary PCA by means of a matrix factorization, akin to the singular value decomposition, that maximizes the Bernoulli log-likelihood. We propose a new formulation of logistic PCA which extends Pearson’s formulation of a low dimensional data representation with minimum error to binary data. Our formulation does not require a matrix factorization, as previous methods do, but instead looks for projections of the natural parameters from the saturated model. Due to this difference, the number of parameters does not grow with the number of observations and the principal component scores on new data can be computed with simple matrix multiplication. We derive explicit solutions for data matrices of special structure and provide a computationally efficient algorithm for solving for the principal component loadings. Through simulation experiments and an analysis of medical diagnoses data, we compare our formulation of logistic PCA to the previous formulation as well as ordinary PCA to demonstrate its benefits.

Analyzing the efficiency of rewards through the principal component method

Subject. As economic processes get digitalized, management needs to address issues with new approaches to raising the efficiency of financial resources allocated for remuneration. Thus, the correlation of rewards and ultimate results of entities become a priority. Therefore, factors influencing the motivation and incentives of the staff and its performance underlie the reward system and the effectiveness of the entity. It is important for the management to measure the effect of the factors on the return on financial resources used for remuneration in order to attain goals and operate in the business environment effectively. Objectives. The study is to model the efficiency of rewards with reference to factors influencing the ratio of output to rewards, expertise and qualification of the staff. We use a linear model of multivariate regression on the basis of the Principal Component Analysis (PCA). Methods. The study relies upon the ordinary least squares methods and PCA, and general methods of research, such as an analysis, synthesis, induction, and deduction. Results. Conducting the managerial analysis of reward efficiency with reference of some determinants and their correlation (the output to reward ratio, staff's expertise and incentives), we evaluate internal corporate factors while handling new issues and outline future actions to set up the comprehensive reward system. The information helps improve the corporate performance and ensure the unbiased remuneration with reference to the employee’s contribution to the accomplishment of business goals. Conclusions and Relevance. The methodological approach we propose helps determine how the factors influence the reward mechanism in order to fulfill the corporate strategy objectives. The findings can be used for operational purposes of entities in order to set up the remuneration policy and tactics harmonizing pursuits of the business and staff.

Investigation of Volatiles in Cork Samples Using Chromatographic Data and the Superposing Significant Interaction Rules (SSIR) Chemometric Tool.

This study describes a new chemometric tool for the identification of relevant volatile compounds in cork by untargeted headspace solid phase microextraction and gas chromatography mass spectrometry (HS-SPME/GC-MS) analysis. The production process in cork industries commonly includes a washing procedure based on water and temperature cycles in order to reduce off-flavors and decrease the amount of trichloroanisole (TCA) in cork samples. The treatment has been demonstrated to be effective for the designed purpose, but chemical changes in the volatile fraction of the cork sample are produced, which need to be further investigated through the chemometric examination of data obtained from the headspace. Ordinary principal component analysis (PCA) based on the numerical description provided by the chromatographic area of several target compounds was inconclusive. This led us to consider a new tool, which is presented here for the first time for an application in the chromatographic field. The superposing significant interaction rules (SSIR) method is a variable selector which directly analyses the raw internal data coming from the spectrophotometer software and, combined with PCA and discriminant analysis, has been able to separate a group of 56 cork samples into two groups: treated and non-treated. This procedure revealed the presence of two compounds, furfural and 5-methylfurfural, which are increased in the case of treated samples. These compounds explain the sweet notes found in the sensory evaluation of the treated corks. The model that is obtained is robust; the overall sensitivity and specificity are 96% and 100%, respectively. Furthermore, a leave-one-out cross-validation calculation revealed that all of the samples can be correctly classified one at a time if three or more PCA descriptors are considered.

Sparse Principal Component Analysis for Natural Language Processing

High dimensional data are rapidly growing in many different disciplines, particularly in natural language processing. The analysis of natural language processing requires working with high dimensional matrices of word embeddings obtained from text data. Those matrices are often sparse in the sense that they contain many zero elements. Sparse principal component analysis is an advanced mathematical tool for the analysis of high dimensional data. In this paper, we study and apply the sparse principal component analysis for natural language processing, which can effectively handle large sparse matrices. We study several formulations for sparse principal component analysis, together with algorithms for implementing those formulations. Our work is motivated and illustrated by a real text dataset. We find that the sparse principal component analysis performs as good as the ordinary principal component analysis in terms of accuracy and precision, while it shows two major advantages: faster calculations and easier interpretation of the principal components. These advantages are very helpful especially in big data situations.

Modified PCA and PLS: Towards a better classification in Raman spectroscopy–based biological applications

AbstractRaman spectra of biological samples often exhibit variations originating from changes of spectrometers, measurement conditions, and cultivation conditions. Such unwanted variations make a classification extremely challenging, especially if they are more significant compared with the differences between groups to be separated. A classifier is prone to such unwanted variations (ie, intragroup variations) and can fail to learn the patterns that can help separate different groups (ie, intergroup differences). This often leads to a poor generalization performance and a degraded transferability of the trained model. A natural solution is to separate the intragroup variations from the intergroup differences and build the classifier based on merely the latter information, for example, by a well‐designed feature extraction. This forms the idea of this contribution. Herein, we modified two commonly applied feature extraction approaches, principal component analysis (PCA) and partial least squares (PLS), in order to extract merely the features representing the intergroup differences. Both of the methods were verified with two Raman spectral datasets measured from bacterial cultures and colon tissues of mice, respectively. In comparison to ordinary PCA and PLS, the modified PCA was able to improve the prediction on the testing data that bears significant difference to the training data, while the modified PLS could help avoid overfitting and lead to a more stable classification.

Random indexing of multidimensional data

Random indexing (RI) is a lightweight dimension reduction method, which is used, for example, to approximate vector semantic relationships in online natural language processing systems. Here we generalise RI to multidimensional arrays and therefore enable approximation of higher-order statistical relationships in data. The generalised method is a sparse implementation of random projections, which is the theoretical basis also for ordinary RI and other randomisation approaches to dimensionality reduction and data representation. We present numerical experiments which demonstrate that a multidimensional generalisation of RI is feasible, including comparisons with ordinary RI and principal component analysis. The RI method is well suited for online processing of data streams because relationship weights can be updated incrementally in a fixed-size distributed representation, and inner products can be approximated on the fly at low computational cost. An open source implementation of generalised RI is provided.

A Data-Adaptive Principal Component Analysis: Use of Composite Asymmetric Huber Function

This article considers a new type of principal component analysis (PCA) that adaptively reflects the information of data. The ordinary PCA is useful for dimension reduction and identifying important features of multivariate data. However, it uses the second moment of data only, and consequently, it is not efficient for analyzing real observations in the case that these are skewed or asymmetric data. To extend the scope of PCA to non-Gaussian distributed data that cannot be well represented by the second moment, a new approach for PCA is proposed. The core of the methodology is to use a composite asymmetric Huber function defined as a weighted linear combination of modified Huber loss functions, which replaces the conventional square loss function. A practical algorithm to implement the data-adaptive PCA is discussed. Results from numerical studies including simulation study and real data analysis demonstrate the promising empirical properties of the proposed approach. Supplementary materials for this article are available online.

Mapping of Fe mineral potential by spatially weighted principal component analysis in the eastern Tianshan mineral district, China

Principal component analysis (PCA) is one of the commonly used methods to integrate multi-source geological data to enhance understanding of geo-information. Each principal component (PC) obtained by PCA reflects a different aspect of the geo-information contained in a dataset. Confined by statistical significance, some PCs are not acceptable for interpretation. In this paper, the same problem in PCA occurs in mapping potential areas of Fe mineralization in eastern Tianshan mineral district, China. By spatially weighting correlation coefficient matrixes between variables, a spatially weighted principal component analysis (SWPCA) can deal with the shortcoming of PCA, thus improving the statistical acceptability of eigenvectors and eigenvalues derived by ordinary PCA. Based on the geological model in the study area, a current weighting factor is defined to enhance the geo-information possessed by the ordinary PC1. Compared with the loading of input layers on ordinary PCA, SWPC1 shows more significant physical meaning than PC1. Meanwhile, remarkable increases on the eigenvalues of SWPC2 and SWPC3 are demonstrated to exist making these spatially weighted principal components more acceptable in a statistical sense. In comparison with both loadings and scores on ordinary PCs, the improved geo-information carried by SWPCs can help with better interpretations of the geological phenomena.

Determining the number of principal factors by eigenvector comparison of the original bi-linear data matrix and the one reconstructed from key variables

It is an essential step in analyzing hyphenated chromatographic data of complex chemical systems to determine the number of principal factors of the bi-linear matrix. The determination is difficult due to the co-existence of non-chemical factors, such as background, noise, etc. A new method was proposed for the determination based on comparing eigenvectors of the original data matrix and the one reconstructed from key spectral variables that are selected with orthogonal projection approach (OPA). The proposed method is mathematically rigorous and the determination is clear. In comparison with other four indices, i.e., NPFPCA (noise perturbation in functional principal component analysis), RESO (the ratio of eigenvalues calculated by smoothed principal component analysis and those calculated by ordinary principal component analysis), DRAUG (determination of rank by augmentation) and DRMAD (determination of rank by median absolute deviation), this proposed method was proven to have good performance in both simulated GC-IR and experimental HPLC-DAD data.

An untargeted gas chromatography mass spectrometry metabolomics platform for marine polychaetes

The development of an appropriate extraction method for untargeted environmental metabolomic analysis of marine polychaetes could promote their use for environmental monitoring purposes. To this end, we compared four extraction methods on the marine polychaete Nereis virens both exposed to crude oil and non-exposed. XCMS was used for feature detection and preprocessing; different normalization and scaling approaches were tested; and principal component analysis (PCA) was used together with basic statistical tests to ascertain common metabolic patterns and determine the most suitable extraction method. We conclude that a two-step extraction procedure with 80:20 (v/v) methanol:water on freeze dried polychaete tissue provides the best trade-off between analysis time, and extraction efficiency and intermediate reproducibility. No definitive conclusions could be drawn about the ability of the method to discriminate controls and crude oils in actual biological replicates because the experiment was carried out by design on analytical replicates only. We show that the normalization to the sum of all the common features, and the use of a weighted least squares criterion to fit the PCA by means of scaling to the median absolute deviation (MAD) of the pooled quality control samples significantly improved the clustering of controls and crude oil exposed samples. The scaling alone led to an increase of 19% in explained variance compared to ordinary PCA.

Noise-free principal component analysis: An efficient dimension reduction technique for high dimensional molecular data

Principal component analysis (PCA) is one of the powerful dimension reduction techniques widely used in data mining field. PCA tries to project the data into lower dimensional space while preserving the intrinsic information hidden in the data as much as possible. Disadvantage of PCA is that, extracted principal components (PCs) are linear combination of all features, hence PCs are may still contaminated with noise in the data. To address this problem we propose a modified version of PCA called noise free PCA (NFPCA), in which regularization is introduced during the PCs extraction step to mitigate the effect of noise. Potentials of the proposed method is assessed in two important application of high-dimensional molecular data: classification and survival prediction. Multiple publicly available real-world data sets are used for this illustration. Experimental results show that, the NFPCA produce highly informative than the ordinary PCA method. This is largely due to the fact that the NFPCA suppress the effect of noise in the PCs more efficiently with minimum information lost. The NFPCA is a promising alternative to existing PCA approaches not only in terms of highly informative PCs, but also its relatively cheap computational cost.

IMPROVED REAL-TIME FACE RECOGNITION BASED ON THREE LEVEL WAVELET DECOMPOSITION-PRINCIPAL COMPONENT ANALYSIS AND MAHALANOBIS DISTANCE

The development of research in the field of real-time face recognition is a study that is being developed in the last decade. Face recognition is used to identify person from an image or video. Recognition rate and computation time of real-time face recognition is one of the big challenges that must be developed. This study proposes a model of face recognition using the method of feature extraction by combining three level wavelet decomposition and Principal Component Analysis (PCA) and using the method of mahalanobis distance for the classification section (3WPCA-MD). A 3-level wavelet decomposition is used to decompose images by reducing the resolution used for those images. Using wavelet decomposition up to level 3 will produce an image with a very low resolution so as to reduce the value of the resulting computation time to be processed using PCA. Mahalanobis distance method is used to determine the degree of similarity among the features to produce a more optimal face recognition. Based on the results of experiments that have been done, they generated improved face recognition with high face recognition accuracy of up to 96% in average and produced faster computation results of face recognition if compared to ordinary PCA method. The average computation speed value obtained using the method of 3WPCA-MD was 5-7 milli-second (ms) for each face recognition process.