Sparse Principal Component Analysis Research Articles

A novel fault detection approach based on multilinear sparse PCA: application onthe semiconductor manufacturing processes

Batch processes are extremely important to researchers since they are widely used in many fields such as biochemistry, pharmacy, and semiconductors. The powerful batch detection method is critical to increase the performance of the overall equipment and to reduce the use of check wafers. Many techniques have been used in batch process monitoring. Among them, the multivariate statistical process control (MSPC) is very useful in batch process monitoring because of the large number of records data. Therefore, batch processes have certain characteristics, such as multimodal batch nonlinearity trajectories, which were challenged by these MSPCs. In this paper, a novel process monitoring methods based on multilinear sparse PCA (MSPCA) are proposed to overcome these shortcomings. MSPCA handle batch data as a matrix (second order), although most of the other multivariate statistical analysis approaches handles batch data as a vector (first order). Vectorization of batch data tends to ignore parts of the information. Furthermore, The MSPCA can extract more useful data from the batch data with less storage requirements and computational complexity compared to current multivariate statistical analysis approaches. The efficiency of the monitoring technique is implemented in the numerical example and Lam 9600 metal etcher process. The performance of MPSCA is characterized by a fault detection rate of 100%, as well as a false alarm rate higher than 83%. Simulation results show that MSPCA outperforms the traditional techniques.

Sparsifying the least-squares approach to PCA: comparison of lasso and cardinality constraint

Sparse PCA methods are used to overcome the difficulty of interpreting the solution obtained from PCA. However, constraining PCA to obtain sparse solutions is an intractable problem, especially in a high-dimensional setting. Penalized methods are used to obtain sparse solutions due to their computational tractability. Nevertheless, recent developments permit efficiently obtaining good solutions of cardinality-constrained PCA problems allowing comparison between these approaches. Here, we conduct a comparison between a penalized PCA method with its cardinality-constrained counterpart for the least-squares formulation of PCA imposing sparseness on the component weights. We compare the penalized and cardinality-constrained methods through a simulation study that estimates the sparse structure’s recovery, mean absolute bias, mean variance, and mean squared error. Additionally, we use a high-dimensional data set to illustrate the methods in practice. Results suggest that using cardinality-constrained methods leads to better recovery of the sparse structure.

ISFun: an R package for integrative dimension reduction analysis.

In the analysis of high-dimensional omics data, dimension reduction techniques-including principal component analysis (PCA), partial least squares (PLS) and canonical correlation analysis (CCA)-have been extensively used. When there are multiple datasets generated by independent studies with compatible designs, integrative analysis has been developed and shown to outperform meta-analysis, other multidatasets analysis, and individual-data analysis. To facilitate integrative dimension reduction analysis in daily practice, we develop the R package iSFun, which can comprehensively conduct integrative sparse PCA, PLS and CCA, as well as meta-analysis and stacked analysis. The package can conduct analysis under the homogeneity and heterogeneity models and with the magnitude- and sign-based contrasted penalties. As a 'byproduct', this article is the first to develop integrative analysis built on the CCA technique, further expanding the scope of integrative analysis. The package is available at https://CRAN.R-project.org/package=iSFun. Supplementary materials are available at Bioinformatics online.

Coordinate Descent Without Coordinates: Tangent Subspace Descent on Riemannian Manifolds

We extend coordinate descent to manifold domains and provide convergence analyses for geodesically convex and nonconvex smooth objective functions. Our key insight is to draw an analogy between coordinate blocks in Euclidean space and tangent subspaces of a manifold. Hence, our method is called tangent subspace descent (TSD). The core principle behind ensuring convergence of TSD is the appropriate choice of subspace at each iteration. To this end, we propose two novel conditions, the (C, r)-norm and C-randomized norm conditions on deterministic and randomized modes of subspace selection, respectively, that promise convergence for smooth functions and that are satisfied in practical contexts. We propose two subspace selection rules, one deterministic and another randomized, of particular practical interest on the Stiefel manifold. Our proof-of-concept numerical experiments on the sparse principal component analysis problem demonstrate TSD’s efficacy. Funding: This work was supported by the National Science Foundation [Grant 1740707] and the Defense Advanced Research Projects Agency Lagrange Program [Grant N660011824020].

Enhancing Characteristic Gene Selection and Tumor Classification by the Robust Laplacian Supervised Discriminative Sparse PCA.

Characteristic gene selection and tumor classification of gene expression data play major roles in genomic research. Due to the characteristics of a small sample size and high dimensionality of gene expression data, it is a common practice to perform dimensionality reduction prior to the use of machine learning-based methods to analyze the expression data. In this context, classical principal component analysis (PCA) and its improved versions have been widely used. Recently, methods based on supervised discriminative sparse PCA have been developed to improve the performance of data dimensionality reduction. However, such methods still have limitations: most of them have not taken into consideration the improvement of robustness to outliers and noise, label information, sparsity, as well as capturing intrinsic geometrical structures in one objective function. To address this drawback, in this study, we propose a novel PCA-based method, known as the robust Laplacian supervised discriminative sparse PCA, termed RLSDSPCA, which enforces the L2,1 norm on the error function and incorporates the graph Laplacian into supervised discriminative sparse PCA. To evaluate the efficacy of the proposed RLSDSPCA, we applied it to the problems of characteristic gene selection and tumor classification problems using gene expression data. The results demonstrate that the proposed RLSDSPCA method, when used in combination with other related methods, can effectively identify new pathogenic genes associated with diseases. In addition, RLSDSPCA has also achieved the best performance compared with the state-of-the-art methods on tumor classification in terms of major performance metrics. The codes and data sets used in the study are freely available at http://csbio.njust.edu.cn/bioinf/rlsdspca/.

Research on Audit Opinion Prediction of Listed Companies Based on Sparse Principal Component Analysis and Kernel Fuzzy Clustering Algorithm

The prediction of audit opinions of listed companies plays a significant role in the security market risk prevention. By introducing machine learning methods, many innovations can be implemented to improve audit quality, lift audit efficiency, and cultivate the keen insight of auditors. However, in a realistic environment, category imbalance and critical feature selection exist in the prediction model of company audit opinions. This paper firstly combines batched sparse principal component analysis (BSPCA) with kernel fuzzy clustering algorithm (KFCM) and proposes a sparse-kernel fuzzy clustering undersampling method (S-KFCM) to deal with the imbalance of sample categories. This method adopts the kernel fuzzy clustering algorithm to down-sample the normal samples, and their features are extracted from abnormal sample sets based on the group sparse component method. The sparse normal sample set can maintain the original distribution space structure and highlight the classification boundary samples. Secondly, considering the company’s characteristic attributes and data sources, 448 original variables are grouped, and then BSPCA is used for feature screening. Finally, the support vector machine (SVM) is adopted to complete the classification prediction. According to the empirical results, the SKFCM-SVM model has the highest prediction accuracy.

Factor analysis, sparse PCA, and Sum of Ranking Differences-based improvements of the Promethee-GAIA multicriteria decision support technique.

The Promethee-GAIA method is a multicriteria decision support technique that defines the aggregated ranks of multiple criteria and visualizes them based on Principal Component Analysis (PCA). In the case of numerous criteria, the PCA biplot-based visualization do not perceive how a criterion influences the decision problem. The central question is how the Promethee-GAIA-based decision-making process can be improved to gain more interpretable results that reveal more characteristic inner relationships between the criteria. To improve the Promethee-GAIA method, we suggest three techniques that eliminate redundant criteria as well as clearly outline, which criterion belongs to which factor and explore the similarities between criteria. These methods are the following: A) Principal factoring with rotation and communality analysis (P-PFA), B) the integration of Sparse PCA into the Promethee II method (P-sPCA), and C) the Sum of Ranking Differences method (P-SRD). The suggested methods are presented through an I4.0+ dataset that measures the Industry 4.0 readiness of NUTS 2-classified regions. The proposed methods are useful tools for handling multicriteria ranking problems, if the number of criteria is numerous.

Emotional Calculation Method of Rural Tourist Based on Improved SPCA-LSTM Algorithm

New technologies such as big data and cloud computing provide new means and tools for rural development. The big rural tourism, with its convenience, quickness, and low threshold, presents great convenience for tourists’ emotional calculation and has become one of the main sources of tourism big data. Under the guidance of big data theory and emotion theory, this paper proposes an emotional calculation method of rural tourists based on improved SPCA-LSTM algorithm, taking big text data as data source. Firstly, the improved TF-IDF algorithm is designed to highlight the importance of feature items, and the word vector trained by word2vec model is applied to represent the rural tourism data text. Then, a weighted sparse PCA (RSPCA) is constructed to reduce the dimension of massive word vector features. RSPCA introduces the weighted l 1 optimization framework and LASSO regression model into the mathematical model of PCA algorithm and establishes a new data dimension reduction model. Thereupon, the long-term and short-term memory convolution network with attention mechanism is employed to extract text features. Finally, the feature vector is utilized to calculate the rural tourist’s emotion by softmax function. The experimental results indicate that the improved SPCA-LSTM algorithm, whose performance index is better than other existing algorithms, is effective in calculating tourists’ emotions. Also, it is more suitable for the research of tourist sentiment calculation in the era of big data.

Classification of Alzheimer's Disease Based on Abnormal Hippocampal Functional Connectivity and Machine Learning.

ObjectiveAlzheimer’s disease (AD) is a neurodegenerative disease characterized by progressive deterioration of memory and cognition. Mild cognitive impairment (MCI) has been implicated as a prodromal phase of AD. Although abnormal functional connectivity (FC) has been demonstrated in AD and MCI, the clinical differentiation of AD, MCI, and normal aging remains difficult, and the distinction between MCI and normal aging is especially problematic. We hypothesized that FC between the hippocampus and other brain structures is altered in AD and MCI, and that measurement of abnormal FC could have diagnostic utility for the classification of different AD stages.MethodsElderly adults aged 60–85 years were assigned to AD, MCI, or normal control (NC) groups based on clinical criteria. Functional magnetic resonance scanning was completed by 119 subjects. Five dimension reduction/classification methods were applied, using hippocampus-derived FC strengths as input features. Classification performance of the five dimensionality reduction methods was compared between AD, MCI, and NC groups.ResultsFCs between the hippocampus and left insula, left thalamus, cerebellum, right lingual gyrus, posterior cingulate cortex, and precuneus were significantly reduced in AD and MCI. Support vector machine learning coupled with sparse principal component analysis demonstrated the best discriminative performance, yielding classification accuracies of 82.02% (AD vs. NC), 81.33% (MCI vs. NC), and 81.08% (AD vs. MCI).ConclusionHippocampus-seed-based FCs were significantly different between AD, MCI, and NC groups. FC assessment combined with widely used machine learning methods can improve AD differential diagnosis, and may be especially useful to distinguish MCI from normal aging.

Sparse PCA on fixed-rank matrices

Sparse PCA is the optimization problem obtained from PCA by adding a sparsity constraint on the principal components. Sparse PCA is NP-hard and hard to approximate even in the single-component case. In this paper we settle the computational complexity of sparse PCA with respect to the rank of the covariance matrix. We show that, if the rank of the covariance matrix is a fixed value, then there is an algorithm that solves sparse PCA to global optimality, whose running time is polynomial in the number of features. We also prove a similar result for the version of sparse PCA which requires the principal components to have disjoint supports.

Geochemical anomaly mapping using sparse principal component analysis in Jining, Inner Mongolia, China

Mapping geochemical anomalies is one of the most important tasks in mineral exploration. Sparse principal component analysis (SPCA) can be applied to investigate the correlations among geochemical elements and geological variables, which forces the loading values of the partial elements to zero. In this paper, the robust SPCA algorithm was used to process geochemical exploration data on a scale of 1:50,000 for identifying geochemical anomalies related to mineralization. The results show SPC1 and SPC3 indicate the spatial distribution of the Hannuoba basalt and Jining Group formation, respectively. The outcropped Hannuoba formations are located in high-value area of scores map of SPC1. The element association of W-Mo on SPC2 is related to molybdenum (Mo) mineralization. The spatial distribution of SPC2 scores suggests the distribution of Mo mineralization. The discovered Mo polymetallic deposits are located in the anomalous area of SPC2 scores. The SPCs identify localized geochemical anomalies and disambiguate the effects of outliers, which improve interpretability. These results indicate SPCA is a powerful tool for mapping geochemical anomalies.

A Fast, Provably Accurate Approximation Algorithm for Sparse Principal Component Analysis Reveals Human Genetic Variation Across the World.

Principal component analysis (PCA) is a widely used dimensionality reduction technique in machine learning and multivariate statistics. To improve the interpretability of PCA, various approaches to obtain sparse principal direction loadings have been proposed, which are termed Sparse Principal Component Analysis (SPCA). In this paper, we present ThreSPCA, a provably accurate algorithm based on thresholding the Singular Value Decomposition for the SPCA problem, without imposing any restrictive assumptions on the input covariance matrix. Our thresholding algorithm is conceptually simple; much faster than current state-of-the-art; and performs well in practice. When applied to genotype data from the 1000 Genomes Project, ThreSPCA is faster than previous benchmarks, at least as accurate, and leads to a set of interpretable biomarkers, revealing genetic diversity across the world.

Convexification of Permutation-Invariant Sets and an Application to Sparse Principal Component Analysis

We develop techniques to convexify a set that is invariant under permutation and/or change of sign of variables and discuss applications of these results. First, we convexify the intersection of the unit ball of a permutation and sign-invariant norm with a cardinality constraint. This gives a nonlinear formulation for the feasible set of sparse principal component analysis (PCA) and an alternative proof of the K-support norm. Second, we characterize the convex hull of sets of matrices defined by constraining their singular values. As a consequence, we generalize an earlier result that characterizes the convex hull of rank-constrained matrices whose spectral norm is below a given threshold. Third, we derive convex and concave envelopes of various permutation-invariant nonlinear functions and their level sets over hypercubes, with congruent bounds on all variables. Finally, we develop new relaxations for the exterior product of sparse vectors. Using these relaxations for sparse PCA, we show that our relaxation closes 98% of the gap left by a classical semidefinite programming relaxation for instances where the covariance matrices are of dimension up to 50 × 50.

Fluorescence Excitation–Emission Matrix Spectroscopy and Boosting Regression Tree Model to Detect Dissolved Organic Carbon in Water

In recent years, optical methods have been proven to be a powerful tool for m onitoring dissolved organic carbon (DOC) in natural waters. However, the effectiveness of this method in marine systems with low DOC concentrations remains to be shown. Herein, a new method based on fluorescence excitation–emission matrix spectroscopy for seawater DOC quantification is proposed. Pre-processing method is investigated to achieve a high signal to noise ratio. Peak-picking operation is then performed to obtain feature peaks. In order to combine the information from sparsely located feature peaks, sparse principal component analysis is applied to identifying important variables used in the following regression procedure. Under these conditions the result of regression analysis can be obtained readily in a given data set coupling with boosting regression tree. The method was tested on samples collected from the East China Sea. Compared to the parallel factor analysis–multivariate linear regression method, experimental results show that the proposed method achieved a more consistent regression output and indicate that the boosting regression tree has potential for DOC quantification even at low concentrations.

Exploring the coupled ocean and atmosphere system with a data science approach applied to observations from the Antarctic Circumnavigation Expedition

Abstract. The Southern Ocean is a critical component of Earth's climate system, but its remoteness makes it challenging to develop a holistic understanding of its processes from the small scale to the large scale. As a result, our knowledge of this vast region remains largely incomplete. The Antarctic Circumnavigation Expedition (ACE, austral summer 2016/2017) surveyed a large number of variables describing the state of the ocean and the atmosphere, the freshwater cycle, atmospheric chemistry, and ocean biogeochemistry and microbiology. This circumpolar cruise included visits to 12 remote islands, the marginal ice zone, and the Antarctic coast. Here, we use 111 of the observed variables to study the latitudinal gradients, seasonality, shorter-term variations, geographic setting of environmental processes, and interactions between them over the duration of 90 d. To reduce the dimensionality and complexity of the dataset and make the relations between variables interpretable we applied an unsupervised machine learning method, the sparse principal component analysis (sPCA), which describes environmental processes through 14 latent variables. To derive a robust statistical perspective on these processes and to estimate the uncertainty in the sPCA decomposition, we have developed a bootstrap approach. Our results provide a proof of concept that sPCA with uncertainty analysis is able to identify temporal patterns from diurnal to seasonal cycles, as well as geographical gradients and “hotspots” of interaction between environmental compartments. While confirming many well known processes, our analysis provides novel insights into the Southern Ocean water cycle (freshwater fluxes), trace gases (interplay between seasonality, sources, and sinks), and microbial communities (nutrient limitation and island mass effects at the largest scale ever reported). More specifically, we identify the important role of the oceanic circulations, frontal zones, and islands in shaping the nutrient availability that controls biological community composition and productivity; the fact that sea ice controls sea water salinity, dampens the wave field, and is associated with increased phytoplankton growth and net community productivity possibly due to iron fertilisation and reduced light limitation; and the clear regional patterns of aerosol characteristics that have emerged, stressing the role of the sea state, atmospheric chemical processing, and source processes near hotspots for the availability of cloud condensation nuclei and hence cloud formation. A set of key variables and their combinations, such as the difference between the air and sea surface temperature, atmospheric pressure, sea surface height, geostrophic currents, upper-ocean layer light intensity, surface wind speed and relative humidity played an important role in our analysis, highlighting the necessity for Earth system models to represent them adequately. In conclusion, our study highlights the use of sPCA to identify key ocean–atmosphere interactions across physical, chemical, and biological processes and their associated spatio-temporal scales. It thereby fills an important gap between simple correlation analyses and complex Earth system models. The sPCA processing code is available as open-access from the following link: https://renkulab.io/gitlab/ACE-ASAID/spca-decomposition (last access: 29 March 2021). As we show here, it can be used for an exploration of environmental data that is less prone to cognitive biases (and confirmation biases in particular) compared to traditional regression analysis that might be affected by the underlying research question.

Riemannian proximal stochastic gradient descent for sparse 2DPCA

Sparse two-dimensional principal component analysis (2DPCA) is an effective dimensionality reduction and data processing technique, which has been successfully applied in face recognition. Sparse 2DPCA is based on 2D image matrices instead of 1D vectors so it can well exploit the structural information of image pixels. The sparse 2DPCA model is regarded as a non-convex and non-smooth composite optimization problem. The manifold proximal gradient algorithm (ManPG) that is efficient and robust can globally converge to an optimal point in solving this kind of problem on the Stiefel manifold, such as sparse PCA. ManPG uses the proximal gradient method for the non-smooth term of this problem, which yields a convex subproblem on the tangent space. By solving this subproblem, a descent direction on the tangent space is obtained. Then a new update point on the manifold is obtained via retraction. However, in solving large-scale problems, the computational complexity of ManPG is quite high since ManPG requires calculating the full gradient in each iteration. In order to reduce the computational complexity of each iteration, a more computationally efficient Riemannian proximal stochastic gradient descent algorithm (RPSGD) is proposed, which utilizes the stochastic gradient instead of the full gradient in ManPG. We prove that the iteration complexity of the proposed RPSGD is O(1/ϵ4) while it maintains the sublinear convergence for the large-scale sparse 2DPCA. The experimental results on face recognition and handwritten digits classification data sets show RPSGD has the advantages of small-time overhead, high test accuracy, good reconstructed image quality, and relatively sparse solution in solving the large-scale sparse 2DPCA.

Marine pollutant exposures and human milk extracellular vesicle-microRNAs in a mother-infant cohort from the Faroe Islands

Background/AimsEarly life exposures to marine contaminants can adversely impact child health but modes of action are unclear. Human milk contains extracellular vesicles (EVs) that can transport biologically relevant cargo from mother to infant, including microRNAs (miRNAs), and may partly mediate the effects of pollutants on child health. However, the role of marine pollutants on miRNA expression in milk EVs is unexplored. MethodsWe isolated EV RNA from 333 milk samples collected between 2 and 74 days postpartum from a Faroese birth cohort born 1997–2000 and sequenced 2083 miRNAs using a targeted library preparation method. We quantified five perfluoroalkyl substances (PFAS), pesticide metabolite p,p’-dichlorodiphenyldichloroethylene (DDE), and the sum of three major polychlorinated biphenyls (ΣPCBs) in maternal serum at 34 weeks of gestation and maternal hair total mercury (Hg) at birth. We used negative binomial regressions to estimate associations between individual pollutants and 418 reliably expressed EV-miRNAs adjusted for potential confounders. We performed sparse principal components (PCs) analysis to derive the first four components of the EV-miRNA data and examined associations between pollutants and PCs using Bayesian kernel machine regression (BKMR). ResultsWe observed no associations between pollutants and individual EV-miRNA expression after controlling the false discovery rate at 0.1. However, BKMR suggested that Hg was positively associated with PC1 and negatively associated with PC3, while ΣPCBs was negatively associated with PC3, and two PFAS were associated with PC4. Exploration of PC loadings followed by pathway analyses suggested that miRNAs in PC1 (miR-200b-3p, miR-664a-3p, miR-6738-5p, miR-429, miR-1236-5p, miR-4464, and miR-30b-5p) may be related to Hg neurotoxicity, while remaining PCs require further research. ConclusionsOur findings suggest that groups of milk EV-miRNAs may better serve as environmental biomarkers than individual miRNAs. Future studies are needed to elucidate the role of milk EV-miRNAs in child health following prenatal exposures.

Novel variation mode decomposition integrated adaptive sparse principal component analysis and it application in fault diagnosis

The sparse principal component analysis (SPCA) is widely used in the fault detection for nonlinear complex chemical processes in recent years. However, insufficient data processing, fixed models and fault type single classification cannot be used in the time-varying process. Therefore, a novel adaptive sparse principal component analysis (ASPCA) algorithm fused with improved variation mode decomposition (IVMD) (ASPCA-IVMD) is proposed for fault detection in chemical processes. The bat algorithm is innovatively integrated to optimize the parameters of the variable modulus decomposition. Then the optimized parameters are used for data preprocessing to suppress noise. In addition, based on the traditional SPCA, the threshold calculation is fused to realize the adaptive selection of principal components. After the principal components are determined, T2 and Q statistics are used for fault detection. Finally, the proposed method is verified by the Tennessee Eastman process case. The results demonstrate that the proposed method can select the principal components adaptively according to the data for having the real-time property of chemical process. Meanwhile, compared with traditional methods (principal component analysis, sparse principal component analysis, deep belief network integrating dropout, adaptive unscented Kalman filter integrating radial basis function and sparse deep belief network), the detection rate of the ASPCA-IVMD method is more than 99%, which shows superiority.

Using ℓ1-Relaxation and Integer Programming to Obtain Dual Bounds for Sparse PCA

Dual Bounds of Sparse Principal Component Analysis Sparse principal component analysis (PCA) is a widely used dimensionality reduction tool in machine learning and statistics. Compared with PCA, sparse PCA enhances the interpretability by incorporating a sparsity constraint. However, unlike PCA, conventional heuristics for sparse PCA cannot guarantee the qualities of obtained primal feasible solutions via associated dual bounds in a tractable fashion without underlying statistical assumptions. In “Using L1-Relaxation and Integer Programming to Obtain Dual Bounds for Sparse PCA,” Santanu S. Dey, Rahul Mazumder, and Guanyi Wang present a convex integer programming (IP) framework of sparse PCA to derive dual bounds. They show the worst-case results on the quality of the dual bounds provided by the convex IP. Moreover, the authors empirically illustrate that the proposed convex IP framework outperforms existing sparse PCA methods of finding dual bounds.

Eigenvectors from Eigenvalues Sparse Principal Component Analysis (EESPCA)

We present a novel technique for sparse principal component analysis. This method, named Eigenvectors from Eigenvalues Sparse Principal Component Analysis (EESPCA), is based on the formula for computing squared eigenvector loadings of a Hermitian matrix from the eigenvalues of the full matrix and associated sub-matrices. We explore two versions of the EESPCA method: a version that uses a fixed threshold for inducing sparsity and a version that selects the threshold via cross-validation. Relative to the state-of-the-art sparse PCA methods of Witten et al., Yuan & Zhang and Tan et al., the fixed threshold EESPCA technique offers an order-of-magnitude improvement in computational speed, does not require estimation of tuning parameters via cross-validation, and can more accurately identify true zero principal component loadings across a range of data matrix sizes and covariance structures. Importantly, the EESPCA method achieves these benefits while maintaining out-of-sample reconstruction error and PC estimation error close to the lowest error generated by all evaluated approaches. EESPCA is a practical and effective technique for sparse PCA with particular relevance to computationally demanding statistical problems such as the analysis of high-dimensional data sets or application of statistical techniques like resampling that involve the repeated calculation of sparse PCs.