Sparse Principal Component Analysis Research Articles

Spike and slab Bayesian sparse principal component analysis

Spike and slab Bayesian sparse principal component analysis

Dynamic sparse PCA: a dimensional reduction method for sensor data in virtual metrology

Virtual metrology (VM) is the ideal technique for predicting the quality of every wafer and reducing overall inspection costs in semiconductor manufacturing. The data from numerous monitoring sensors in semiconductor production poses challenges in terms of data dimensionality. Thus, dimensionality reduction techniques are required before building the prediction model in VM. Additionally, to reduce the sensor cost, it is crucial to determine the sensors that contribute significantly to the output of the prediction model. Therefore, in this paper, we propose a dimensionality reduction technique called dynamic sparse principal component analysis to establish global sparsity in the first few principal components. The proposed method seeks extracted features that involve a limited number of sensors in the VM to reduce sensor cost and increase the interpretability of the features while reasonably capturing data variability. The experimental results demonstrate that the proposed method is not sensitive to the scale of the problem and can solve the large-scale high-dimensional problems that are commonly encountered in semiconductor manufacturing.

Coupling randomisation and sparse modelling for the exploratory analysis of large hyperspectral datasets

Sparse-based models are a powerful tools for data compression, variable reduction, and model complexity reduction. Nevertheless, their major issue is the high computational time needed in large matrices. This manuscript proposes, for the first time, to couple randomised decomposition as a first step before sparsity calculations, followed by a projection of the full data onto a reduced-sparse set of loadings that will drastically reduce the time needed for calculations and built models that are equally reliable as their sparse-based homologous. While this new approach might be valid for several scenarios (exploration, regression and classification), we will focus on exploration methods (like Principal Component Analysis – PCA) applied to large datasets of hyperspectral images. Two datasets of different complexity have been tested, and the benefits of the coupled randomisation and sparse PCA (rsPCA) are extensively studied.

High-dimensional Statistical Analysis and Its Application to an ALMA Map of NGC 253

In astronomy, if we denote the dimension of data as d and the number of samples as n, we often find a case with n ≪ d. Traditionally, such a situation is regarded as ill-posed, and there was no choice but to discard most of the information in data dimensions to let d < n. The data with n ≪ d is referred to as the high-dimensional low sample size (HDLSS). To deal with HDLSS problems, a method called high-dimensional statistics has rapidly developed in the last decade. In this work, we first introduce high-dimensional statistical analysis to the astronomical community. We apply two representative methods in the high-dimensional statistical analysis methods, noise-reduction principal component analysis (NRPCA) and automatic sparse principal component analysis (A-SPCA), to a spectroscopic map of a nearby archetype starburst galaxy NGC 253 taken by the Atacama Large Millimeter/submillimeter Array (ALMA). The ALMA map is an example of a typical HDLSS data set. First, we analyzed the original data, including the Doppler shift due to the systemic rotation. High-dimensional PCA can precisely describe the spatial structure of the rotation. We then applied to the Doppler-shift corrected data to analyze more subtle spectral features. NRPCA and R-SPCA were able to quantify the very complicated characteristics of the ALMA spectra. Particularly, we were able to extract information on the global outflow from the center of NGC 253. This method can also be applied not only to spectroscopic survey data, but also to any type of data with a small sample size and large dimension.

Principal Component Analysis variants for Parkinson datasets

Principal Component Analysis (PCA) is one of the most fundamental dimension reduction methods that need further research. With the widespread popularity of machine learning and the arrival of the era of big data, dimension reduction has become a hot topic and principal component analysis is a hot topic. However, although there are a lot of researchers who focus on the methods of the PCA, few researches on Parkinson Datasets have been made. As a result, the aim of our work is to discuss the PCA variants for Parkinson Datasets. This paper first introduces the three most commonly used PCA methods: PCA, Sparse PCA and Kernel PCA, and then introduces the Support Vector Machine (SVM) used to measure the dimension reduction effect. After that, we introduced the Parkinson's dataset and the meanings of root mean square error (RMSE), overall accuracy, Cohens kappa (Kappa) and computational time, the indicators that are used to measure the dimensionality reduction effect. Finally, we identified the variants among different PCA methods on the Parkinson dataset by comparing the indicators of the data obtained after dimensionality reduction using different methods.

Lithium exploration targeting through robust variable selection and deep anomaly detection: An integrated application of sparse principal component analysis and stacked autoencoders

Lithium is a strategic metal for high-technology industries that plays a vital role in realizing electromobility and effective energy storage for smartphones and electric/hybrid vehicles, in addition to being important in accumulating energy from renewable sources. Motivated by this, the development of state-of-the-art workflows is imperative to make Li exploration more efficient and guarantee a sustainable supply to the demanding markets in the future. In this paper, a computational protocol was described to address two critical challenges raised in the Li exploration: (i) selection of mineralization-related pathfinder variables and (ii) recognition of multi-element geochemical anomalies related to ore-forming processes. Robust sparse principal component analysis (SPCA) was employed to reduce active (non-zero) loading coefficients and straightforward selection of Li pathfinder variables. The selected variables were then fed into the training network of stacked autoencoders (SAE) to learn deep representations of multivariate input signals and quantify reconstruction errors linked to Li-vectoring geochemical anomalies. As a representative example, the proposed workflow, i.e., SPCA + SAE, was implemented in the R programming language and applied to a real-case experiment with stream sediment geochemical data pertaining to the Moalleman district, NE Iran. Moreover, compositional robust principal component analysis (RPCA) and Mahalanobis distance (MD) technique were adopted to constitute two comparative models, RPCA + SAE and SPCA + MD, as benchmarks to judge the competence of the SPCA + SAE model in variable selection and anomaly detection, respectively. A performance appraisal by success-rate curves and relevant area under the curves (AUCs) indicated that the SPCA + SAE model brings, by calculating the highest AUC, spatial patterns that are more promising for vectoring towards Li-mineralized grounds. Moreover, AUCSPCA+MD was measured to be greater than AUCRPCA+SAE, suggesting robust variable selection is a more important paradigm than deep anomaly detection for Li exploration targeting. Student's t–statistic method was eventually applied to the anomaly map from the SPCA + SAE model to define relevant thresholds for narrowing down prospective areas for the next round of Li exploration within the study area.

Robust and Sparse Principal Component Analysis With Adaptive Loss Minimization for Feature Selection.

Principal component analysis (PCA) is one of the most successful unsupervised subspace learning methods and has been used in many practical applications. To deal with the outliers in real-world data, robust principal analysis models based on various measure are proposed. However, conventional PCA models can only transform features to unknown subspace for dimensionality reduction and cannot perform features' selection task. In this article, we propose a novel robust PCA (RPCA) model to mitigate the impact of outliers and conduct feature selection, simultaneously. First, we adopt σ -norm as reconstruction error (RE), which plays an important role in robust reconstruction. Second, to conduct feature selection task, we apply l2,0 -norm constraint to subspace projection. Furthermore, an efficient iterative optimization algorithm is proposed to solve the objective function with nonconvex and nonsmooth constraint. Extensive experiments conducted on several real-world datasets demonstrate the effectiveness and superiority of the proposed feature selection model.

Beyond PCA: Additional Dimension Reduction Techniques to Consider in the Development of Climate Fingerprints

Abstract Dimension reduction techniques are an essential part of the climate analyst’s toolkit. Due to the enormous scale of climate data, dimension reduction methods are used to identify major patterns of variability within climate dynamics, to create compelling and informative visualizations, and to quantify major named modes such as El Niño–Southern Oscillation. Principal components analysis (PCA), also known as the method of empirical orthogonal functions (EOFs), is the most commonly used form of dimension reduction, characterized by a remarkable confluence of attractive mathematical, statistical, and computational properties. Despite its ubiquity, PCA suffers from several difficulties relevant to climate science: high computational burden with large datasets, decreased statistical accuracy in high dimensions, and difficulties comparing across multiple datasets. In this paper, we introduce several variants of PCA that are likely to be of use in climate sciences and address these problems. Specifically, we introduce non-negative, sparse, and tensor PCA and demonstrate how each approach provides superior pattern recognition in climate data. We also discuss approaches to comparing PCA-family results within and across datasets in a domain-relevant manner. We demonstrate these approaches through an analysis of several runs of the E3SM climate model from 1991 to 1995, focusing on the simulated response to the Mt. Pinatubo eruption; our findings are consistent with a recently identified stratospheric warming fingerprint associated with this type of stratospheric aerosol injection.

Predicting porosity in wire arc additive manufacturing (WAAM) using wavelet scattering networks and sparse principal component analysis

Predicting porosity in wire arc additive manufacturing (WAAM) using wavelet scattering networks and sparse principal component analysis

Optimizing the Combination of Common Clinical Concussion Batteries to Predict Persistent Postconcussion Symptoms in a Prospective Cohort of Concussed Youth.

Studies have evaluated individual factors associated with persistent postconcussion symptoms (PPCS) in youth concussion, but no study has combined individual elements of common concussion batteries with patient characteristics, comorbidities, and visio-vestibular deficits in assessing an optimal model to predict PPCS. To determine the combination of elements from 4 commonly used clinical concussion batteries and known patient characteristics and comorbid risk factors that maximize the ability to predict PPCS. Cohort study; Level of evidence, 2. We enrolled 198 concussed participants-87 developed PPCS and 111 did not-aged 8 to 19 years assessed within 14 days of injury from a suburban high school and the concussion program of a tertiary care academic medical center. We defined PPCS as a Post-Concussion Symptom Inventory (PCSI) score at 28 days from injury of ≥3 points compared with the preinjury PCSI score-scaled for younger children. Predictors included the individual elements of the visio-vestibular examination (VVE), Sport Concussion Assessment Tool, 5th Edition (SCAT-5), King-Devick test, and PCSI, in addition to age, sex, concussion history, and migraine headache history. The individual elements of these tests were grouped into interpretable factors using sparse principal component analysis. The 12 resultant factors were combined into a logistic regression and ranked by frequency of inclusion into the combined optimal model, whose predictive performance was compared with the VVE, initial PCSI, and the current existing predictive model (the Predicting and Prevention Postconcussive Problems in Pediatrics (5P) prediction rule) using the area under the receiver operating characteristic curve (AUC). A cluster of 2 factors (SCAT-5/PCSI symptoms and VVE near point of convergence/accommodation) emerged. A model fit with these factors had an AUC of 0.805 (95% CI, 0.661-0.929). This was a higher AUC point estimate, with overlapping 95% CIs, compared with the PCSI (AUC, 0.773 [95% CI, 0.617-0.912]), VVE (AUC, 0.736 [95% CI, 0.569-0.878]), and 5P Prediction Rule (AUC, 0.728 [95% CI, 0.554-0.870]). Among commonly used clinical assessments for youth concussion, a combination of symptom burden and the vision component of the VVE has the potential to augment predictive power for PPCS over either current risk models or individual batteries.

High-dimensional sparse predictive modeling applied to varied correlation structure via the generalized additive model

This paper explores the characteristics of a two-step procedure (dimension reduction and function approximation) in discrete choice modeling with high-dimension data. This study proposes the SS-GAM procedure which is an extension of the Super Sparse Principal Component Analysis (SSPCA) where the results are further processed with the Generalized Additive Model (GAM) in a classification problem. Moreover, the Orthogonal Sparse Principal Component Analysis with GAM (OS-GAM) is also proposed. For baseline comparison, the General Adaptive Sparse-PCA with GAM (GAS-GAM) is considered in this paper. The performance of these three sparse PCA methods are investigated with varied underlying correlation structure. In the simulation study, it is demonstrated that with varied degree of dimensionality, and levels of correlation structure, SS-GAM performed better compared to OS-GAM and GAS-GAM in terms of its predictive rate, on the average. It was observed that the OS-GAM performed best when data exhibits low correlation structure. However, with high correlation structure, OS-GAM and GAS-GAM obtained comparable result. Moreover, in terms of computational time, OS-GAM seems to be not affected by the increase of feature dimension.

Exploring the Impact of the New Crown Epidemic on Tourism in Qiannan, Guizhou Using Principal Component Analysis Methods

Abstract Principal component analysis is usually a linear combination of all variables, which is very detrimental to the interpretation of the results. Therefore, this paper proposes sparse principal component analysis for analyzing the impact of the Xinguang epidemic on tourism, which can find linear combinations containing only a few input variables, effectively filter out sparse principal components, and achieve the purpose of explaining a high percentage of changes with sparse linear combinations. By observing the trend of accommodation, food and beverage, and tourism turnover data in Guizhou Qiannan region before and after the epidemic from 2015 to 2021, it was confirmed that the data of 2020 was mainly used as the dataset. For sparse principal component analysis, four data sets were selected, each with 14 indicators and 32 samples. The findings revealed that the tourism topics of interest during the epidemic were more spread out and had a lower concentration of links among each other. In the accommodation industry, the 1st principal component is turnover, which has a common factor variance of 0.995 and an eigenvalue of 13.408. In the catering industry, the 2 principal components can be interpreted as the major category of operating costs and the major category of turnover, with component matrix values of (0.997, -0.073) and (0.996, 0.064), respectively.

Performance Evaluation of Matrix Factorization for fMRI Data.

A hypothesis in the study of the brain is that sparse coding is realized in information representation of external stimuli, which has been experimentally confirmed for visual stimulus recently. However, unlike the specific functional region in the brain, sparse coding in information processing in the whole brain has not been clarified sufficiently. In this study, we investigate the validity of sparse coding in the whole human brain by applying various matrix factorization methods to functional magnetic resonance imaging data of neural activities in the brain. The result suggests the sparse coding hypothesis in information representation in the whole human brain, because extracted features from the sparse matrix factorization (MF) method, sparse principal component analysis (SparsePCA), or method of optimal directions (MOD) under a high sparsity setting or an approximate sparse MF method, fast independent component analysis (FastICA), can classify external visual stimuli more accurately than the nonsparse MF method or sparse MF method under a low sparsity setting.

A New Basis for Sparse Principal Component Analysis

Previous versions of sparse principal component analysis (PCA) have presumed that the eigen-basis (a p × k matrix) is approximately sparse. We propose a method that presumes the p × k matrix becomes approximately sparse after a k × k rotation. The simplest version of the algorithm initializes with the leading k principal components. Then, the principal components are rotated with an k × k orthogonal rotation to make them approximately sparse. Finally, soft-thresholding is applied to the rotated principal components. This approach differs from prior approaches because it uses an orthogonal rotation to approximate a sparse basis. One consequence is that a sparse component need not to be a leading eigenvector, but rather a mixture of them. In this way, we propose a new (rotated) basis for sparse PCA. In addition, our approach avoids “deflation” and multiple tuning parameters required for that. Our sparse PCA framework is versatile; for example, it extends naturally to a two-way analysis of a data matrix for simultaneous dimensionality reduction of rows and columns. We provide evidence showing that for the same level of sparsity, the proposed sparse PCA method is more stable and can explain more variance compared to alternative methods. Through three applications—sparse coding of images, analysis of transcriptome sequencing data, and large-scale clustering of social networks, we demonstrate the modern usefulness of sparse PCA in exploring multivariate data. An R package, epca, and the supplementary materials for this article are available online.

Maximizing the Accuracy of Adolescent Concussion Diagnosis Using Individual Elements of Common Standardized Clinical Assessment Tools.

Multiple clinical evaluation tools exist for adolescent concussion with various degrees of correlation, presenting challenges for clinicians in identifying which elements of these tools provide the greatest diagnostic utility. To determine the combination of elements from 4 commonly used clinical concussion batteries that maximize discrimination of adolescents with concussion from those without concussion. Cross-sectional study. Suburban school and concussion program of a tertiary care academic center. A total of 231 participants with concussion (from a suburban school and a concussion program) and 166 participants without concussion (from a suburban school) between the ages of 13 and 19 years. Individual elements of the visio-vestibular examination (VVE), Sport Concussion Assessment Tool, fifth edition (SCAT5; including the modified Balance Error Scoring System), King-Devick test (K-D), and Postconcussion Symptom Inventory (PCSI) were evaluated. The 24 subcomponents of these tests were grouped into interpretable factors using sparse principal component analysis. The 13 resultant factors were combined with demographic and clinical covariates into a logistic regression model and ranked by frequency of inclusion into the ideal model, and the predictive performance of the ideal model was compared with each of the clinical batteries using the area under the receiver operating characteristic curve (AUC). A cluster of 4 factors (factor 1 [VVE saccades and vestibulo-ocular reflex], factor 2 [modified Balance Error Scoring System double-legged stance], factor 3 [SCAT5/PCSI symptom scores], and factor 4 [K-D completion time]) emerged. A model fit with the top factors performed as well as each battery in predicting concussion status (AUC = 0.816 [95% CI = 0.731, 0.889]) compared with the SCAT5 (AUC = 0.784 [95% CI = 0.692, 0.866]), PCSI (AUC = 0.776 [95% CI = 0.674, 0.863]), VVE (AUC = 0.711 [95% CI = 0.602, 0.814]), and K-D (AUC = 0.708 [95% CI = 0.590, 0.819]). A multifaceted assessment for adolescents with concussion, comprising symptoms, attention, balance, and the visio-vestibular system, is critical. Current diagnostic batteries likely measure overlapping domains, and the sparse principal component analysis demonstrated strategies for streamlining comprehensive concussion assessment across a variety of settings.

Utilization of information from CNN feature maps for offline word-level writer identification

This paper proposes a text-independent method for authorship identification using handwritten word images. Our method is text-independent and imposes no limitations on the size of the input word images being analyzed. To begin with, the SIFT algorithm is utilized to extract key-point regions (fragments) across different levels of abstraction, encompassing allographs, characters, or character combinations. These fragments are subsequently processed through a CNN network, yielding feature maps corresponding to convolution layers. The information in these maps is then transformed into a fixed-dimension representation using a modified version of the HOG feature descriptor.The noteworthy contribution of our proposal lies in harnessing additional cues from CNN’s feature maps for writer identification. We propose a measure to gauge the importance or ‘saliency’ of feature maps within a CNN layer during training. This measure originates from applying Sparse Principal Component Analysis (SPCA) to Histogram Of Gradient (HOG) features. Once saliency values are obtained, we combine them with HOG representations to create writer descriptors tailored to a CNN layer. The derived descriptors are then passed through a set of SVM classifiers that are scored at two levels to determine the identity of the handwritten word image.The efficacy of our system has been demonstrated on the word-level data of the CVL, IAM and CERUG-EN datasets with an accuracy of 82.10%, 87.68%, and 75.80% respectively. The results obtained are shown to be promising when compared with previous works.

Sparse discriminant PCA based on contrastive learning and class-specificity distribution

Much mathematical effort has been devoted to developing Principal Component Analysis (PCA), which is the most popular feature extraction method. To suppress the negative effect of noise on PCA performance, there have been extensive studies and applications of a large number of robust PCAs achieving outstanding results. However, existing methods suffer from at least two shortcomings: (1) They expressed PCA as a reconstruction model measured by Euclidean distance, which only considers the relationship between the data and its reconstruction and ignores the differences between different data points; (2) They did not consider the class-specificity distribution information contained in the data itself, thus lacking discriminative properties. To overcome the above problems, we propose a Sparse Discriminant Principal Components Analysis (SDPCA) model based on contrastive learning and class-specificity distribution. Specifically, we use contrastive learning to measure the relationship between samples and their reconstructions, which fully takes the discriminative information between data into account in PCA. In order to make the extracted low-dimensional features profoundly reflect the class-specificity distribution of the data, we minimize the squared ℓ1,2-norm of the low-dimensional embedding. In addition, to reduce the effects of redundant features and noise and to improve the interpretability of PCA at the same time, we impose sparsity constraints on the projection matrix using the squared ℓ1,2-norm. Our experimental results on different types of benchmark databases demonstrate that our model has state-of-the-art performance.

Exactly Uncorrelated Sparse Principal Component Analysis

Sparse principal component analysis (PCA) aims to find principal components as linear combinations of a subset of the original input variables without sacrificing the fidelity of the classical PCA. Most existing sparse PCA methods produce correlated sparse principal components. We argue that many applications of PCA prefer uncorrelated principal components. However, handling sparsity and uncorrelatedness properties in a sparse PCA method is nontrivial. This article proposes an exactly uncorrelated sparse PCA method named EUSPCA, whose formulation is motivated by original views and motivations of PCA as advocated by Pearson and Hotelling. EUSPCA is a non-smooth constrained non-convex manifold optimization problem. We solve it by combining augmented Lagrangian and non-monotone proximal gradient methods. We observe that EUSPCA produces uncorrelated components and maintains a similar or better level of fidelity based on adjusted total variance through simulated and real data examples. In contrast, existing sparse PCA methods produce significantly correlated components. Supplemental materials for this article are available online.

Peptides From the Variable Domain of Immunoglobulin G as Biomarkers in Chronic Inflammatory Demyelinating Polyradiculoneuropathy.

Chronic inflammatory demyelinating polyradiculoneuropathy (CIDP) is a clinically heterogeneous immune-mediated disease. Diagnostic biomarkers for CIDP are currently lacking. Peptides derived from the variable domain of circulating immunoglobulin G (IgG) have earlier been shown to be shared among patients with the same immunologic disease. Because humoral immune factors are hypothesized to be involved in the pathogenesis of CIDP, we evaluated IgG variable domain-derived peptides as diagnostic biomarkers in CIDP (primary objective) and whether IgG-derived peptides could cluster objective clinical entities in CIDP (secondary objective). IgG-derived peptides were determined in prospectively collected sera of patients with CIDP and neurologic controls by means of mass spectrometry. Peptides of interest were selected through statistical analysis in a discovery cohort followed by sequence determination and confirmation. Diagnostic performance was evaluated for individual selected peptides and for a multipeptide model incorporating selected peptides, followed by performance reassessment in a validation cohort. Clustering of patients with CIDP based on IgG-derived peptides was evaluated through unsupervised sparse principal component analysis followed by k-means clustering. Sixteen peptides originating from the IgG variable domain were selected as candidate biomarkers in a discovery cohort of 44 patients with CIDP and 29 neurologic controls. For all 16 peptides, univariate logistic regressions and ROC curve analysis demonstrated increasing peptide abundances to associate with increased odds for CIDP (area under the curves [AUCs] ranging from 64.6% to 79.6%). When including age and sex in the logistic regression models, this remained the case for 13/16 peptides. A model composed of 5/16 selected peptides showed strong discriminating performance between patients with CIDP and controls (AUC 91.5%; 95% CI 84.6%-98.4%; p < 0.001). In the validation cohort containing 45 patients and 43 controls, 2/16 peptides demonstrated increasing abundances to associate with increased odds for CIDP, while the five-peptide model demonstrated an AUC of 61.2% (95% CI 49.3%-73.2%; p = 0.064). Peptide-based patient clusters did not associate with clinical features. IgG variable domain-derived peptides showed a valid source for diagnostic biomarkers in CIDP, albeit with challenges toward replication. Our proof-of-concept findings warrant further study of IgG-derived peptides as biomarkers in more homogeneous cohorts of patients with CIDP and controls. This study provides Class III evidence that the pattern of serum IgG-derived peptide clusters may help differentiate between patients with CIDP and those with other peripheral neuropathies.

Identity Recognition System Based on Multi-Spectral Palm Vein Image

A multi-spectral palm vein image acquisition device based on an open environment has been designed to achieve a highly secure and user-friendly biometric recognition system. Furthermore, we conducted a study on a supervised discriminative sparse principal component analysis algorithm that preserves the neighborhood structure for palm vein recognition. The algorithm incorporates label information, sparse constraints, and local information for effective supervised learning. By employing a robust neighborhood selection technique, it extracts discriminative and interpretable principal component features from non-uniformly distributed multi-spectral palm vein images. The algorithm addresses challenges posed by light scattering, as well as issues related to rotation, translation, scale variation, and illumination changes during non-contact image acquisition, which can increase intra-class distance. Experimental tests are conducted using databases from the CASIA, Tongji University, and Hong Kong Polytechnic University, as well as a self-built multi-spectral palm vein dataset. The results demonstrate that the algorithm achieves the lowest equal error rates of 0.50%, 0.19%, 0.16%, and 0.1%, respectively, using the optimal projection parameters. Compared to other typical methods, the algorithm exhibits distinct advantages and holds practical value.