Variant Of Principal Component Analysis Research Articles

Principal Component Analysis variants for Parkinson datasets

Principal Component Analysis (PCA) is one of the most fundamental dimension reduction methods that need further research. With the widespread popularity of machine learning and the arrival of the era of big data, dimension reduction has become a hot topic and principal component analysis is a hot topic. However, although there are a lot of researchers who focus on the methods of the PCA, few researches on Parkinson Datasets have been made. As a result, the aim of our work is to discuss the PCA variants for Parkinson Datasets. This paper first introduces the three most commonly used PCA methods: PCA, Sparse PCA and Kernel PCA, and then introduces the Support Vector Machine (SVM) used to measure the dimension reduction effect. After that, we introduced the Parkinson's dataset and the meanings of root mean square error (RMSE), overall accuracy, Cohens kappa (Kappa) and computational time, the indicators that are used to measure the dimensionality reduction effect. Finally, we identified the variants among different PCA methods on the Parkinson dataset by comparing the indicators of the data obtained after dimensionality reduction using different methods.

High-abundance peaks and peak clusters associate with pharmaceutical polymers and excipients in urinary untargeted clinical metabolomics data: exploration of their origin and possible impact on label-free quantification.

Pharmaceutical polymers and excipients represent interesting but often overlooked chemical classes in clinical exposure and bioanalytical research. These chemicals may cause hypersensitivity reactions, they can be useful to confirm exposure to pharmaceuticals, and they may pose bioanalytical challenges, including ion suppression in liquid chromatography-mass spectrometry (LC-MS-)based workflows. In this work, we assessed these chemicals in light of a rather surprising finding presented in two previously published studies, namely that usage of cyclosporine A, an immunosuppressive drug which is known to be cleared through excretion in the bile, explained the largest amount of variance in principal component analysis of urinary LC-SWATH/MS small-molecule profiling data. Specifically, we examined the freely-accessible 24-hour urine metabolomics data of 570 kidney transplant recipients included in the TransplantLines Biobank and Cohort Study (NCT03272841). These data unveiled thousands of high-abundance polymer peaks in some samples, which were associated with the use of the macrogol (i.e., polyethylene glycol) 3350 oral laxative agent. In addition, we found multiple clusters of high-abundance peaks which were linked to the exposure to two pharmaceutical excipients, namely short-chain polyethylene glycol (molecular weight <1000 Da) and polyethoxylated castor oil (also known as Kolliphor® EL or Cremophor® EL). Respectively, these excipients are used in temazepam capsules and cyclosporine A capsules, and the latter provides a plausible explanation for the rather surprising finding that instigated our work. Moreover, such explanation and our findings in general put emphasis on taking into consideration these and other pharmaceutical polymers and excipients when exploring, processing, and interpreting clinical small-molecule profiling data.

Correlations between different protein species of oral rinse MMP-8 and patient-related factors.

The aim of this study is to examine correlations between different oral rinse matrix metalloproteinase (MMP)-8 protein species in western blot (WB) analysis, quantitative MMP-8 measurements, and patient-related factors. Elevated activated MMP-8 (aMMP-8) associate with periodontitis and a diagnostic point-of-care technology has been developed based on aMMP-8. In WB, different MMP-8 protein species can be analyzed. Relative abundancyof fragmented 20-25 kDa forms in WB has been associated with and reflects MMP-8 activation and related fragmentation and elevated quantitative aMMP-8 measurements. A random sample of 192 participants from a periodontal disease screening study was used for this study. Oral rinse samples for biomarker analyses were collected before clinical periodontal examinations. aMMP-8 immunofluorometric (IFMA) and WB analysis (utilizing the same monoclonal antibody, 8708), polymorphonuclear leukocyte (PMN) elastase activity test and tissue inhibitor of metalloproteinases (TIMP)-1 ELISA levels were performed from the oral rinse samples. Distinct MMP-8 protein species were differentiated in the WB analysis. Principal component (PC) analysis was conducted to explore correlation patterns between the different species. Adjusted correlation analysis between the extracted PCs of WB and aMMP-8 IFMA levels and multilevel regression analysis were conducted to explore if the other periodontal disease-related biomarkers and clinical surrogate measures and patient-related factors are co-variating with the extracted components. Distinct correlation patterns between the MMP-8 protein species were observed. The first four PCs explained 89% of the whole variance in PC analysis. Statistically significant correlation (p < 0.05) were observed as follows: PC1 positively with 21 kDa (r = .69) and 25 kDa fragments (r = .55) and negatively with 150 kDa complexes (r = -.46). PC2 correlated with 45 (r = .70) and 55 kDa (r = .65) activated forms, PC3 with 70-80 kDa latent proforms (r = .63) and 90-100 kDa complexes (r = .67), and PC4 with 35 kDa fragments (r = .81). There were significant correlations between quantitative (IFMA) aMMP-8 measurements and PC1 (p < 0.001), PC2 (<0.05) and PC3 (<0.05) but not with PC4. In multilevel regression models age, PMN elastase activity, TIMP-1 levels, and a number of 4-5 mm periodontal pockets were associated with PC1, nonsmoking with PC2, ageand PMN elastase activity with PC3, and age and smoking with PC4. Relative abundancy of fragmented 21-25 kDa protein species was correlated with the quantitative aMMP-8 (IFMA) measurements, which is in line with previous results. Different patient-related factors (smoking, age, proteolytic activity) may modify the formation of different MMP-8 protein species in oral rinse samples and may cause variability in quantitative aMMP-8 measurement.

Supervised categorized principal component analysis for imagined speech classification via applying singular value decomposition on a symmetry matrix

The dimension of the feature vectors for performing the imagined speech is large. Hence, it is required to reduce their dimension as well as perform the classification simultaneously. This is a categorized principal component analysis (CatPCA) problem. However, the conventional CatPCA methods are unsupervised learning methods and the design of the unitary matrix in the conventional principal component analysis (PCA) algorithms is only based on the PCA approach. Therefore, the classification performance is limited. This paper proposes a supervised CatPCA approach for performing the imagined speech classification. Also, the objective function of an optimization problem for designing the unitary matrix is formulated based on both the PCA algorithm and the k means algorithm. To find the analytical solution of the optimization problem, a property on computing the singular value decomposition (SVD) of a symmetric matrix is employed. Then, the design of the cluster centers in the k means algorithm is formulated as a quadratic programming. Finally, the above two procedures are iterated until the algorithm converges. Since the analytical solutions of these two optimization problems can be derived analytically, the required computational power for finding the solution of the optimization problem is low. To demonstrate the effectiveness of our proposed CatPCA algorithm, the Track 3 of the International brain computer interface (BCI) Competition Database is employed as the dataset for performing the performance evaluation. The computer numerical simulation results show that our proposed CatPCA algorithm yields the higher classification accuracy compared to PCA variants and the dimension reduction related algorithms. Also, the required execution time of our proposed method is lower than those of the states of the arts methods. This demonstrates the effectiveness and the efficiency of our proposed CatPCA algorithm.

GC/MS and DART-MS as complementary methods for investigating the effects of weathering on chemical profiles of ignitable liquids: A case study for paint thinner

In this study, gas chromatography/mass spectrometry (GC/MS) and direct analysis in real time mass spectrometry (DART-MS) were used to analyze paint thinner samples weathered under different conditions. As GC/MS total ion chromatograms (TIC) of weathered paint thinner samples varied significantly due to the disparate evaporation rates of organic compounds in the mixture, DART-MS spectra have shown more consistent profiles among different samples. The distinctive ion clusters identified in the DART-MS spectra were attributed to the less volatile glycol ethers and polymeric compounds with alkyl chains in the paint thinner samples. Analysis of variance-principal component analysis (ANOVA-PCA) was applied to study the contributions of weathering degree and temperature factors to the total variance of GC/MS TIC and DART-MS spectral datasets and found that both factors played significant roles in the weathering process (p < 0.05), though weathering degree contributed to considerably more total variance to GC/MS TIC profiles (i.e., 93.4%). The paint thinner GC/MS and DART-MS profiles were compared with those from other common ignitable liquids, such as gasoline, torch fuel, lighter fluid, and Japan drier. The partial least squares discriminant analysis (PLS-DA) models were constructed to classify the ignitable liquids with the classification rates of 99.97 ± 0.02% and 99.80 ± 0.08% for models based on GC/MS and DART-MS data, respectively. Our study has shown that DART-MS can provide discriminative information for distinguishing ignitable liquid samples and complement the existing GC/MS method by the detection of less volatile compounds.

A novel regression method: Partial least distance square regression methodology

Partial Least Squares (PLS) is a multivariate linear statistical analysis method integrating principal component analysis (PCA), canonical correlation analysis (CCA), and multiple linear regression (MLR) analysis. It can effectively solve the problems of multiple correlation between variables and small sample size. However, PLS also has some imperfections, on the one hand, PLS extracts the latent variables based on the maximization of Pearson correlation coefficient of independent and dependent variables. Here, Pearson correlation coefficient usually cannot measure the nonlinear relationship between variables, so the latent variables cannot ensure the strongest interpretation. On the other hand, PLS applies MLR to the extracted latent variables, which cannot truly reflect the nonlinear relationship of data, so the regression function is usually under-fitting. The above two imperfect designs are the main reasons for the low regression accuracy and prediction performance of PLS for nonlinear data. According to the above problems, this paper proposes a Partial Least Distance Square (PLDS) regression method. PLDS reflects the original data information through distance variance in PCA, employs distance correlation coefficient to measure the correlation between independent and dependent variables in CCA, and performs quasilinear regression and model solution method to obtain the final regression equation in MLR. In order to verify the effectiveness of the proposed method, the UCI datasets and Chinese medicine datasets with dose-effect relationship was used to compare the new method with other classical regression methods. Finally, the experimental results show that PLSD has better prediction performance regardless of whether there is significant nonlinear relationship between variables. In addition, PLDS has high regression accuracy and low computation complexity for high-dimensional and small-sample data.

The effectiveness of big data classification control based on principal component analysis

Large-scale datasets are becoming more common, yet they can be challenging to understand and interpret. When dealing with big datasets, principal component analysis (PCA) is used to minimize the dimensionality of the data while maintaining interpretability and avoiding information loss. It accomplishes this by producing new uncorrelated variables that gradually reduce the variance of the system. In the field of data analysis, PCA is a multivariate statistical technique commonly used to obtain rules explaining the separation of groups in a given situation. Classes are predicted using a classification algorithm, a supervised learning technique that indicates which type of data points will be presented. Creating a classification model using classification algorithms is required before any successful classification can be achieved. It is possible to predict the future using a variety of categorized strategies. It is necessary to reduce the dimensionality of data sets using the PCA approach. This article will begin by introducing the basic ideas of PCA and discussing what it can and cannot do. It will then describe some variants of PCA and their application and then shows how PCA improves the performance using a series of experiments.

Batch correction methods for nontarget chemical analysis data: application to a municipal wastewater collection system

Nontarget chemical analysis using high-resolution mass spectrometry has increasingly been used to discern spatial patterns and temporal trends in anthropogenic chemical abundance in natural and engineered systems. A critical experimental design consideration in such applications, especially those monitoring complex matrices over long time periods, is a choice between analyzing samples in multiple batches as they are collected, or in one batch after all samples have been processed. While datasets acquired in multiple analytical batches can include the effects of instrumental variability over time, datasets acquired in a single batch risk compound degradation during sample storage. To assess the influence of batch effects on the analysis and interpretation of nontarget data, this study examined a set of 56 samples collected from a municipal wastewater system over 7 months. Each month’s samples included 6 from sites within the collection system, one combined influent, and one treated effluent sample. Samples were analyzed using liquid chromatography high-resolution mass spectrometry in positive electrospray ionization mode in multiple batches as the samples were collected and in a single batch at the conclusion of the study. Data were aligned and normalized using internal standard scaling and ComBat, an empirical Bayes method developed for estimating and removing batch effects in microarrays. As judged by multiple lines of evidence, including comparing principal variance component analysis between single and multi-batch datasets and through patterns in principal components and hierarchical clustering analyses, ComBat appeared to significantly reduce the influence of batch effects. For this reason, we recommend the use of more, small batches with an appropriate batch correction step rather than acquisition in one large batch.Graphical abstract

Analysis of Factors Involving Drinking Water Contamination by Glyphosate and/or Nitrate in Urban Areas

This study investigated factors that could be related to drinking water contamination in urban areas in order to obtain quality profiles that characterized presence of the glyphosate and nitrate contaminants. Thus, in a period of one year, 4,853 tap water samples from 89 cities in the northeastern region of the State of São Paulo, Brazil, were analyzed in 21 physicochemical and 2 microbiological parameters. Additionally, 4 demographic variables were also included in multivariate data analysis. Principal Component Analysis of physicochemical and microbiological data showed that glyphosate concentration is positively correlated with nitrate concentration, especially in cities that make exclusive use of groundwater, besides correlating with conductivity and with concentrations of calcium, magnesium, fluoride, chloride, phosphate and free residual chlorine. The inclusion of demographic variables in Principal Component Analysis did not significantly change waters physicochemical profiles, but in cities that exclusive use groundwater for public supply the number of hospitalizations for diarrhea correlated positively with glyphosate, nitrate and chloride concentrations, in addition to conductivity. Linear Discriminant Analysis models involving 5 variables (conductivity and concentrations of calcium, magnesium, chloride and nitrate) were able to predict the cities vulnerability to groundwater contamination by nitrate.

Supported Diagnosis of Attention Deficit and Hyperactivity Disorder from EEG Based on Interpretable Kernels for Hidden Markov Models.

As a neurodevelopmental pathology, Attention Deficit Hyperactivity Disorder (ADHD) mainly arises during childhood. Persistent patterns of generalized inattention, impulsivity, or hyperactivity characterize ADHD that may persist into adulthood. The conventional diagnosis relies on clinical observational processes yielding high rates of overdiagnosis due to varying interpretations among specialists or missing information. Although several studies have designed objective behavioral features to overcome such an issue, they lack significance. Despite electroencephalography (EEG) analyses extracting alternative biomarkers using signal processing techniques, the nonlinearity and nonstationarity of EEG signals restrain performance and generalization of hand-crafted features. This work proposes a methodology to support ADHD diagnosis by characterizing EEG signals from hidden Markov models (HMM), classifying subjects based on similarity measures for probability functions, and spatially interpreting the results using graphic embeddings of stochastic dynamic models. The methodology learns a single HMM for EEG signal from each patient, so favoring the inter-subject variability. Then, the Probability Product Kernel, specifically developed for assessing the similarity between HMMs, fed a support vector machine that classifies subjects according to their stochastic dynamics. Lastly, the kernel variant of Principal Component Analysis provided a means to visualize the EEG transitions in a two-dimensional space, evidencing dynamic differences between ADHD and Healthy Control children. From the electrophysiological perspective, we recorded EEG under the Stop Signal Task modified with reward levels, which considers cognitive features of interest as insufficient motivational circuits recruitment. The methodology compares the supported diagnosis in two EEG channel setups (whole channel set and channels of interest in frontocentral area) and four frequency bands (Theta, Alpha, Beta rhythms, and a wideband). Results evidence an accuracy rate of 97.0% in the Beta band and in the channels where previous works found error-related negativity events. Such accuracy rate strongly supports the dual pathway hypothesis and motivational deficit concerning the pathophysiology of ADHD. It also demonstrates the utility of joining inhibitory and motivational paradigms with dynamic EEG analysis into a noninvasive and affordable diagnostic tool for ADHD patients.

Chemometric Optimization of a Demineralization Method for Analysing Sediment Organic Matter by Fourier Transform Infrared Spectroscopy

Fourier transform infrared (FTIR) spectroscopy is a widespread and useful technique for characterizing the chemical composition of soil/sediment organic matter (SOM). However, the mineral components of the samples limit spectral analysis since they mask SOM’s distinctive signals. Big efforts have been made in this matter to eliminate the spectral interferences of mineral constituents of the samples, by demineralization with hydrofluoric acid (DHF), or spectral subtraction. This work aimed to upgrade reported DHF methods and optimize a sample preparation methodology for SOM characterization by FTIR spectroscopy combining the potential of spectral and multivariate analysis. Semi-quantitative FTIR ratios were introduced as input variables for principal component analysis to determine the optimal DFH conditions to characterize SOM. One-way analysis of variance (ANOVA, α = 0.05) test revealed that the optimal demineralization was achieved with two cycles of HF 10% v/v at 60 °C, against five HF cycles and room temperature, previously reported. The DHF at the optimal conditions not only improved the demineralization efficiency against that of the reported methods but also did not cause significant undesirable modifications in SOM that could distort the FTIR analysis. These conclusions have statistical significance and are based on the qualitative information of the spectra and also on a comprehensive semi-quantitative study of them.

Quality of Education and Late-Life Cognitive Function in a Population-Based Sample From Puerto Rico.

Background and ObjectivesWe examined quality of education, literacy, and years of education in relation to late-life cognitive function and decline in older Puerto Ricans.Research Design and MethodsOur sample consisted of 3,385 community-dwelling adults aged 60 years and older from the Puerto Rican Elderly: Health Conditions study. Quality of education was based on principal component analysis of variables gathered from Department of Education and Census reports. Literacy (yes/no) and years of education were self-reported. Cognitive function was assessed in participants’ homes at baseline and 4 years later using a previously validated Spanish-language 20-point global screening measure for dementia, the minimental Cabán. Regression models were adjusted for sociodemographic and life course covariates.ResultsQuality of education was positively correlated with both educational attainment and cognitive performance. Independent of years of education, literacy, childhood economic hardship, and adult economic hardship, compared to participants in the lowest quartile of education quality, those in the highest quartile had significantly better baseline cognitive performance (β = 0.09, p < .001). Quality of education did not consistently show an association with change in cognitive function over 4 years. Literacy and greater educational attainment were each independently associated with better cognitive function at baseline and less cognitive decline.Discussion and ImplicationsQuality of education, literacy, and years of education, while interrelated, also show independent associations with cognitive functioning in older Puerto Ricans. The downstream factors of literacy and years of education were more closely related to age-related cognitive decline than quality of education.

Pomological and Phenological Identification of Some Walnut (Juglans regia L.) Genotypes and Cultivars

The Persian walnut (Juglans regia L.) is one of the most valuable crops in Iran. The present study was conducted in The rabor agriculture research station (RARS) in Kerman province to investigate 21 important pomological and phenological variables among 37 walnut native genotypes and commercial cultivars in the year 2020. In general, results showed that walnuts from RARS were phenotypically diverse with the significant variation found across most traits examined. The highest and lowest coefficient of variation (CV) was for kernel percentage (CV= 29.48%), and time of male flowering (CV= 14.77%), respectively. Results also showed a strong positive correlation between nut weight and kernel weight (r= 0.727). The date of nut maturity had the lowest correlation with kernel intensity of the ground color (r= -0.476). Principal component analysis of variables revealed that the first eight principal components (PCs) accounted for 79.43% of the total variation. Cluster analysis based on variables produced a dendrogram with five main clusters. The grouping of genotypes was not following their geographical origin. This study revealed the presence of high phenotypic diversity in the walnut genotypes of Rabor Agricultural Research Station (RARS), supporting their breeding value for possible use in the future.

Robust Principal Component Analysis Based On Hypergraph Regularization for Sample Clustering and Co-Characteristic Gene Selection.

Extracting genes involved in cancer lesions from gene expression data is critical for cancer research and drug development. The method of feature selection has attracted much attention in the field of bioinformatics. Principal Component Analysis (PCA) is a widely used method for learning low-dimensional representation. Some variants of PCA have been proposed to improve the robustness and sparsity of the algorithm. However, the existing methods ignore the high-order relationships between data. In this paper, a new model named Robust Principal Component Analysis via Hypergraph Regularization (HRPCA) is proposed. In detail, HRPCA utilizes L2,1-norm to reduce the effect of outliers and make data sufficiently row-sparse. And the hypergraph regularization is introduced to consider the complex relationship among data. Important information hidden in the data are mined, and this method ensures the accuracy of the resulting data relationship information. Extensive experiments on multi-view biological data demonstrate that the feasible and effective of the proposed approach.

A comparison of quasar emission reconstruction techniques for z ≥ 5.0 Lyman α and Lyman β transmission

ABSTRACT Reconstruction techniques for intrinsic quasar continua are crucial for the precision study of Lyman α (Ly α) and Lyman β (Ly β) transmission at z &amp;gt; 5.5, where the λ &amp;lt; 1215 Å emission of quasars is nearly completely absorbed. While the number and quality of spectroscopic observations have become theoretically sufficient to quantify Ly α transmission at 5.0 &amp;lt; z &amp;lt; 6.0 to better than $1{{\ \rm per\ cent}}$, the biases and uncertainties arising from predicting the unabsorbed continuum are not known to the same level. In this paper, we systematically evaluate eight reconstruction techniques on a unified testing sample of 2.7 &amp;lt; z &amp;lt; 3.5 quasars drawn from the Extended Baryon Oscillation Spectroscopic Survey. The methods include power-law extrapolation, stacking of neighbours, and six variants of principal component analysis (PCA) using direct projection, fitting of components, or neural networks to perform weight mapping. We find that power-law reconstructions and the PCA with fewest components and smallest training sample display the largest biases in the Ly α forest (${-}9.58{{\ \rm per\ cent}}/{+}8.22{{\ \rm per\ cent}}$, respectively). Power-law extrapolations have larger scatters than previously assumed of ${+}13.1{{\ \rm per\ cent}}/{-}13.2{{\ \rm per\ cent}}$ over Ly α and ${+}19.9{{\ \rm per\ cent}}/{-}20.1{{\ \rm per\ cent}}$ over Ly β. We present two new PCAs that achieve the best current accuracies of $9{{\ \rm per\ cent}}$ for Ly α and $17{{\ \rm per\ cent}}$ for Ly β. We apply the eight techniques after accounting for wavelength-dependent biases and scatter to a sample of 19 quasars at z &amp;gt; 5.7 with IR X-Shooter spectroscopy, obtaining well-characterized measurements for the mean flux transmission at 4.7 &amp;lt; z &amp;lt; 6.3. Our results demonstrate the importance of testing and, when relevant, training, continuum reconstruction techniques in a systematic way.

The first application of compositional data analysis (CoDA) in a multivariate perspective for detection of pollution source in sea sediments: The Pozzuoli Bay (Italy) case study

In the last decades, investigating geochemistry of sea sediments has been challenging in the eastern sector of Pozzuoli Bay, source of the metal(loid)s has been a matter of debate and the proposed origin of potentially toxic elements (PTEs) has been occasionally inconsistent. In this study, compositional data analysis (CoDA) was used because the results are independent of the measurement unit, the selected subgroup of elements and the order of chemicals in the dataset. The robust variant of principal component analysis (PCA) indicated that Hg, Cd, Cu, Pb and Zn were positively correlated with mud and organic matter in the sediments deposited in front of the former industrial site. Concentrations of these metals decrease along the cores and in the distal zone. Nevertheless, Al, As, V, Fe, Cr, Ni and sand form an association along the coast which strengthens with increasing distance from fumaroles in the proximal zone. It suggests that arsenic was mainly originated from the pyroclastic deposits of Campi Flegrei and some of the seepages with hydrothermal component, supported by low contribution of the variables in robust PCA of the sediments from distal zone. Therefore, this pioneering article suggests CoDA as a powerful tool for answering the long-lasting questions over sediment geochemistry in polluted areas.

Iterated and exponentially weighted moving principal component analysis

The principal component analysis (PCA) is a staple statistical and unsupervised machine learning technique in finance. The application of PCA in a financial setting is associated with several difficulties, such as numerical instability and nonstationarity. We attempt to resolve them by proposing two new variants of PCA: an iterated principal component analysis (IPCA) and an exponentially weighted moving principal component analysis (EWMPCA). Both variants rely on the Ogita-Aishima iteration as a crucial step.

Some Multilinear Variants of Principal Component Analysis: Examples in Grayscale Image Recognition and Reconstruction

Principal component analysis (PCA) has long been used in computer vision applications such as face recognition. Here, we present an overview of some variants of PCA, including 2D PCA (2DPCA), bidirectional 2DPCA (B2DPCA), and coupled subspace analysis (CSA). Unlike conventional PCA, the variants 2DPCA, B2DPCA, and CSA preserve the original image structure, often providing better recognition and reconstruction results than those obtained with PCA. This article considers the background for these techniques and steps involved in applying these methods, including typical preprocessing of sample images, algorithm description, and classification. These variants of PCA have been successfully used in a number of different areas such as identification of wood species, biometrics (not limited to face recognition), medical imaging, and image compression, to name a few examples; we briefly mention some of these to provide an idea of the scope of applications. We address some advantages and disadvantages of these variants in relation to PCA. Utilizing the Modified National Institute of Standards and Technology (MNIST) digits and Fashion-MNIST image sets, we demonstrate application of CSA for image recognition and reconstruction compared to PCA. Finally, we mention how these PCA variants fit into a more general framework using tensors.

Review of Dimension Reduction Methods

Purpose: This study sought to review the characteristics, strengths, weaknesses variants, applications areas and data types applied on the various Dimension Reduction techniques. Methodology: The most commonly used databases employed to search for the papers were ScienceDirect, Scopus, Google Scholar, IEEE Xplore and Mendeley. An integrative review was used for the study where 341 papers were reviewed. Results: The linear techniques considered were Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Singular Value Decomposition (SVD), Latent Semantic Analysis (LSA), Locality Preserving Projections (LPP), Independent Component Analysis (ICA) and Project Pursuit (PP). The non-linear techniques which were developed to work with applications that have complex non-linear structures considered were Kernel Principal Component Analysis (KPCA), Multi-dimensional Scaling (MDS), Isomap, Locally Linear Embedding (LLE), Self-Organizing Map (SOM), Latent Vector Quantization (LVQ), t-Stochastic neighbor embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP). DR techniques can further be categorized into supervised, unsupervised and more recently semi-supervised learning methods. The supervised versions are the LDA and LVQ. All the other techniques are unsupervised. Supervised variants of PCA, LPP, KPCA and MDS have been developed. Supervised and semi-supervised variants of PP and t-SNE have also been developed and a semi supervised version of the LDA has been developed. Conclusion: The various application areas, strengths, weaknesses and variants of the DR techniques were explored. The different data types that have been applied on the various DR techniques were also explored.

Robust PCA via Regularized Reaper with a Matrix-Free Proximal Algorithm

Principal component analysis (PCA) is known to be sensitive to outliers, so that various robust PCA variants were proposed in the literature. A recent model, called reaper, aims to find the principal components by solving a convex optimization problem. Usually the number of principal components must be determined in advance and the minimization is performed over symmetric positive semi-definite matrices having the size of the data, although the number of principal components is substantially smaller. This prohibits its use if the dimension of the data is large which is often the case in image processing. In this paper, we propose a regularized version of reaper which enforces the sparsity of the number of principal components by penalizing the nuclear norm of the corresponding orthogonal projector. If only an upper bound on the number of principal components is available, our approach can be combined with the L-curve method to reconstruct the appropriate subspace. Our second contribution is a matrix-free algorithm to find a minimizer of the regularized reaper which is also suited for high-dimensional data. The algorithm couples a primal-dual minimization approach with a thick-restarted Lanczos process. This appears to be the first efficient convex variational method for robust PCA that can handle high-dimensional data. As a side result, we discuss the topic of the bias in robust PCA. Numerical examples demonstrate the performance of our algorithm.