Local Principal Component Analysis Research Articles

An extension of neural gas to local PCA

We suggest an extension of the neural gas vector quantization method to local principal component analysis. The distance measure for the competition between local units combines a normalized Mahalanobis distance in the principal subspace and the squared reconstruction error, with the weighting of both measures depending on the residual variance in the minor subspace. A recursive least-squares method performs the local principal component analysis. The method is tested on synthetic two- and three-dimensional data and on the recognition of handwritten digits.

Quality evaluation of fingerprints of herbal medicine with chromatographic data

In this study, local least squares (LLS) and principal component analysis (PCA) were applied to deal with the disturbances in a data set of chromatographic fingerprints after necessary data transformations. It has been demonstrated that PCA with standard normal variate (SNV) transformation of data led to meaningful classification of 33 different Erigeron breviscapus herbal samples. The result was also corroborated by variance squares discriminant method. The quality of herbal objects was further evaluated, and the causes of this fact have been explained from a chemical point of view. At the same time, it implied an idea for qualitative evaluation of the herbal objects with a common class pattern of chromatographic fingerprints.

Almost autonomous training of mixtures of principal component analyzers

In recent years, a number of mixtures of local PCA models have been proposed. Most of these models require the user to set the number of submodels (local models) in the mixture and the dimensionality of the submodels (i.e., number of PC's) as well. To make the model free of these parameters, we propose a greedy expectation-maximization algorithm to find a suboptimal number of submodels. For a given retained variance ratio, the proposed algorithm estimates for each submodel the dimensionality that retains this given variability ratio. We test the proposed method on two different classification problems: handwritten digit recognition and 2-class ionosphere data classification. The results show that the proposed method has a good performance.

Integrated methodology for forensic oil spill identification.

A new integrated methodology for forensic oil spill identification is presented. It consists of GC-MS analysis, chromatographic data processing, variable-outlier detection, multivariate data analysis, estimation of uncertainties, and statistical evaluation. The methodology was tested on four groups of diagnostic ratios composed of petroleum biomarkers and ratios within homologous PAH categories. Principal component analysis (PCA) was employed and enabled the simultaneous analysis of many diagnostic ratios. Weathering was taken into account by considering the sampling uncertainties estimated from replicate spill samples. Statistical evaluation ensured an objective matching of oil spill samples with suspected source oils as well as classification into positive match, probable match, and nonmatch. The data analysis is further refined if two or more source oils are classified as probable match by using weighted least squares fitting of the principal components, local PCA models, and additional information relevant to the spill case. The methodology correctly identified the source of two spill samples (i.e., crude oils from Oseberg East and Oseberg Field Centre) and distinguished them from closely related source oils.

Linear Fuzzy Clustering Techniques With Missing Values and Their Application to Local Principal Component Analysis

In this paper, we propose two methods for partitioning an incomplete data set with missing values into several linear fuzzy clusters by extracting local principal components. One is an extension of fuzzy c-varieties clustering that can be regarded as the algorithm for the local principal component analysis of fuzzy covariance matrices. The other is a simultaneous application of fuzzy clustering and principal component analysis of fuzzy correlation matrices. Both methods estimate prototypes ignoring only missing values and they need no preprocessing of data such as the elimination of samples with missing values or the imputation of missing elements. Numerical examples show that the methods provide useful tools for interpretation of the local structures of a database.

A View-Invariant Face Detection Method Based on Local PCA Cells

This paper presents a view-invariant face detection method based on local PCA cells. In order to extract the general features of faces at each view and position, Gabor filters and local PCA (Principal Component Analysis) are used. Local PCA cells specialized for each view and position are made by applying the Gaussian to the outputs of local PCA of Gabor features. By applying the Gaussian, only local PCA cells having a view similar to an input give large values. This decreases an adverse influence of the local PCA cells of other views. As a result, only one classifier can treat multi-view faces well by integrating the outputs of local PCA cells. The effectiveness of the proposed method is confirmed by comparison with Support Vector Machine using Gaussian kernel and Radial Basis Function network. It is also confirmed that generalization ability is improved by selecting local PCA cells using the reconstruction error of local PCA.

Improvement of reliability in banknote classification using reject option and local PCA

The improvement of reliability in banknote neuro-classifier is investigated and a reject option is proposed based on the probability density function of the input data. The classification reliability is evaluated through two reliability parameters, which are associated with the winning class probability and the second maximal probability. Then a threshold value is set up to reject the unreliable classifications. As for modeling the non-linear correlation among the data variables and extracting the features, a local principal components analysis (PCA) is applied. The method is tested with a learning vector quantization (LVQ) classifier using 1440 data samples of various US dollar bills. The results show that by taking a suitable reject threshold value and also a proper number of regions for the local PCA, the reliability of system can be improved significantly.

Combining spatial and colour information for content based image retrieval

Colour is one of the most important features in content based image retrieval. However, colour is rarely used as a feature that codes local spatial information, except for colour texture. This paper presents an approach to represent spatial colour distributions using local principal component analysis (PCA). The representation is based on image windows which are selected by two complementary data driven attentive mechanisms: a symmetry based saliency map and an edge and corner detector. The eigenvectors obtained from local PCA of the selected windows form colour patterns that capture both low and high spatial frequencies, so they are well suited for shape as well as texture representation. Projections of the windows selected from the image database to the local PCs serve as a compact representation for the search database. Queries are formulated by specifying windows within query images. System feedback makes both the search process and the results comprehensible for the user.

Topological local principal component analysis

In help of the Kohonen's self-organizing maps we present a topological local principal component analysis model which is capable of exploiting both the global topological structure and each local cluster structure. A newly proposed self-organizing strategy that can enhance the learning speed is introduced to train the model.

Strip line detection and thinning by RPCL-based local PCA

We solve the tasks of strip line detection and thinning in image processing and pattern recognition with the help of a statistical learning technique called rival penalized competitive learning based local principal component analysis. Due to its model selection and noise resistance ability, the technique is experimentally shown to outperform conventional Hough transform and thinning algorithms.

Improved system for object detection and star/galaxy classification via local subspace analysis

The two traditional tasks of object detection and star/galaxy classification in astronomy can be automated by neural networks because the nature of the problems is that of pattern recognition. A typical existing system can be further improved by using one of the local Principal Component Analysis (PCA) models. Our analysis in the context of object detection and star/galaxy classification reveals that local PCA is not only superior to global PCA in feature extraction, but is also superior to gaussian mixture in clustering analysis. Unlike global PCA which performs PCA for the whole data set, local PCA applies PCA individually to each cluster of data. As a result, local PCA often outperforms global PCA for data of multi-modes. Moreover, since local PCA can effectively avoid the trouble of having to specify a large number of free elements of each covariance matrix of gaussian mixture, it can give a better description of local subspace structures of each cluster when applied on high dimensional data with small sample size. In this paper, the local PCA model proposed by Xu [IEEE Trans. Neural Networks 12 (2001) 822] under the general framework of Bayesian Ying Yang (BYY) normalization learning will be adopted. Endowed with the automatic model selection ability of BYY learning, the BYY normalization learning-based local PCA model can cope with those object detection and star/galaxy classification tasks with unknown model complexity. A detailed algorithm for implementation of the local PCA model will be proposed, and experimental results using both synthetic and real astronomical data will be demonstrated.

Robust Linear Clustering with Least Squares Criterion and Its Application to Collaborative Filtering

Non-linear extensions of Principal Component Analysis (PCA) have been developed for detecting the lower-dimensional representations of real world data sets. Fuzzy c-Varieties (FCV) is the linear fuzzy clustering algorithm that can be regarded as a Local PCA technique. However least squares techniques often fail to account for “outliers”. This paper proposes a technique for making the FCV algorithm robust to intra-sample outliers. The objective function based on the lower rank approximation of the data matrix is minimized by a robust M-estimation algorithm that is similar to FCM-type iterative procedures. The new method is also useful for estimating missing values and a numerical experiment of Collaborative Filtering reveals an improvement in recommendation performance.

Nonlinear Noise Estimation in International Capital Markets

We analyzed six stock exchange markets through the nonlinear dynamics concept. We used daily data from the Toronto Stock Exchange, NYSE, London Stock Exchange, Hong Kong Stock Market, Tokyo Stock Exchange, and the Singapore Stock Exchange. The period studied is from January 1, 1988 to June 30, 1999. We performed Local Principal Components Analysis in order to estimate the dimension of each underlying attractor. Our main interest is the noise level estimation of each time series. The results indicate weak determinism and strong noise influence. The noise-to-signal ratio for almost all time series is above 50%. Noise is leptokurtic in the eastern stock markets, and mesokurtic in western ones.

Integrated Soil Fertility Management on Local Coconut Hybrid in a Farmer- Managed Coconut-Based Cropping System

An integrated soil fertility management or ISFM on-farm trial was conducted at Barangay Tawan-tawan, Baguio District, Davao City in 1993 to 1999 to know the effects of organic and inorganic fertilizers on the vegetative growth, nutrient status and yield performance of intercropped Catigan dwarf x Laguna tall (CATD x LAGT) coconut hybrid better known as PCA 15–1 or CATLAG.
 
 Vegetative growth of CATD x LAGT palms revealed that ammonium sulfate + NaCl and goat manure + NaCl produced significantly bigger girth size on the first year and more number of leaves and living fronds on the second and fourth year, respectively. These treatments also enhanced early flowering (>50% @ 4.5 years) and yield (51-53 nuts/tree/yr @) 5 years from field planting) in hybrid palms. The organic fertilizers, i.e. goat manure and coconut husk, become more effective in their effect on the vegetative and reproductive characters of palms when combined with a Cl source (NaCl). The application of Cl nutrient contributed to the better performance of palms during the drought condition. The application of goat manure resulted in improved levels of soil nutrients as K, Ca, Mg and P, increased percent base saturation and lower soil acidity.
 
 The use of local hybrid, PCA 15-1 and ISFM-based low cost production inputs such as ammonium sulfate, goat manure/coco waste (husk) and common table salt as well as intercropping suitable crops under young coconuts is a profitable package of technologies capable of improving and sustaining coconut farm productivity in the Philippines.

Peak purity assessment in liquid chromatography–mass spectrometry

Fixed-size moving window evolving factor analysis and base peak chromatograms have been used for peak purity detection in data generated with LC–MS. The two methods were evaluated with both real and simulated data and were found to be fast and complementary to each other. When a possibly impure peak is detected, it is suggested that further information can be obtained from local principal component analysis modelling and comparative mass chromatogram plots.

Interactions in chromatographic separation of resins from deasphaltened crude oils studied by means of infrared spectroscopy and principal component analysis

Resins were extracted from crude oils by adsorption onto various particles followed by desorption using various solvent combinations in a two-step desorption procedure. The extracted resins were analyzed using infrared spectroscopy, followed by local principal component analysis on the polar and aromatic absorption bands. It was found that the adsorption and desorption processes can be related to interactions involving specific functional groups such as amides, esters and aromatics in the resins with the adsorbents and solvents. The results are discussed in terms of polar forces, hydrogen bonds, dispersion forces and acid-based characteristics. Polar functional groups in the resins were found to interact with polar groups at the adsorbents and with solvents by polar forces and hydrogen bonds. It is also found that the aromatic character of the extracted resins is greater when benzene is used as solvent than if polar solvents are used. When using a low-polarity adsorbent the extracted resins contain less polar functional groups than when using a high-polarity solvent, showing that dispersion forces are important in the extraction. Some intercrude differences were also found and discussed.

A nonlinear neural network model of mixture of local principal component analysis: application to handwritten digits recognition

Principal component analysis (PCA) is a popular tool in multivariate statistics and pattern recognition. Recently, some mixture models of local principal component analysis have attracted attention due to a number of benefits over global PCA. In this paper, we propose a mixture model by concurrently performing global data partition and local linear PCA. The partition is optimal or near optimal, which is realized by a soft competition algorithm called ‘neural gas’. The local PCA type representation is approximated by a neural learning algorithm in a nonlinear autoencoder network, which is set up on the generalization of the least-squares reconstruction problem leading to the standard PCA. Such a local PCA type representation has a number of numerical advantages, for example, faster convergence and insensitive to local minima. Based on this mixture model, we describe a modular classification scheme to solve the problem of handwritten digits recognition. We use 10 networks (modules) to capture different features in the 10 classes of handwritten digits, with each network being a mixture model of local PCA type representations. When a test digit is presented to all the modules, each module provides a reconstructed pattern by a prescribed principle and the system outputs the class label by comparing the reconstruction errors from the 10 networks. Compared with some traditional neural network-based classifiers, our scheme converges faster and recognizes with higher accuracy. For a relatively small size of each module, the classification accuracy reaches 98.6% on the training set and 97.8% on the testing set.

Rough sets for enhancements of local subspace classifier

Developed recently local subspace classifier (LSC) based on local principal component analysis (LPCA) though has high performance (99.4% on NIST database of handwritten digits) lacks a mechanism of recovery from errors. Having confidence levels associated with the decision function in the classifier could help in creation of such procedure. In this paper it is shown how rough set idea can be used to incorporate confidence mechanism. In this way we make the algorithm more adequate to practical use as human actions are now possible at low confidence decisions. Besides main goal of the paper the necessary algorithmic tools for LPCA and their theoretical base are presented, too.

GMM based on local PCA for speaker identification

An efficient Gaussian mixture modelling (GMM) method based on local principal component analysis (PCA) with vector quantisation (VQ) for speaker identification is proposed. The proposed method firstly partitions the data space into several disjoint regions by VQ, and then performs PCA in each region. Finally, the GMM for the speaker is obtained from the transformed feature vectors in each region. Compared to the conventional GMM method with diagonal covariance matrix, under the same performance, the proposed method requires less storage and shows faster results.

Local PCA algorithms

Within the last years various principal component analysis (PCA) algorithms have been proposed. In this paper we use a general framework to describe those PCA algorithms which are based on Hebbian learning. For an important subset of these algorithms, the local algorithms, we fully describe their equilibria, where all lateral connections are set to zero and their local stability. We show how the parameters in the PCA algorithms have to be chosen in order to get an algorithm which converges to a stable equilibrium which provides principal component extraction.