Nonlinear Principal Component Analysis Method Research Articles

Nonlinear PCA for Spatio-Temporal Analysis of Earth Observation Data

Remote sensing observations, products and simulations are fundamental sources of information to monitor our planet and its climate variability. Uncovering the main modes of spatial and temporal variability in Earth data is essential to analyze and understand the underlying physical dynamics and processes driving the Earth System. Dimensionality reduction methods can work with spatio-temporal datasets and decompose the information efficiently. Principal Component Analysis (PCA), also known as Empirical Orthogonal Functions (EOF) in geophysics, has been traditionally used to analyze climatic data. However, when nonlinear feature relations are present, PCA/EOF fails. In this work, we propose a nonlinear PCA method to deal with spatio-temporal Earth System data. The proposed method, called Rotated Complex Kernel PCA (ROCK-PCA for short), works in reproducing kernel Hilbert spaces to account for nonlinear processes, operates in the complex kernel domain to account for both space and time features, and adds an extra rotation for improved flexibility. The result is an explicitly resolved spatio-temporal decomposition of the Earth data cube. The method is unsupervised and computationally very efficient.We illustrate its ability to uncover spatio-temporal patterns using synthetic experiments and real data. Results of the decomposition of three essential climate variables are shown: satellite-based global Gross Primary Productivity (GPP) and Soil Moisture (SM), and reanalysis Sea Surface Temperature (SST) data. The ROCK-PCA method allows identifying their annual and seasonal oscillations, as well as their non-seasonal trends and spatial variability patterns.

TECHNOLOGIES OF THE FOURTH INDUSTRIAL REVOLUTION: RESEARCH OF TECHNOLOGICAL ACCEPTANCE OF INDUSTRIAL COMPANIES WITH THE HELP OF NONLINEAR PRINCIPAL COMPONENT ANALYSIS METHOD (NLPCA)

Industrial companies’ readiness to accept the industry technologies 4.0 is studied and results of industrial companies’ questioning are analyzed in the work. The nonlinear principal component analysis method (NLPCA) is applied as a data processing instrument in understanding the Gifi system taking into account the variegated statistic nature of the questioning indexes.

A new reconstruction-based auto-associative neural network for fault diagnosis in nonlinear systems

Auto-associative neural network (AANN) is a typical nonlinear principal component analysis method, which is widely used in industry for fault diagnosis purposes, especially in nonlinear systems. However, the basic AANN often suffers from “smearing effects” problems that may lead to misdiagnosis, particularly with regards to the complex faults involving multiple variables. In this work, a new reconstruction-based AANN (RBAANN) method is proposed to enhance the capacity of fault diagnosis. In RBAANN, a generic derivative equation is developed to investigate the effects of AANN model inputs on the prediction error between model inputs and outputs. Based on the derivative equation, the reconstruction-based index for single or multiple variables, which is defined as the minimum prediction error, is obtained by tuning the corresponding model inputs iteratively. However, without the prior knowledge of the real faulty variables, all the possible variable sets need to be evaluated by the reconstruction-based index, and this may result in an exhaustive search and cause a huge computational burden. Thus, a branch and bound algorithm is introduced into RBAANN to solve the variable selection problem. Finally, an efficient fault diagnosis strategy by integrating RBAANN and branch and bound algorithm (BAB-RBAANN) is implemented to further pinpoint the source of the detected faults. This BAB-RBAANN method can handle both single and multiple variable(s) faults for nonlinear systems without prior knowledge efficiently. The effectiveness of the proposed methods is evaluated on a validation example and an industrial example. Comparisons with other methods, including principal component analysis techniques, are also presented.

Distinct Variation Pattern Discovery Using Alternating Nonlinear Principal Component Analysis.

Autoassociative neural networks (ANNs) have been proposed as a nonlinear extension of principal component analysis (PCA), which is commonly used to identify linear variation patterns in high-dimensional data. While principal component scores represent uncorrelated features, standard backpropagation methods for training ANNs provide no guarantee of producing distinct features, which is important for interpretability and for discovering the nature of the variation patterns in the data. Here, we present an alternating nonlinear PCA method, which encourages learning of distinct features in ANNs. A new measure motivated by the condition of orthogonal loadings in PCA is proposed for measuring the extent to which the nonlinear principal components represent distinct variation patterns. We demonstrate the effectiveness of our method using a simulated point cloud data set as well as a subset of the MNIST handwritten digits data. The results show that standard ANNs consistently mix the true variation sources in the low-dimensional representation learned by the model, whereas our alternating method produces solutions where the patterns are better separated in the low-dimensional space.

A nonlinear principal component analysis to study archeometric data

Statistical techniques, when applied to data obtained by chemical investigations on ancient artworks, are usually expected to recognize groups of objects to classify the archeological finds, to attribute the provenance of items compared with earlier investigated ones, or to determine whether an archaelogical attribution is possible or not. The statistical technique most frequently used in archeometry is the principal component analysis (PCA), because of its simplicity in theory and implementation. However, the application of PCA to archeometric data showed severe limitations because of its linear feature. Indeed, PCA is inadequate to classify data whose behavior describe a curve or a curved subspace of the original data space. As a consequence of it, an amount of information is lost because the multi‐dimensional data space is compressed into a lower‐dimensional subspace including principal components. The aim of this work is then to test a novel statistical technique for archeometry. We propose a nonlinear PCA method to extract maximum chemical information by plotting data on the smallest number of principal components and to answer archeological questions. The higher accuracy and effectiveness of nonlinear PCA approach with respect to standard PCA for the analysis of archeometric data are shown through the study of Apulian red figured pottery (fifth–fourth century BC) coming from some of the most relevant archeological sites of ancient Apulia (Monte Sannace (Gioia del Colle), Egnatia (Fasano), Canosa, Altamura, Conversano, and Arpi(Foggia)). Copyright © 2016 John Wiley & Sons, Ltd.

An evaluation of regionalization and watershed classification schemes for continuous daily streamflow prediction in ungauged watersheds

Regionalization – the process of transferring hydrological information from gauged to ungauged watersheds – has the potential to perform significantly better if these watersheds are classified in advance. In this study, we demonstrate the benefits of classification by a systematic combination of watershed classification techniques, regionalization methods, and rainfall-runoff models. Basins are first classified, then regionalization methods are applied, for continuous daily streamflow estimation at ungauged watersheds in Ontario, Canada. Nonlinear data-driven methods are used as regionalization and watershed classification schemes to transfer the parameters of two conceptual hydrologic models – namely McMaster University Hydrologiska Byråns Vattenbalansavdelning (MAC-HBV) and Sacramento Soil Moisture Accounting (SAC-SMA) – from gauged to ungauged watersheds. Our results suggest that a certain combination of watershed classification technique, regionalization method and hydrologic model can significantly improve the estimation of continuous streamflow at ungauged basins by improving the accuracy of estimated daily mean, low and peak flows. However, some combinations do not provide a clear improvement when compared to the scenario of unclassified basins. For example, the MAC-HBV model coupled with a counter propagation neural network as a regionalization technique provides a clear improvement in estimated daily mean, low and peak flow when the watersheds are first classified using a nonlinear principal component analysis method. Interestingly, a higher improvement is achieved for low flow as well, which is usually difficult to estimate in ungauged basins.

Image Feature Extraction using Non Linear Principle Component Analysis

In feature extraction technique for face recognition, to maximize the ratio of between-class scatter to that of within-class scatter and keeps high generalization performance, a nonlinear Evolutionary Weighted Principal Component Analysis (EWPCA) based on Genetic Algorithms is proposed in this paper. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two commonly used feature extraction techniques. However, PCA has many drawbacks. One is that its linearity can limit its relevance to the highly nonlinear systems frequently encountered in face recognition applications. To overcome this problem, nonlinear PCA method has been proposed. Genetic Algorithms (GA) are chosen as the searching method to select optimal weights for the WPCA. In face recognition, Evolutionary facial feature obtained by performing WPCA is used as the representation of original face images.Simulation is done using MATLAB software tool.

Complexity of free energy landscapes of peptides revealed by nonlinear principal component analysis

Employing the recently developed hierarchical nonlinear principal component analysis (NLPCA) method of Saegusa et al. (Neurocomputing 2004;61:57-70 and IEICE Trans Inf Syst 2005;E88-D:2242-2248), the complexities of the free energy landscapes of several peptides, including triglycine, hexaalanine, and the C-terminal beta-hairpin of protein G, were studied. First, the performance of this NLPCA method was compared with the standard linear principal component analysis (PCA). In particular, we compared two methods according to (1) the ability of the dimensionality reduction and (2) the efficient representation of peptide conformations in low-dimensional spaces spanned by the first few principal components. The study revealed that NLPCA reduces the dimensionality of the considered systems much better, than did PCA. For example, in order to get the similar error, which is due to representation of the original data of beta-hairpin in low dimensional space, one needs 4 and 21 principal components of NLPCA and PCA, respectively. Second, by representing the free energy landscapes of the considered systems as a function of the first two principal components obtained from PCA, we obtained the relatively well-structured free energy landscapes. In contrast, the free energy landscapes of NLPCA are much more complicated, exhibiting many states which are hidden in the PCA maps, especially in the unfolded regions. Furthermore, the study also showed that many states in the PCA maps are mixed up by several peptide conformations, while those of the NLPCA maps are more pure. This finding suggests that the NLPCA should be used to capture the essential features of the systems.

Kernel mutual subspace method and its application for object recognition

In this paper, the authors propose a new object recognition algorithm called the kernel mutual subspace method. The mutual subspace method proposed by Maeda is a superior technique for implementing robust object recognition by performing a principal component analysis on multiple input images. However, like with the ordinary subspace method, a shortcoming of this technique is that performance deteriorates when the category distribution has a nonlinear structure. To solve this problem, the authors theoretically derived a new object recognition algorithm called the kernel mutual subspace method by applying the kernel nonlinear principal component analysis, which is known as a powerful nonlinear principal component analysis method, to the mutual subspace method. When the proposed technique was applied to an individual identification problem based on facial images, it was apparent that the relationship between the degrees of freedom of the object motion and the subspace dimensionality indicating a high recognition rate could be consistently explained through experiments that used the proposed method, which did not differ significantly from the conventional method at the highest precision. They also showed that the proposed technique could be effective for large-scale recognition problems and that its recognition dictionary has a more compact structure. © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 88(6): 45–53, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.20190

Classification of Direct Load Control Curves for Performance Evaluation

In evaluating the performance of direct load control (DLC) programs, an essential task is to classify the DLC curves into either the one complying with the program or not. This paper presents an efficient approach to clustering the DLC curves through a structure of self-organizing maps (SOM). Aiming at selecting significant features of DLC curves, methods of nonlinear principal component analysis (NLPCA) and periodic analysis are proposed for feature extraction. The dual multilayer neural networks (DMNN) model is employed in the proposed NLPCA method. In the periodic analysis method, the periodic characteristics of the DLC curves are investigated. In the SOM, Davies-Bouldin (DB) indexes and a k-means algorithm decide the best number of clusters to be classified. Through the proposed methods, the DLC curves are thus divided into the two categories by the SOM: DLC complying and DLC noncomplying loads. Results obtained from the comparison of six different approaches show that the clusters obtained from the proposed approach exhibit lowest degrees of misclassification for the practical data on Taiwan Power Company (TPC) DLC programs.

Improvements to the non‐linear principal component analysis method, with applications to ENSO and QBO

Two improvements to the Non‐linear Principal Component Analysis (NLPCA) method are presented. In the normal application of this method, a non‐linear curve C is found that best fits the data. The method provides a projection function mapping from the data space to the curve C. However, this projection function is faulty in that points in the data space are generally not projected onto their closest neighbours on C. Here, a new projection function is introduced which ensures that the data points are projected onto their closest neighbours on C, resulting in an increase in the amount of variance explained by the NLPCA mode. This is illustrated by an analysis of the sea surface temperature anomaly data from the tropical Pacific, where the El Niño‐Southern Oscillation (ENSO) phenomenon is manifested. A second shortcoming of the NLPCA method is that the curve C comes with a parametrization which is arbitrary and has no physical interpretation. Here, the curve is re‐parametrized by arc length. This allows the computation of more meaningful time series, which we illustrate through an analysis of the Quasi‐Biennial Oscillation (QBO) in the equatorial stratospheric zonal wind data.

Neurofuzzy state identification using prefiltering

A new state estimator algorithm is based on a neurofuzzy network and the Kalman filter algorithm. The major contribution of the paper is recognition of a bias problem in the parameter estimation of the state-space model and the introduction of a simple, effective prefiltering method to achieve unbiased parameter estimates in the state-space model, which will then be applied for state estimation using the Kalman filtering algorithm. Fundamental to this method is a simple prefiltering procedure using a nonlinear principal component analysis method based on the neurofuzzy basis set. This prefiltering can be performed without prior system structure knowledge. Numerical examples demonstrate the effectiveness of the new approach.

Non-linear principal components analysis for process fault detection

Principal component analysis (PCA) has been applied widely for monitoring plant performance across a range of industrial processes. PCA is a linear technique and it is therefore not strictly applicable for handling industrial problems which exhibit significant non-linear behaviour. A novel non-linear PCA method is proposed based upon the Input-Training neural network. Multivariate statistical process control charts with non-parametric control limits are then defined to overcome the limitations of the conventional approach of defining the limits based upon the assumption of normality. A contribution plot capable of identifying the potential source of the fault in a non-linear situation is then proposed prior to applying the methodology to a continuous industrial reactor.

Nonlinear principal component analysis—Based on principal curves and neural networks

Many applications of principal component analysis (PCA) can be found in recently published papers. However principal component analysis is a linear method, and most engineering problems are nonlinear. Sometimes using the linear PCA method can be inadequate when nonlinearities are involved in the data. So there is a need for a nonlinear principal component analysis (NLPCA) method. The principal curve [Hastie and Stuetzle (1989) J. Am. Stat. Ass. 84 (406), 502–516] is a generalization of a linear principal component, but when applied to data sets the algorithm does not yield an NLPCA model in the sense of principal loadings. In this paper we present an NLPCA method which integrates the principal curve algorithm and neural networks. The results of both simulated and real problems show that the method is excellent for solving nonlinear principal component problems. Potential applications of NLPCA are discussed.