Linear Dimensionality Reduction Methods Research Articles

A novel method to recognize and classify based on an E-nose

The electronic nose (E-nose) has been widely reported as a new and convenient detection technology. While making continuous breakthroughs, the electronic nose is also facing a series of new problems and challenges. Especially in the actual application process, it faces high-dimensional and non-linear problems in the perception data of odor under non-ideal conditions. In this paper, a new method based on multiple optimal mapping is proposed, called superimposed mapping analysis (SMA). This method optimizes the loss function and mapping feature space of the model by making full use of the relationship between dependencies and mutual relationships within the class. The optimization method is used to construct the optimal loss function of the reconstruction matrix, which can achieve dimensionality reduction and feature extraction. The experimental results show that compared with the classical linear and non-linear dimensionality reduction methods, the performance index of SMA is more in line with practical application requirements.

Dimensionality reduction and unsupervised clustering for EELS-SI

A novel combination of machine learning algorithms is proposed for the differentiation of distinct spectra in a large electron energy loss spectroscopy spectrum image (EELS-SI) dataset. For clustering of the EEL spectra including similar fine structures in an efficient space, linear and nonlinear dimensionality reduction methods are used to project the EEL spectra onto a low-dimensional space. Then, a density-based clustering algorithm is applied to distinguish the meaningful data clusters. By applying this strategy to various experimental EELS-SI datasets, differentiation of several groups of EEL spectra representing specific fine structures was achieved. It is possible to investigate particular fine structures by averaging all of the spectra in each cluster. Also, the spatial distributions of each cluster in the scanning regions can be observed, which enables investigation of the locations of different fine structures in materials. This method does not require any prior knowledge, i.e., it is a data-driven analysis; therefore, it can be applied to any hyperspectral image.

Supervised discriminant Isomap with maximum margin graph regularization for dimensionality reduction

As one of the most popular nonlinear dimensionality reduction methods, Isomap has been widely used in pattern recognition and machine learning. However, Isomap has the following problems: (1) Isomap is an unsupervised dimensionality reduction method, it cannot use class label information to obtain discriminative low dimensional embedding for classification; (2) The embedding performance of Isomap is sensitive to neighborhood size parameter; (3) Isomap cannot deal with outside new data by direct embedding. In this paper, a novel dimensionality reduction method called supervised discriminant Isomap is proposed to solve the first two problems mentioned above. Specifically, first, raw data points are partitioned into different manifolds by using their class label information. Then, supervised discriminant Isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the Isomap criterion, and to enhance the discriminating capability by maximizing the distances between data points of different classes and the maximum margin graph regularization term. Finally, the corresponding optimization problems are solved by using eigen-decomposition algorithm. Further, we extend supervised discriminant Isomap to a linear dimensionality reduction method called supervised discriminant Isomap projection for handling the above three problems. Moreover, our approaches have three important characteristics: (1) Proposed methods adaptively estimate the local neighborhood surrounding each sample based on data density and similarity; (2) The objective functions of proposed methods can maximize margins between the each classes in the dimension-reduced feature space; (3) The objective functions of proposed methods have closed-form solutions. Furthermore, our methods can capture more discriminative information from raw data than other Isomap based methods. Extensive experiments on nine data sets demonstrate that the proposed methods are superior to the related state-of-the-art methods.

Dynamic time warping similarity measurement based on low-rank sparse representation

Similarity measurement of time series is one of the focus issues in time series analysis and mining. Morphology maintenance of time series is a better way for performing a similarity measurement, and phase space reconstruction has the advantage of analyzing the morphology of time series, whereas it is prone to generate high-dimensional data. Linear dimensionality reduction methods have difficulty preserving complete information of time series data. Manifold-based learning methods can better preserve the local characteristics of data. Low-rank representation (LRR) finds the lowest rank representation of all data and is capable of capturing the global structure of data. Therefore, in this paper, we propose a dynamic time warping similarity measurement method based on low-rank sparse representation (LRSE_DTW) to reduce the dimensionality of time series data. We learn the low-rank sparse representation of the phase space and then embed it into low-dimensional space to maintain the morphology of the phase space. DTW is used to measure the distance between discriminant information obtained from the l2,1-norm constraint on the projection matrix. To confirm the effectiveness of LRSE_DTW, time series classification experiments are carried out on public UCR time series classification archive. The results show that LRSE_DTW is superior to several other state-of-the-art time series similarity measurement methods.

Forward Stepwise Deep Autoencoder-Based Monotone Nonlinear Dimensionality Reduction Methods

Dimensionality reduction is an unsupervised learning task aimed at creating a low-dimensional summary and/or extracting the most salient features of a dataset. Principal component analysis is a linear dimensionality reduction method in the sense that each principal component is a linear combination of the input variables. To allow features that are nonlinear functions of the input variables, many nonlinear dimensionality reduction (NLDR) methods have been proposed. In this article, we propose novel NLDR methods based on bottleneck deep autoencoders. Our contributions are 2-fold: (1) We introduce a monotonicity constraint into bottleneck deep autoencoders for estimating a single nonlinear component and propose two methods for fitting the model. (2) We propose a new, forward stepwise deep learning architecture for estimating multiple nonlinear components. The former helps extract interpretable, monotone components when the assumption of monotonicity holds, and the latter helps evaluate reconstruction errors in the original data space for a range of components. We conduct numerical studies to compare different model fitting methods and use two real data examples from the studies of human immune responses to HIV to illustrate the proposed methods. Supplementary materials for this article are available online.

An Orthogonal Locality and Globality Dimensionality Reduction Method Based on Twin Eigen Decomposition

Dimensionality reduction is a hot research topic in pattern recognition. Traditional dimensionality reduction methods can be separated into linear dimensionality reduction methods and nonlinear dimensionality reduction methods. Linear dimensionality reduction methods usually utilize Euclidean distances to explore global geometric structures, and nonlinear dimensionality reduction methods can preserve local manifold structures in learned low-dimensional subspaces. However, redundant information and noises of the raw high-dimensional data restrict the classification performance of these methods. To solve the problem, we propose a novel orthogonal dimensionality reduction method based on twin eigen decomposition called orthogonal locality and globality preserving projections (OLGPP). Orthogonality, as a commonly used criterion in pattern recognition, is insensitive to the redundant information and the noises. OLGPP not only combines the advantages of global Euclidean structures and local manifold structures, but also is insensitive to the redundant information and noises of the raw high-dimensional data by embedding the orthogonality criterion. Additionally, the objective function based on twin eigen decomposition can be solved sequentially to obtain analytical solutions. On four well-known datasets, we design extensive experiments to evaluate the performance of OLGPP. From the experimental results, OLGPP has the optimal experimental results, and its average recognition rates are about 10% higher than the classic local dimensionality reduction method (i.e., locality preserving projection), which proves OLGPP is an effective dimensionality reduction method.

Evaluation of economics journals based on structural equation dimension reduction method

ABSTRACT Evaluation of economic journals is helpful to improve the quality of these journals. In this study, we adopt a new structural equation dimension reduction method to evaluate economics journals. We classify evaluation indexes based on cluster analysis and factor analysis to establish a structural equation model, normalize the regression coefficients to obtain the weight, and then weigh and summarize the first-level evaluation indexes for the ultimate dimension reduction and evaluation. The evaluation based on JCR 2019 Economics journals shows that the structural equation dimension reduction method overcomes the randomness of manual classification of evaluation indicators. The linear dimension reduction method is conducive to preserving a large amount of information of the original indicators; evaluating the first-level indicators facilitates the comprehensive evaluation of journals; the first-level indicators evaluation approach is more objective and reflects systematic academic evaluation. It should be noted that the stability of the structural equation has a significant impact on evaluation, which is generally suitable for a relatively mature academic evaluation.

An empirical analysis of graph‐based linear dimensionality reduction techniques

SummaryMany emerging applications such as social networks have prompted remarkable attention in graph data analysis. Graph data is typically high‐dimensional in nature, and dimensionality reduction is critical regarding storage, analysis, and querying of such data efficiently. Although there are many dimensionality reduction methods, it is not clear to what extent the performances of the various dimensionality reduction techniques differ. In this article, we review some of the well‐known linear dimensionality reduction methods and perform an empirical analysis of these approaches using large multidimensional graph datasets. Our results show that in linear unsupervised learning methods, the principal component analysis, singular value decomposition, and neighborhood preserving embedding methods achieve better retrieval data performance than other methods of the statistical information category, dictionary methods, and embedding methods, respectively. Regarding supervised learning methods, the experimental results demonstrate that linear discriminant analysis and partial least squares presented almost similar results.

Minimum Eigenvector Collaborative Representation Discriminant Projection for Feature Extraction

High-dimensional signals, such as image signals and audio signals, usually have a sparse or low-dimensional manifold structure, which can be projected into a low-dimensional subspace to improve the efficiency and effectiveness of data processing. In this paper, we propose a linear dimensionality reduction method—minimum eigenvector collaborative representation discriminant projection—to address high-dimensional feature extraction problems. On the one hand, unlike the existing collaborative representation method, we use the eigenvector corresponding to the smallest non-zero eigenvalue of the sample covariance matrix to reduce the error of collaborative representation. On the other hand, we maintain the collaborative representation relationship of samples in the projection subspace to enhance the discriminability of the extracted features. Also, the between-class scatter of the reconstructed samples is used to improve the robustness of the projection space. The experimental results on the COIL-20 image object database, ORL, and FERET face databases, as well as Isolet database demonstrate the effectiveness of the proposed method, especially in low dimensions and small training sample size.

Ivis Dimensionality Reduction Framework for Biomacromolecular Simulations.

Molecular dynamics (MD) simulations have been widely applied to study macromolecules including proteins. However, the high dimensionality of the data sets produced by simulations makes thorough analysis difficult and further hinders a deeper understanding of biomacromolecules. To gain more insights into the protein structure-function relations, appropriate dimensionality reduction methods are needed to project simulations onto low-dimensional spaces. Linear dimensionality reduction methods, such as principal component analysis (PCA) and time-structure-based independent component analysis (t-ICA), could not preserve sufficient structural information. Though better than linear methods, nonlinear methods, such as t-distributed stochastic neighbor embedding (t-SNE), still suffer from the limitations in avoiding system noise and keeping inter-cluster relations. ivis is a novel deep learning-based dimensionality reduction method originally developed for single-cell data sets. Here, we applied this framework for the study of light, oxygen, and voltage (LOV) domains of diatom Phaeodactylum tricornutum aureochrome 1a (PtAu1a). Compared with other methods, ivis is shown to be superior in constructing a Markov state model (MSM), preserving information of both local and global distances, and maintaining similarity between high and low dimensions with the least information loss. Moreover, the ivis framework is capable of providing new perspectives for deciphering residue-level protein allostery through the feature weights in the neural network. Overall, ivis is a promising member of the analysis toolbox for proteins.

Randomized independent component analysis and linear discriminant analysis dimensionality reduction methods for hyperspectral image classification

Dimensionality reduction is an important step in processing the hyperspectral images (HSI) to overcome the curse of dimensionality problem. Linear dimensionality reduction methods such as Independent component analysis (ICA) and Linear discriminant analysis (LDA) are commonly employed to reduce the dimensionality of HSI. These methods fail to capture non-linear dependency in the HSI data, as data lies in the nonlinear manifold. To handle this, nonlinear transformation techniques based on kernel methods were introduced for dimensionality reduction of HSI. However, the kernel methods involve cubic computational complexity while computing the kernel matrix, and thus its potential cannot be explored when the number of pixels (samples) are large. In literature a fewer number of pixels are randomly selected to partial to overcome this issue, however this sub-optimal strategy might neglect important information in the HSI. In this paper, we propose randomized solutions to the ICA and LDA dimensionality reduction methods using Random Fourier features, and we label them as RFFICA and RFFLDA. Our proposed method overcomes the scalability issue and to handle the non-linearities present in the data more efficiently. Experiments conducted with two real-world hyperspectral datasets demonstrates that our proposed randomized methods outperform the conventional kernel ICA and kernel LDA in terms overall, per-class accuracies and computational time.

Collaborative representation‐based locality preserving projections for image classification

Graph embedding has attracted much more research interests in dimensionality reduction. In this study, based on collaborative representation and graph embedding, the authors propose a new linear dimensionality reduction method called collaborative representation-based locality preserving projection (CRLPP). In the CRLPP, they assume that the similar samples should have similar reconstructions by collaborative representation and the similar reconstructions should also have the similar low-dimensional representations in the projected subspace. CRLPP first reconstructs each training sample using the collaborative representation of the other remaining training samples, and then designs the graph construction of all training samples, finally establishes the objective function of graph embedding using the collaborative reconstructions and the constructed graph. The proposed CRLPP can well preserve the intrinsic geometrical and discriminant structures of high-dimensional data in low-dimensional subspace. The effectiveness of the proposed is verified on several image datasets. The experimental results show that the proposed method outperforms the state-of-art dimensionality reduction.

Geochemical pattern recognitions of deep thermal groundwater in South Korea using self-organizing map: Identified pathways of geochemical reaction and mixing

The hydrogeochemistry of deep groundwater needs to be characterized for geological CO2 storage or radioactive waste disposal. However, various origins and their interactions and complex hydrogeological conditions make it difficult to assess. Moreover, it is challenging to interpret a large national dataset of hydrochemical variables due to wide composition ranges. This study aimed to define the representative geochemical facies of deep groundwater (average well depth of 624 ± 262 m) obtained from spa areas over South Korea by applying both linear (principal component analysis; PCA) and non-linear (self-organizing map; SOM) dimensionality reduction methods to a large dataset (n = 355) with 16 hydrochemical variables. The SOM results combined with hierarchical cluster analysis showed that deep thermal groundwater in South Korea is classified into five major geochemical groups (G1 to G5) with four mixing groups (M1 to M4). G1 to G5 represent high-TDS saline (7% of the samples), acidic CO2-rich (4%), high-pH alkaline (14%), sulfate-rich (9%), and dilute freshwater (11%), respectively. More than half of the samples (56%) belonged to the four mixing groups. In particular, the SOM reduced the number of samples (n = 355) to 104 neurons, visualizing the cluster structure of samples and the relationship among hydrochemical variables in a 2D array of neurons, which made it possible to distinguish the facies (G4 and G5 and M1 to M4) that could not be defined by PCA due to the extremely distinct geochemistry of G1 and G2. Based on the compositional changes of neurons between the geochemical (G1 to G5) and mixing (M1 to M4) groups, major reaction pathways were identified for each geochemical group. The hydrochemistry of each group mainly evolves through distinct water–rock interactions but is modified by varying degrees of mixing with dilute shallow groundwater during ascent. This study provides a state-of-the-art method to interpret a large and complex hydrogeochemical dataset; the SOM is expected to be a useful alternative to PCA. Besides, the results of this study can be useful to select a site suitable for geological CO2 storage or radioactive waste disposal.

Comparison of incremental linear dimension reduction methods for streaming data

Traditional linear dimension reduction methods such as Principal component analysis (PCA) and linear discriminant analysis (LDA) have been used in many application areas due to simplicity and high performance. However, in data streams where data instances are generated continuously over time, it is difficult to apply traditional PCA or LDA. Moreover, data streams can have drifting concepts over time. In this paper, we compared several incremental linear dimension reduction algorithms which can be applied for classification in streaming data. Also, the performance comparison for prediction accuracy and time complexity was conducted in various streaming environments such as low dimensional data streams, high dimensional data streams, and data streams with concept drifts. Experimental results showed that incremental least squares formulation (ILS) combined with incremental PCA can be used effectively for classification in streaming data.

Exponential sparsity preserving projection with applications to image recognition

Sparsity preserving projection (SPP), as a widely used linear unsupervised dimensionality reduction (DR) method, is designed to preserve the sparse reconstructive relationship of the raw data. SPP constructs an affinity weight matrix by solving a sparse representation model which does not need any parameters. Moreover, the obtained projection may contain some discriminating information even if no prior knowledge is provided. Although SPP may be more conveniently used in practice due to these advantages, it still suffers from the so-called small-sample-size problem as may other DR methods do. To solve this problem, we propose an exponential sparsity preserving projection (ESPP) by using matrix exponential, and present two efficiently numerical methods for solving the corresponding large-scale matrix exponential eigenvalue problem. ESPP avoids the singularity of the coefficient matrices, and obtains more valuable information for the SPP. Image recognition experiments are conducted on several real-world image databases and the experimental results illustrate the outperformances of ESPP.

Locality preserving projection based on Euler representation

Locality preserving projection (LPP) is a widely used linear dimensionality reduction method, which preserves the locality structure of the original data. Motivated by the fact that kernel technique can capture nonlinear similarity of features and help to improve separability between nearby data points, this paper proposes locality preserving projection model based on Euler representation (named as ELPP). This model first projects the data into a complex space with Euler representation, then learns the dimensionality reduction projection with preserving locality structure in this complex space. We also extend ELPP to F-ELPP by replacing the squared F-norm with F-norm, which will weaken the exaggerated errors and be more robustness to outliers. The optimization algorithms of the two models are given, and the convergence of F-ELPP is proved. A large number of experiments on several public databases have demonstrated that the two proposed models have good robustness and feature extraction ability.

A new variant of radial visualization for supervised visualization of high dimensional data

Radial Visualization technique is a non linear dimensionality reduction method. Radial Visualization projects multivariate data in the 2-dimensional visual space inside the unit circle. Radial Visualization supports display both the samples and the attributes that provides useful information of data structures. In this article, we introduced a new variant of Radial Visualization for visualizing high dimensional data set that named Arc Radial Visualization. The new proposal that modified Radial Visualization supported more space to display high dimensional datasets. Our method provides an improvement in visualizing cluster structures of high dimensional data sets on the Radial Visualization. We present our proposal method with two quality measurements and proved the effectiveness of our approach for several real datasets.

A new variant of radial visualization for supervised visualization of high dimensional data

Radial Visualization technique is a non linear dimensionality reduction method. Radial Visualization projects multivariate data in the 2-dimensional visual space inside the unit circle. Radial Visualization supports display both the samples and the attributes that provides useful information of data structures. In this article, we introduced a new variant of Radial Visualization for visualizing high dimensional data set that named Arc Radial Visualization. The new proposal that modified Radial Visualization supported more space to display high dimensional datasets. Our method provides an improvement in visualizing cluster structures of high dimensional data sets on the Radial Visualization. We present our proposal method with two quality measurements and proved the effectiveness of our approach for several real datasets.

Effect of thermal oxidation on detection of adulteration at low concentrations in extra virgin olive oil: Study based on laser-induced fluorescence spectroscopy combined with KPCA–LDA

The fluorescence spectra of oil samples were obtained by laser-induced fluorescence spectroscopy and thermal oxidation stoichiometry at room temperature and 80 °C respectively. The Support Vector Machine, combined with Principal Component Analysis-Linear Discriminant Analysis (PCA-LDA), could distinguish pure extra virgin olive oils (EVOO) from oils adulterated with 2% soybean oil, with a recognition rate of 100%. Besides, as the intensity of the fluorescence spectra and concentration of the adulterants showed a non-linear relationship, linear dimension reduction methods may lead to overlapping of the different adulterated concentrations features, resulting in large errors in quantifying adulteration. In this paper, Kernel Principal Component Analysis-Linear Discriminant Analysis (KPCA-LDA) was applied instead of PCA-LDA to extract fluorescence spectra features, and a Partial Least Squares Regression model was established, which could quantify adulterants such as low percentages of soybean oil in EVOO. The coefficient of determination and root mean squared error were 0.92 and 2.72%, respectively.

Local tangent space alignment and relevance vector machine as nonlinear methods for estimating sensory quality of tea using NIR spectroscopy

The quality of green tea is conventionally tested by human sensory panel, who assign quality scores to different teas. In order to overcome the deficiency of sensory evaluation methods, near infrared (NIR) reflectance spectroscopy combined with nonlinear methods was used to evaluate the sensory quality of green tea in this study. Nonlinear dimension reduction methods such as Kernel Principal Component Analysis (KPCA), Sparse Principal Component Analysis (SPCA), and Local Tangent Space Alignment (LTSA) were used to reduce the dimension of the spectral data, and compared with linear dimension reduction methods such as Principal Component Analysis (PCA), Multidimensional Scaling (MDS). Relevance vector machine (RVM) as a nonlinear quantitative analysis method was performed comparatively to develop calibration models between the sensory scores of the sensory panels and objective determination (NIR spectra measurements), and compared with back-propagation artificial network and linear PLS model. Experiment results showed that the non-linear reduction algorithm LTSA could obtain better effect on spatial reduction dimensional for NIR spectral data of tea. And when the number K of the neighborhoods in LTSA was 15, the RMSECV of RVM model was the minimum. The performance of LTSA-RVM model was superior to the others when 8 intrinsic variables were used. In the prediction set, Rp, RMSEP and RPD were 0.963, 1.461 and 3.788, respectively. The absolute value of prediction error of all prediction samples in LTSA-RVM are below 3.0, which indicated that the sensory scores predicted by NIR were consistent with the scores by tea assessors. The results of recovery study were 98.42%–102.81% at 5 real tea samples and the relative error lower than 3%. The research demonstrates that NIR technology combined with nonlinear analysis methods can effectively estimate sensory quality of tea and also present a potential way for fast and effective determination of the sensory quality of green tea.