Linear Dimensionality Reduction Methods Research Articles

Detection of maize seed germination rate based on improved locally linear embedding

Locally linear embedding (LLE) is a nonlinear dimensionality reduction method, which has great advantages over linear dimensionality reduction methods. However, the traditional LLE takes the Euclidean distance as the distance measure, which is difficult to accurately reflect the spatial position relationship between the high-dimensional data of the near-infrared spectrum, resulting in a poor modeling effect. This paper tries to improve the LLE with different distance metric methods and proposes a rapid detection method for maize seed germination rate based on improved local linear embedding and near-infrared spectroscopy. In this paper, a total of 315 samples from 7 different types of maize seeds, purchased from the seed market, were adopted as the research object. We performed artificial aging tests with 8 different gradients (from 0d to 7d with interval of 1d) on them and completed the germination rate test after collecting the near-infrared spectral data for each sample. The Monte Carlo cross-validation (MCCV) algorithm, combined with PLSR and SVM, was used to remove abnormal samples from the spectral and germination rate data. And then, for comparison, we used several different improvement strategies for LLE (traditional Euclidean distance, Manhattan distance, Chebyshev distance, Correlation coefficient, and Cosine similarity) to reduce the spectral data dimension and established PLS and SVM germination rate prediction models. We compared the prediction effects of different models to explore the optimal improvement strategy of LLE dimension reduction distance measurement. The results showed that the cosine similarity was the best improvement strategy under the same modeling method. The R2 of the LLE_ cos-PLS model's test set can reach 0.8384, and the R2 of the LLE_cos-SVM model's test set can reach 0.8765. The results showed that the cosine similarity could better reflect the spatial distribution in the spectral data of aged maize seeds, and the precision of the model was higher after LLE_cos dimensionality reduction. Compared with the linear modeling method PLS, the nonlinear modeling method SVM is more suitable for predicting the germination rate of maize seeds. This study can provide a reference method for the quality inspection of other agricultural products.

Projection expectile regression for sufficient dimension reduction

Many existing sufficient dimension reduction methods are designed for regression with predictors that are elliptically distributed, which limits their application in real data analyses. Projection expectile regression (PER) is proposed as a new linear sufficient dimension reduction method for handling complex predictor structures, which includes continuous, discrete, and mixed predictor variables. PER requires the link function between the response and the predictor to be monotone, but not necessarily smooth, which makes it suitable for handling stratified response surfaces. By design, PER does not involve matrix inversion or high-dimensional smoothing. Thus, PER is ideal for controlling problems associated with multicollinearity, high dimensionality, and sparsity in the predictor. An extensive simulation study demonstrates the performance of projection expectile regression in synthetic data. A real data analysis of health insurance charges in the United States is also provided. The asymptotic properties of the PER estimator are included as well.

Dimensionality reduction through convolutional autoencoders for fracture patterns prediction

Dimensionality reduction methods construct a low dimensional or reduced space. Complex structural mechanics problems can be approximated on a low dimensional space with a Reduced-Order Model or surrogate model. For instance, Principal Components Analysis can compute a reduced basis from a database of full-order snapshots. Principal Components Analysis, however, does not reconstruct low dimensional spaces efficiently for highly nonlinear problems, mainly because it is a linear dimensionality reduction method. In this paper, an original approach based on an autoencoder neural network is developed to construct a nonlinear Reduced-Order Model for a highly nonlinear brittle fracture problem. We show on a set of simulations how the autoencoder can be efficient for dimensionality reduction or compression of highly nonlinear data. A complete deep learning framework is proposed to predict crack propagation patterns directly from the loading conditions. Finally, we validate our nonlinear Reduced-Order Model with data sets generated for two problems using proportional and non-proportional loading conditions. In the first problem, the crack always initiates at the same location but propagates in various directions, which may even vary during the simulation if loading conditions are non-proportional. In the second problem, the crack may additionally initiate at various locations. Results for the two problems evaluate the capabilities of the proposed approach.

An ISOMAP Analysis of Sea Surface Temperature for the Classification and Detection of El Niño & La Niña Events

Isometric feature mapping (ISOMAP) is a nonlinear dimensionality reduction method used for extracting features from spatiotemporal data. The traditional principal component analysis (PCA), a linear dimensionality reduction method, measures the distance between two data points based on the Euclidean distance (line segment), which cannot reflect the actual distance between the data points in a nonlinear space. By contrast, the ISOMAP measures the distance between two data points based on the geodesic distance, which more closely reflects the actual distance by the view of tracing along the local linearity in the original nonlinear structure. Thus, ISOMAP-reconstructed data points can reflect the features of real structures and can be classified more accurately than traditional PCA-reconstructed data points. Moreover, these ISOMAP-reconstructed data points can be used for cluster analysis by emphasizing the differences among the points more than those by the traditional PCA. In this study, sea surface temperature (SST) data points reconstructed using the traditional PCA and ISOMAP were compared. The classification based on these reconstructed SST points was tested using the Niño 3.4 index, which labels El Niño, La Niña, or normal events. The mean differences from the ISOMAP data points were larger than those from the traditional PCA data points. The ISOMAP not only helped differentiate the points in two different events but also provided better difference measurement of the points belonging to the same class (e.g., 82/83 and 97/98 El Niño events). On examining the evolution of the leading three temporal eigen components of the SST PCA, or especially the SST ISOMAP, we found that the trajectories were similar to the Lorenz 63 model on a phase space figure. This implies that NWP perturbations can be traced using the ISOMAP to measure growing unstable behaviors. Spatial eigenmodes (empirical orthogonal function) between the traditional PCA and ISOMAP were also determined and compared herein.

An Evaluation of Supervised Dimensionality Reduction For Large Scale Data

Experimenters today frequently quantify millions or even billions of characteristics (measurements) each sample to address critical biological issues, in the hopes that machine learning tools would be able to make correct data-driven judgments. An efficient analysis requires a low-dimensional representation that preserves the differentiating features in data whose size and complexity are orders of magnitude apart (e.g., if a certain ailment is present in the person's body). While there are several systems that can handle millions of variables and yet have strong empirical and conceptual guarantees, there are few that can be clearly understood. This research presents an evaluation of supervised dimensionality reduction for large scale data. We provide a methodology for expanding Principal Component Analysis (PCA) by including category moment estimations in low-dimensional projections. Linear Optimum Low-Rank (LOLR) projection, the cheapest variant, includes the class-conditional means. We show that LOLR projections and its extensions enhance representations of data for future classifications while retaining computing flexibility and reliability using both experimental and simulated data benchmark. When it comes to accuracy, LOLR prediction outperforms other modular linear dimension reduction methods that require much longer computation times on conventional computers. LOLR uses more than 150 million attributes in brain image processing datasets, and many genome sequencing datasets have more than half a million attributes.

Improved Multi-Echo Gradient-Echo-Based Myelin Water Fraction Mapping Using Dimensionality Reduction.

Multi-echo gradient-echo (mGRE)-based myelin water fraction (MWF) mapping is a promising myelin water imaging (MWI) modality but is vulnerable to noise and artifact corruption. The linear dimensionality reduction (LDR) method has recently shown improvements with regard to these challenges. However, the magnitude value based low rank operators have been shown to misestimate the MWF for regions with [Formula: see text] anisotropy. This paper presents a nonlinear dimensionality reduction (NLDR) method to estimate the MWF map better by encouraging nonlinear low dimensionality of mGRE signal sources. Specifically, we implemented a fully connected deep autoencoder to extract the low-dimensional features of complex-valued signals and incorporated a sparse regularization to separate the anomaly sources that do not reside in the low-dimensional manifold. Simulations and in vivo experiments were performed to evaluate the accuracy of the MWF map under various situations. The proposed NLDR-based MWF improves the accuracy of the MWF map over the conventional nonlinear least-squares method and the LDR-based MWF and maintains robustness against noise and artifact corruption.

Divergent Projection Analysis for Unsupervised Dimensionality Reduction

Principal component analysis (PCA) has been extended to a series of classical methods in dimensionality reduction for unsu¬pervised learning. However, the differences among the projected data samples may not be sufficient by PCA, causing problems both in data visualization and recognition. Though some nonlinear dimensionality reduction methods have been proposed to preserve the samples’ differences, their nonlinear mappings hide the relationships among the data features, e.g., Isometric mapping (Isomap) and Locally Linear Embedding (LLE). In this paper, we propose a linear dimensionality reduction method, called Divergent Projection Analysis (DPA), to preserve the largest differences among the samples. Our DPA projects the sam¬ples into a low-dimensional space such that any two projected samples have a certain distance, resulting in solving a nonconvex optimization problem. The nonconvex problem is solved by the subgradient descent algorithm and particle swarm optimiza¬tion algorithm, and their solution qualities are compared in the experiments. Experiments on a synthetic dataset and three face recognition problems confirm the effectiveness of the proposed method compared with other state-of-the-art methods.

Forecasting Near-equivalence of Linear Dimension Reduction Methods in Large Panels of Macro-variables

In an extensive pseudo out-of-sample horserace, classical estimators (dynamic factor models, RIDGE and partial least squares regression) and the novel to forecasting, Regularized Sliced Inverse Regression, exhibit almost near-equivalent forecasting accuracy in a large panel of macroeconomic variables across targets, horizons and subsamples. This finding motivates the theoretical contributions in this paper. Most widely used linear dimension reduction methods are shown to solve closely related maximization problems with solutions that can be decomposed in signal and scaling components. They are organized under a common scheme that sheds light on their commonalities and differences as well as on their functionality. Regularized Sliced Inverse Regression delivers the most parsimonious forecast model and obtains the greatest reduction of the complexity of the forecasting problem. Nevertheless, the study’s findings are that (a) the intrinsic relationship between forecast target and the other macroseries in the panel is linear and (b) targeting contributes in reducing the complexity of modeling yet does not induce significant gains in macroeconomic forecasting accuracy.

On principal graphical models with application to gene network

The principal graphical model is introduced by incorporating the ideas from the linear sufficient dimension reduction (SDR) methods such as the sliced inverse regression and sliced average variance estimation to a nonparametric graphical model. A nonparametric graphical model is a widely used method to investigate undirected graphs. However, when the number of nodes is large, a nonparametric graphical model suffers from the ‘curse of dimensionality’ because they contain intrinsic high dimensional kernels. The parametric graphical models such as the Gaussian or copula Gaussian graphical models are also well known for their intuitive structure and interpretability. However, they hinge on strong parametric model assumptions. The principal graphical model applies well-known linear SDR techniques to the nonparametric graphical models to enhance performance in high dimensional networks, avoid model assumptions, and maintain interpretability. We use components of linear SDR as modules and implement them in the (p2) pairs of variables in the network to evaluate conditional independence. In the numerical experiment, our methods have competitive accuracy in both low and high-dimensional settings. Our methods are applied to the DREAM 4 challenge gene network dataset and they work well in high dimensional settings with a limited number of observations.

Asymptotic and bootstrap tests for subspace dimension

Many linear dimension reduction methods proposed in the literature can be formulated using an appropriate pair of scatter matrices. The eigen-decomposition of one scatter matrix with respect to another is then often used to determine the dimension of the signal subspace and to separate signal and noise parts of the data. Three popular dimension reduction methods, namely principal component analysis (PCA), fourth order blind identification (FOBI) and sliced inverse regression (SIR) are considered in detail and the first two moments of subsets of the eigenvalues are used to test for the dimension of the signal space. The limiting null distributions of the test statistics are discussed and novel bootstrap strategies are suggested for the small sample cases. In all three cases, consistent test-based estimates of the signal subspace dimension are introduced as well. The asymptotic and bootstrap tests are illustrated in real data examples.

Verbal and Non-Verbal Color-Emotional Subjective Spaces Are Four-Dimensional, Spherical and Isomorphic

The brain at wakefulness is active even in the absence of goal-directed behavior or salient stimuli. However, patterns of this resting-state (RS) activity can undergo long-term alterations following exposure to preceding meaningful stimuli. This study was aimed to develop an unbiased method to detect such changes in the RS activity after exposure to emotionally meaningful stimuli. For this purpose, we used functional magnetic resonance imaging (fMRI) of RS brain activity before and after acquisition and extinction of experimental conditioned fear. A group of healthy volunteers participated in three fMRI sessions: a RS before fear conditioning, a fear extinction session, and a RS immediately after fear extinction. The fear-conditioning paradigm consisted of three neutral visual stimuli paired with a partial reinforcement by a mild electric current. We used both linear and non-linear dimensionality reduction approaches to distinguish between the initial RS and the RS after stimuli exposure. The principal component analysis (PCA) as a linear dimensionality reduction method showed significantly worse results than non-linear methods (Isomap, LLE, Laplacian eigenmaps). Using the Laplacian eigenmaps manifold learning method, we were able to show significant differences between the two RSs at the level of individual participants. This detection was further improved by smoothing the BOLD signal with the wavelet multiresolution analysis. The developed method can improve the discrimination of functional states collected in longitudinal fMRI studies.

Comparative study of different dimensionality reduction methods in hyperspectral image classification

Considering the high-dimensional characteristics of hyperspectral image (HSI) data, researchers generally adopt the dimensionality reduction (DR) methods to reduce the complexity and computing time of subsequent classification or regression tasks while preserving the intrinsic structure information of the data. At present, the research on DR of HSIs data mostly focuses on the application performance of a single method in specific tasks and few studies have been conducted on the adaptability of different DR methods to HSIs data. From the perspective of spectral domain and spatial domain, this paper makes a comparative study on the performance of various linear and nonlinear DR methods in the task of HSI classification of aerial HSI of Matiwan Village in Xiongan New Area. Specifically, it includes principal component analysis, independent component analysis, isometric mapping, Laplacian eigenmaps, autoencoder, etc. The results show that the intrinsic structure in Xiongan HSI is mainly linear structure. Compared with the nonlinear DR methods, the linear DR methods can better preserve the intrinsic structure information of data at a lower time-consuming cost. For the subsequent classification tasks, the linear DR methods have better classification performance and are more suitable for HSI data.

Dimensionality reduction and classification for hyperspectral image based on robust supervised ISOMAP

ABSTRACT This study contributes to classification and recognition of hyperspectral image in industrial sorting, mineral identification, geological exploration, environmental survey, and other industrial production fields. The finding eliminates the redundancy of hyperspectral data and improves the classification accuracy of hyperspectral image. Hyperspectral remote sensing data are inherently nonlinear, and traditional linear dimensionality reduction methods cannot effectively reveal the nonlinear structure contained in the data. Manifold learning is a nonlinear dimensionality reduction method which has been developed rapidly in recent years. The intrinsic structure of the data is preserved by Isometric Feature Mapping (ISOMAP), but it is sensitive to noise and is not conducive to final classification. Prior studies neglected a higher dimensional manifold real structure in hyperspectral data dimensionality reduction due to lack of consideration credibility information and class information of samples in calculating manifold distance. This study proposes a Robust Supervised ISOMAP method. Data sample credibility mode is constructed, and a data similarity measure named triple geodesic distance is defined by introducing samples’ credibility information, class information, and neighborhood information. Triple geodesic distance fitting is proven. Multidimensional scaling analysis (MDS) and generalized regression neural network are used to construct the embedded coordinates of training samples and test samples. The results show that the proposed method has higher anti-noise performance than the traditional one. Also, the runtime is feasible, and can effectively improve the classification precision of the hyperspectral images, in comparison to traditional ISOMAP methods. The conclusion is that Robust Supervised ISOMAP is an effective nonlinear dimensionality reduction method for hyperspectral remote sensing image.

An appearance-driven space to create new BRDFs

The search for more realism in renderings led to increased interest in tabular bidirectional reflectance distribution function (BRDF) samples captured from real-world materials. This representation provides a high degree of realism at the cost of a long acquisition time and high storage space. Because of those drawbacks, the literature presents few BRDF databases captured from real materials, and the available ones present a limited number of samples. Aiming to create new and realistic-looking materials, this work proposes an approach to generate new BRDFs from a tabular BRDF database. Our goal is to provide a pipeline where the user expresses appearance necessities by selecting materials from a BRDF database. We generate an appearance-driven space shaped by the user’s interests, in which it is feasible to navigate and retrieve new tabular BRDFs. This new space should enable smooth navigation and, to provide visual richness and realism, not rely only on the materials selected by the user. To this end, we first preprocess a tabular BRDF database by keeping BRDF samples with relevant reflectance features. A low-dimensional representation of this preprocessed database is obtained by applying a dimensionality reduction method, and a clustering algorithm is applied to obtain clusters of materials with similar features. Based on the materials selected by the user, we define one or more clusters of materials to build the appearance-driven space. Any point in this lower-dimensional space can be mapped to the original BRDFs space. Therefore, new BRDFs that present similar properties to the ones which the user selected are created. We compare the performance of linear (Multidimensional Scaling - MDS) and nonlinear (Isometric Feature Mapping - ISOMAP) dimensionality reduction methods in this task and analyze the obtained materials.

Linear reduced order method for design-space dimensionality reduction and flow-field learning in hull form optimization

In the earlier stage of hull form optimization design, a series of design variables is usually needed to control the hull shape, so as to find optimal hull forms with better performance. In the surrogate-based hydrodynamic performance optimization for ships, with the increase of the dimensionality of design space, the number of new sample hulls to construct surrogate model needs to be larger, which will bring a large amount of calculation. Through reduced order method, the dimensionality of the optimization design space can be reduced while keeping the deformation range of the original design space to a great extent, for instance, using the linear combination of a smaller number of bases to represent the deformation range. In addition, in the later stage of hull form optimization design, flow field results of the new sample hulls can be fully utilized to do the dimensionality reduction multi-physics field learning. In this paper, the principle of the Proper Orthogonal Decomposition method is used and briefly introduced, the steps of dimensionality reduction of the design space are shown then, and some important problems for the design-space dimensionality reduction in the specific field of hull form optimization, such as retainability of fixed control points, irrelevance of the relative order of data to dimensionality reduction results, and decision of the new design space range after dimensionality reduction, are deep analyzed. Furthermore, taking the resistance optimization of the modified Wigley ship as an example, the specific application and error analysis of the dimensionality reduction method for design-space dimensionality reduction in the earlier stage of hull form optimization and the multi-physics field learning in the later stage of hull form optimization are given, and the applicability and reliability of the method are demonstrated by analyzing the influence of mode order and sample number on reconstruction effect of the hull shape or flow field, and the prediction effect of flow field for not-in-the-database new hull form in detail. Results show that the linear dimensionality reduction method can reduce samples needed for optimization, thus reduce the amount of calculation for the surrogate-based hull form optimization, and be used for quick prediction of multi-physics fields of any new form in the design space. Furthermore, it can not only be applied to the sensitivity analysis or a Pareto frontier selection in comprehensive performance optimization of hull form based on CFD, but also be implemented in the real-time forecast of the flow field and influence analysis of the ship performance when adjusting the hull form (or hull appendages).

Nondestructive Detection of Keemun Black Tea Grade Based on Hyperspectral Imaging Technique

In order to evaluate the grade of tea products quickly and nondestructive, the near infrared (900~1700 nm) hyperspectral imaging technology was applied to classify six grades of black tea. Firstly, the hyperspectral data were visualized by linear and nonlinear dimension reduction methods. The visualization algorithms included the linear method of principal component analysis (PCA), multi-dimensional scaling (MDS), the nonlinear method of T-distributed stochastic embedding (T-SNE) and Sammon nonlinear mapping. Secondly, the classification model was established by using support vector machine (SVM) and extreme learning machine (ELM) to identify the different grades of Keemun black tea. Finally, SVM and ELM classification model were used to identify each pixel of hyperspectral image, and the prediction map was obtained. The results showed that T-SNE could divide the six grades of Keemun black tea into six different clusters, and the accuracy of SVM and ELM prediction set was 100% and 96.35%, respectively. The T-SNE visualization effect was the best, and the detection model of SVM could effectively identify the six grades of Keemun black tea. This paper provides an effective method for rapid and nondestructive testing of tea product grade, which had great significance for quality control, authenticity and adulteration detection of tea products.

Fingerprinting sources of salinity in a coastal chalk aquifer in Denmark using trace elements

Salinity levels above the drinking water standard (>250 mg/l Cl–) are observed at shallow depth in a Maastrichtian chalk aquifer on the island of Falster, south-eastern Denmark. To understand the source of the salt, 63 samples from 12 individual, 1 m, screened intervals between 14 and 26 m b.s. were collected from 1 May to 4 June 2018. The samples were collected during a tracer test to estimate the dual porosity properties of the chalk and were analysed for a wide range of elements. Furthermore, samples from the Baltic Sea and from deeper saline aquifers in the area (40 and 85 m b.s.) were analysed for comparison. The geochemical data were analysed using an unsupervised machine-learning algorithm, self-organising maps, to fingerprint water sources. The water composition in the screened intervals at various stratigraphic levels has specific geochemical fingerprints that are maintained for the first days of pumping and are distinct amongst the different levels. This suggests an evolution in water composition because of reaction with the chalk. Water composition is distinct from both seawater from the nearby Baltic Sea and salty water from deeper levels of the reservoir. Thus, neither up-coning of salty water nor intrusion of seawater caused the elevated salinity levels in the area. The slightly saline composition of groundwater in the shallow aquifer (14–26 m b.s.) is more likely because of incomplete refreshing of the salty connate water in the chalk during the Pleistocene and Holocene. Furthermore, the geochemical fingerprint of salty water from the deeper aquifer at 40 m was similar to water from the Baltic Sea, suggesting a Baltic Sea source for salt in the aquifer at 40 m b.s., c. 100 m from the coast. Statistical analysis based on self-organising maps is an effective tool for interpreting a large number of variables to understand the compositional variation in an aquifer and a useful alternative to linear dimensionality-reduction methods such as principal component analysis. The approach using the multi-element analysis combined with the analysis of self-organising maps may be useful in future studies of groundwater quality.

A Monte Carlo Comparison of Two Linear Dimension Reduction Matrices for Statistical Discrimination

We compare two linear dimension-reduction methods for statisti cal discrimination in terms of average probabilities of misclassification in re duced dimensions. Using Monte Carlo simulation we compare the dimension reduction methods over several different parameter configurations of multi variate normal populations and find that the two methods yield very different results. We also apply the two dimension-reduction methods examined here to data from a study on football helmet design and neck injuries.

The Laplacian eigenmaps dimensionality reduction of fMRI data for discovering stimulus-induced changes in the resting-state brain activity

The brain at wakefulness is active even in the absence of goal-directed behavior or salient stimuli. However, patterns of this resting-state (RS) activity can undergo long-term alterations following exposure to preceding meaningful stimuli. This study was aimed to develop an unbiased method to detect such changes in the RS activity after exposure to emotionally meaningful stimuli. For this purpose, we used functional magnetic resonance imaging (fMRI) of RS brain activity before and after acquisition and extinction of experimental conditioned fear. A group of healthy volunteers participated in three fMRI sessions: a RS before fear conditioning, a fear extinction session, and a RS immediately after fear extinction. The fear-conditioning paradigm consisted of three neutral visual stimuli paired with a partial reinforcement by a mild electric current. We used both linear and non-linear dimensionality reduction approaches to distinguish between the initial RS and the RS after stimuli exposure. The principal component analysis (PCA) as a linear dimensionality reduction method showed significantly worse results than non-linear methods (Isomap, LLE, Laplacian eigenmaps). Using the Laplacian eigenmaps manifold learning method, we were able to show significant differences between the two RSs at the level of individual participants. This detection was further improved by smoothing the BOLD signal with the wavelet multiresolution analysis. The developed method can improve the discrimination of functional states collected in longitudinal fMRI studies.

Dimensionality reduction by UMAP reinforces sample heterogeneity analysis in bulk transcriptomic data.

Transcriptomic analysis plays a key role in biomedical research. Linear dimensionality reduction methods, especially principal-component analysis (PCA), are widely used in detecting sample-to-sample heterogeneity, while recently developed non-linear methods, such as t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP), can efficiently cluster heterogeneous samples in single-cell RNA sequencing analysis. Yet, the application of t-SNE and UMAP in bulk transcriptomic analysis and comparison with conventional methods have not been achieved. We compare four major dimensionality reduction methods (PCA, multidimensional scaling [MDS], t-SNE, and UMAP) in analyzing 71 large bulk transcriptomic datasets. UMAP is superior to PCA and MDS but shows some advantages over t-SNE in differentiating batch effects, identifying pre-defined biological groups, and revealing in-depth clusters in two-dimensional space. Importantly, UMAP generates sample clusters uncovering biological features and clinical meaning. We recommend deploying UMAP in visualizing and analyzing sizable bulk transcriptomic datasets to reinforce sample heterogeneity analysis.