Unsupervised Dimensionality Reduction Method Research Articles

Dimensionality reduction beyond neural subspaces with slice tensor component analysis

Recent work has argued that large-scale neural recordings are often well described by patterns of coactivation across neurons. Yet the view that neural variability is constrained to a fixed, low-dimensional subspace may overlook higher-dimensional structure, including stereotyped neural sequences or slowly evolving latent spaces. Here we argue that task-relevant variability in neural data can also cofluctuate over trials or time, defining distinct ‘covariability classes’ that may co-occur within the same dataset. To demix these covariability classes, we develop sliceTCA (slice tensor component analysis), a new unsupervised dimensionality reduction method for neural data tensors. In three example datasets, including motor cortical activity during a classic reaching task in primates and recent multiregion recordings in mice, we show that sliceTCA can capture more task-relevant structure in neural data using fewer components than traditional methods. Overall, our theoretical framework extends the classic view of low-dimensional population activity by incorporating additional classes of latent variables capturing higher-dimensional structure.

Learning adaptive Grassmann neighbors for image-set analysis

Representing image sets as subspaces on Grassmann manifold and leveraging the Riemannian geometry of this space has proven to be highly effective in various visual recognition tasks. However, the computational cost associated with operating on Riemannian manifolds, especially of high-dimensional ones, poses a significant limitation to the applicability of existing techniques. Moreover, the image sets with labels are scarce and valuable in real-world scenarios, where the manual annotation is time-consuming and costly. To tackle these issues, in this paper, we propose an unsupervised Grassmannian dimensionality reduction method for image set classification and clustering, which simultaneously performs local structure learning and dimensionality reduction under the subspace view. Specifically, instead of relying on the pre-defined similarity matrix that obtained from raw data and may contain much noise, we adaptively learn the local structures from the projected low-dimensional Grassmann manifold. The refined structures then provide more discriminative power to learn the accurate manifold-to-manifold mapping. To better preserve the global structure of the data, this mapping is constrained to satisfy the orthogonal condition. In addition, our model can be compatible with F-norm OLPP for Grassmann manifold when the squared F-norm is transformed into projection metric. A re-weighted iterative scheme is carefully developed to optimize the problem. Experimental results on several benchmark datasets for image set classification and clustering tasks clearly demonstrate that our model consistently outperforms baselines, yielding significant improvements.

A novel untrained SSVEP-EEG feature enhancement method using canonical correlation analysis and underdamped second-order stochastic resonance.

Compared with the light-flashing paradigm, the ring-shaped motion checkerboard patterns avoid uncomfortable flicker or brightness modulation, improving the practical interactivity of brain-computer interface (BCI) applications. However, due to fewer harmonic responses and more concentrated frequency energy elicited by the ring-shaped checkerboard patterns, the mainstream untrained algorithms such as canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA) methods have poor recognition performance and low information transmission rate (ITR). To address this issue, a novel untrained SSVEP-EEG feature enhancement method using CCA and underdamped second-order stochastic resonance (USSR) is proposed to extract electroencephalogram (EEG) features. In contrast to typical unsupervised dimensionality reduction methods such as common average reference (CAR), principal component analysis (PCA), multidimensional scaling (MDS), and locally linear embedding (LLE), CCA exhibits higher adaptability for SSVEP rhythm components. This study recruits 42 subjects to evaluate the proposed method and experimental results show that the untrained method can achieve higher detection accuracy and robustness. This untrained method provides the possibility of applying a nonlinear model from one-dimensional signals to multi-dimensional signals.

Unsupervised Adaptive Bipartite Graph Embedding

In traditional graph embedding methods, graph construction is sensitive to high-dimensional data with noise and outliers, making an effective exploration of the neighborhood structure of the data difficult. Besides, with these methods, constructing graphs and reducing dimensions are disconnected and cannot be mutually optimized. To address these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, named unsupervised adaptive bipartite graph embedding (UABGE). First, the anchors are generated from the raw data by K-means or random sampling. Second, the bipartite graph, which is constructed between the samples and the anchors in the low-dimensional subspace, utilizes the adaptive allocation method to assign neighbors for each sample, so that the local structure of high-dimensional data can be captured effectively. Third, we present an objective function that combines bipartite graph construction and projection matrix learning to achieve mutual optimization between them, which can be solved with an alternating optimization algorithm. Finally, the computational complexity and the convergence of the algorithm are analyzed. Experimental results on synthetic data and publicly available datasets illustrate the effectiveness of the proposed method.

Unsupervised Feature Representation of Sleep EEG Data with Transient Deep Boltzmann Machine.

The supervised sleep staging methods are challenged by their strict requirements of a labelled and large dataset. This study considers an unsupervised dimensionality reduction method, the Deep Boltzmann Machine (DBM), trained to a transient state for binary classification of sleep stages. First, the joint time-frequency domain features from the polysomnographic recordings are extracted. Second, the extracted features are smoothed using 2 min rolling window to include contextual temporal information, and finally, they serve as an input for unsupervised training of DBM_transient. The results show that our method effectively separates the sleep stages in two-dimensional feature space with a large Fisher's discriminant value. The classification performance by the DBM_transient achieves a 96.1% F1 score, which is higher than DBM converged to an equilibrium state (95.2%), Principal Component Analysis (92.5%), Isometric Feature Mapping (95.9%), t-distributed Stochastic Neighbor Embedding (94.9%), and Uniform Manifold Approximation (95.0%) on the widely used sleep-EDF database. Additionally, Fisher's discriminant function demonstrates the superiority of the DBM_transient. The significance of the DBM transient lies in its ease of interpretability in two-dimensional space, and future multi-class implementation of the method may facilitate its usage in clinical applications.

Decorrelated spectral regression: An unsupervised dimension reduction method under data selection bias

In manipulating high-dimensional data arising from real applications, dimension reduction methods have been proven to be effective and necessary. Nevertheless, the widespread selection bias, induced in the process of data collection, has long been unaware and its negative effects have been ignored. The accidental spurious correlation between the features brought by selection bias will deteriorate the performance of dimension reduction approaches. It is vital to remove such bias and consider its influence on dimension reduction. In this paper, we propose a novel decorrelated spectral regression algorithm for unsupervised dimension reduction on data with sample selection bias. In this method, the sample weights of the global feature distribution are learned by introducing the decorrelation regularizer. The learned weights are then combined with the regression model to obtain the dimension reduction mapping. Thus, the dimension reduction results are less affected by the accidental spurious correlation of data caused by selection bias. In addition, we also propose the updating rules of the parameters in our algorithm. A large number of experimental results on real-world datasets show that our method has achieved a significant improvement, indicating that it is effective and necessary to remove the accidental spurious correlation between features caused by selection bias in dimension reduction.

Fast and Robust Unsupervised Dimensionality Reduction with Adaptive Bipartite Graphs

Recent unsupervised dimension reduction algorithms use similarity graphs between data point pairs to preserve local structure while reducing dimension. However, the time complexity of these methods is proportional to the square of the number of samples, which limits their application to large-scale datasets. Moreover, the square Euclidean calculation criterion between sample point pairs will magnify the bad influence of outliers on the graph. In addition, these methods only preserve the local structure while losing other important structural information. To this end, we propose a fast adaptive unsupervised projection model termed Fast and Robust Unsupervised Dimensionality Reduction with Adaptive Bipartite Graph (FRUDR-ABG), which uses a few anchor points and sample points to build a bipartite graph to preserve the local geometric structure of the data to reduce the running time and improve efficiency. We propose a criterion based on the l2,1 norm to calculate the distance between anchor points and data points to reduce the negative influence of outliers on graph construction. A practical strategy is also proposed to realize joint learning of global and local structures. According to the characteristics of graph construction and dimensionality reduction adaptive learning in the algorithm, we design an iterative reweighting method to solve the model. Experimental results on several benchmark datasets show that FRUDR-ABG has higher efficiency and recognition performance than existing unsupervised dimensionality reduction methods.

RRNMF-MAGL: Robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning for dimensionality reduction

In this paper, a new unsupervised dimensionality reduction method named robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning (RRNMF-MAGL) is developed. Compared with the existing methods, RRNMF-MAGL can extract robust low-dimensional features and adaptively learn a manifold structure to well reflect the data distribution. Specifically, to alleviate the influence of outliers and noises, a robust non-negative matrix factorization (RNMF) approach with L21-norm loss function is first constructed. Then, a multi-constraint adaptive graph learning (MAGL) model is designed based on low-dimensional features to reduce the computational complexity and avoid the adverse influence of redundant information. Moreover, multiple constraints (i.e., sparsity and locality) are incorporated into graph learning for enhancing the discrimination ability of graph structure. Next, to make the learned low-dimensional features well maintain the local geometric structure information of data, a graph Laplacian regularization (GLR) is designed. Finally, an iterative update strategy is proposed to optimize RRNMF-MAGL and its convergence is verified by numerical experiments. The superior effectiveness and robustness of RRNMF-MAGL is demonstrated by extensive experiments on eight publicly available datasets.

Dimension reduction method of high-dimensional fault datasets based on C_M_t-SNE under unsupervised background

The unlabeled fault datasets often contain much non-sensitive redundant, and uncertain information. This study designs a novel interpretable and unsupervised dimension reduction method for unlabeled data containing redundancy and uncertainty. Firstly, a fuzzy-based way for pseudo-label generation is given, and feature cloud models under pseudo labels are established; Secondly, this study takes the expectation, entropy, and hyper entropy of the cloud models representing uncertainty in features as spatial vectors. The difference degree between vectors is treated as the evaluation standard to filter out non-sensitive features based on the maximum initial difference; Moreover, redundant elements are fused by t-SNE, and lower dimensional feature components conducive for fault classification are obtained; Finally, the effectiveness of the method is demonstrated by comparative experiments. The results show that this method has a higher factor, which means that the method can better mine the difference among different faults and improve the performance of fault identification.

Linear dimensionality reduction method based on topological properties

Dimensionality reduction is an important data preprocessing technique that has been extensively studied in machine learning and data mining. Locality Preserving Projection (LPP) is a widely used linear unsupervised dimensionality reduction method, which maps high-dimensional data into low-dimensional subspace through linear transformation. Although various variants of LPP have been proposed to tackle different drawbacks of LPP, it is identified in this article that LPP does not possess the important topological property of translation invariance, that is, the linear transformation given by LPP is strongly related to the relative position between the data and the origin of the coordinate system. In this article, we theoretically analyze the reason why this drawback exists in LPP and propose to resolve it by introducing a kind of centralization to the model. Moreover, as topological properties are prominent information to characterize the structure of the data, this article proposes a further improvement of LPP to maintain topological connectivity of data after dimensionality reduction. Experiments on multiple synthetic and real-world datasets show that the new model incorporating topological properties outperforms not only the original LPP model but also several other classic linear or non-linear dimensionality reduction methods.

Comparison of dimension reduction methods for the identification of heart-healthy dietary patterns

Most nutritional epidemiology studies investigating diet-disease trends use unsupervised dimension reduction methods, like principal component regression (PCR) and sparse PCR (SPCR), to create dietary patterns. Supervised methods, such as partial least squares (PLS), sparse PLS (SPLS), and Lasso, offer the possibility of more concisely summarizing the foods most related to a disease. In this study we evaluate these five methods for interpretable reduction of food frequency questionnaire (FFQ) data when analyzing a univariate continuous cardiac-related outcome via a simulation study and data application. We also demonstrate that to control for covariates, various scientific premises require different adjustment approaches when using PLS. To emulate food groups, we generated blocks of normally distributed predictors with varying intra-block covariances; only nine of 24 predictors contributed to the normal response. When block covariances were informed by FFQ data, the only methods that performed variable selection were Lasso and SPLS, which selected two and four irrelevant variables, respectively. SPLS had the lowest prediction error, and both PLS-based methods constructed four patterns, while PCR and SPCR created 24 patterns. These methods were applied to 120 FFQ variables and baseline body mass index (BMI) from the Multi-Ethnic Study of Atherosclerosis, which includes 6814 participants aged 45-84, and we adjusted for age, gender, race/ethnicity, exercise, and total energy intake. From 120 variables, PCR created 17 BMI-related patterns and PLS selected one pattern; SPLS only used five variables to create two patterns. All methods exhibited similar predictive performance. Specifically, SPLS’s first pattern highlighted hamburger and diet soda intake (positive associations with BMI), reflecting a fast food diet. By selecting fewer patterns and foods, SPLS can create interpretable dietary patterns while maintaining predictive ability.

Unsupervised Adaptive Embedding for Dimensionality Reduction.

High-dimensional data are highly correlative and redundant, making it difficult to explore and analyze. Amount of unsupervised dimensionality reduction (DR) methods has been proposed, in which constructing a neighborhood graph is the primary step of DR methods. However, there exist two problems: 1) the construction of graph is usually separate from the selection of projection direction and 2) the original data are inevitably noisy. In this article, we propose an unsupervised adaptive embedding (UAE) method for DR to solve these challenges, which is a linear graph-embedding method. First, an adaptive allocation method of neighbors is proposed to construct the affinity graph. Second, the construction of affinity graph and calculation of projection matrix are integrated together. It considers the local relationship between samples and global characteristic of high-dimensional data, in which the cleaned data matrix is originally proposed to remove noise in subspace. The relationship between our method and local preserving projections (LPPs) is also explored. Finally, an alternative iteration optimization algorithm is derived to solve our model, the convergence and computational complexity of which are also analyzed. Comprehensive experiments on synthetic and benchmark datasets illustrate the superiority of our method.

Unsupervised learning of ferroic variants from atomically resolved STEM images

An approach for the analysis of atomically resolved scanning transmission electron microscopy data with multiple ferroic variants in the presence of imaging non-idealities and chemical variabilities based on a rotationally invariant variational autoencoder (rVAE) is presented. We show that an optimal local descriptor for the analysis is a sub-image centered at specific atomic units, since materials and microscope distortions preclude the use of an ideal lattice as a reference point. The applicability of unsupervised clustering and dimensionality reduction methods is explored and is shown to produce clusters dominated by chemical and microscope effects, with a large number of classes required to establish the presence of rotational variants. Comparatively, the rVAE allows extraction of the angle corresponding to the orientation of ferroic variants explicitly, enabling straightforward identification of the ferroic variants as regions with constant or smoothly changing latent variables and sharp orientational changes. This approach allows further exploration of the chemical variability by separating the rotational degrees of freedom via rVAE and searching for remaining variability in the system. The code used in this article is available at https://github.com/saimani5/ferroelectric_domains_rVAE.

The Origin and Discrimination of High-Ti Magnetite in Magmatic-Hydrothermal Systems: Insight from Machine Learning Analysis

Abstract High-titanium (high-Ti, more than 1 wt % Ti) magnetite, commonly containing ilmenite exsolution, has long been attributed to an igneous origin and has been used as the most critical factor in previously developed discriminant diagrams. However, recent studies have shown that high-Ti magnetite can be present in high-temperature hydrothermal deposits, suggesting a probable hydrothermal origin. This also calls for reconsideration and necessary modification of the currently available discriminant diagrams. This high-Ti magnetite issue is particularly acute in iron oxide-apatite (IOA) deposits and raises controversy in the discussion of the origin of the high-Ti magnetite. With statistical analysis and machine learning techniques, this study applies two unsupervised dimensionality reduction methods—principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE)—on a compiled data set consisting of 876 laser ablation-inductively coupled plasma-mass spectrometry analyses of primary high-Ti magnetite from high-temperature ore-forming systems worldwide. Three models are built with different element combinations to identify magnetite of different origins. The models were further evaluated by the support vectors machine (SVM) and receiver operating characteristic (ROC) curves and proved to be able to describe the characteristics of trace element compositions of high-Ti magnetite of different origins. Our models suggest that Mg, Mn, Al, Ti, V, and Co from 59 analyzed trace elements show promising properties as effective discriminators, and on this basis, a new discrimination diagram of lg(Al) + lg(Ti) + lg(V) versus lg(Mn)/[lg(Co) + lg(Mg)] is developed for distinguishing high-Ti magnetite of igneous and hydrothermal origin. Our results also show that the high-Ti magnetite in the IOA deposits has chemical compositions similar to those of high-temperature hydrothermal deposits, including the iron oxide copper-gold and porphyry deposits, but significantly distinct from the igneous magnetite. Our study, hence, implies a magmatic-hydrothermal origin for the magnetite in IOA deposits.

Unsupervised double weighted graphs via good neighbours for dimension reduction of hyperspectral image

ABSTRACT As the major research in pattern recognition, unsupervised dimension reduction is a challenging problem because of no label information. Most unsupervised dimension reduction methods usually construct similarity graph by k-nearest neighbour to preserve local structure in the low-dimensional subspace. However, k-nearest neighbour is calculated by Euclidean distance, which is sensitive to noise and outliers. And only considering local structure will reduce the classification accuracy. In this paper, a new unsupervised dimension reduction method called unsupervised double-weighted graphs via good neighbours (uDWG-GN) is proposed. First, uDWG-GN proposes a local structure Low-Rank Representation to learn similarity matrix and then uses the similarity matrix to find good neighbours of each sample. Second, according to good neighbours of the sample, uDWG-GN considers similar and dissimilar relationship between samples and constructs double-weighted graphs. Finally, based on norm, uDWG-GN finds the optimal projection matrix by maximizing the distance of dissimilar samples and minimizing the distance of similar samples. Experimental results on three hyperspectral images demonstrate the superiority and effectiveness of our method compared with other dimension reduction methods.

Visual Clustering of Transcriptomic Data from Primary and Metastatic Tumors-Dependencies and Novel Pitfalls.

Personalized oncology is a rapidly evolving area and offers cancer patients therapy options that are more specific than ever. However, there is still a lack of understanding regarding transcriptomic similarities or differences of metastases and corresponding primary sites. Applying two unsupervised dimension reduction methods (t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP)) on three datasets of metastases (n = 682 samples) with three different data transformations (unprocessed, log10 as well as log10 + 1 transformed values), we visualized potential underlying clusters. Additionally, we analyzed two datasets (n = 616 samples) containing metastases and primary tumors of one entity, to point out potential familiarities. Using these methods, no tight link between the site of resection and cluster formation outcome could be demonstrated, or for datasets consisting of solely metastasis or mixed datasets. Instead, dimension reduction methods and data transformation significantly impacted visual clustering results. Our findings strongly suggest data transformation to be considered as another key element in the interpretation of visual clustering approaches along with initialization and different parameters. Furthermore, the results highlight the need for a more thorough examination of parameters used in the analysis of clusters.

Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

Hypergraph learning is largely used to handle dimensional disaster problem of high-dimensional data in view of its flexible ability to model complex data correlations. However, the following three problems limit the performance of unsupervised hypergraph-based dimensionality reduction methods: (1) most of them are unable to obtain consistently good performance for different data sets; (2) noise or outliers in the data can lead to unreliability of the fixed hypergraph; and (3) the simple K-nearest neighbor strategy generates hypergraph with a fixed number of vertices in each hyperedge. To solve the above problems, an unsupervised dimensionality reduction method named joint dynamic hypergraph and low-rank embedding (DHLRE) is proposed. Specifically, DHLRE unifies hypergraph learning and low-rank learning into a single objective function. By doing so, both global and higher-order local structural information of the data can be extracted by DHLRE, and the two structural information complement each other to make the projection matrix have stronger discriminative power. In addition, the weight of hyperedge and hypergraph in DHLRE are dynamically generated in dimension-reduced space, which suppresses the ability of noise or outlier to affect the projection matrix. To divide hyperedge in a flexible way, a new approach without neighborhood parameter is proposed. By using this approach, the number of vertices of the hyperedge can be adaptively decided for different data sets. Further, an iterative algorithm is designed to solve the objective function of DHLRE. Finally, experimental results on several real-world data sets show that DHLRE outperforms others related methods for both classification and clustering tasks.

Organization of gene programs revealed by unsupervised analysis of diverse gene-trait associations.

Genome wide association studies provide statistical measures of gene–trait associations that reveal how genetic variation influences phenotypes. This study develops an unsupervised dimensionality reduction method called UnTANGLeD (Unsupervised Trait Analysis of Networks from Gene Level Data) which organizes 16,849 genes into discrete gene programs by measuring the statistical association between genetic variants and 1,393 diverse complex traits. UnTANGLeD reveals 173 gene clusters enriched for protein–protein interactions and highly distinct biological processes governing development, signalling, disease, and homeostasis. We identify diverse gene networks with robust interactions but not associated with known biological processes. Analysis of independent disease traits shows that UnTANGLeD gene clusters are conserved across all complex traits, providing a simple and powerful framework to predict novel gene candidates and programs influencing orthogonal disease phenotypes. Collectively, this study demonstrates that gene programs co-ordinately orchestrating cell functions can be identified without reliance on prior knowledge, providing a method for use in functional annotation, hypothesis generation, machine learning and prediction algorithms, and the interpretation of diverse genomic data.

Double information preserving canonical correlation analysis

The methods based on canonical correlation analysis (CCA-based methods) are typical and effective methods for unsupervised dimensionality reduction of multi-view data. However, the traditional CCA-based methods ignore the dissimilarity information while considering the similarity information of samples, which will make the heterogeneous samples in the subspace cannot be well separated. In this paper, we propose a novel unsupervised multi-view dimensionality reduction method: Double Information Preserving Canonical Correlation Analysis (DIPCCA). DIPCCA aims at finding two projection matrices by integrating two cross double weight graphs with CCA to explore the similarity and dissimilarity information of data in cross views. Furthermore, on the basis of consistency and complementarity, DIPCCA uses the similarity information to maintain the local structure, and the dissimilarity information to disperse embedded samples of distinct clusters, so as to extract more discriminative features. Moreover, CCA and a new locality-preserving CCA are two special cases of DIPCCA when parameters take special values. In order to better extract the features of nonlinear data and more than two views, DIPCCA is extended to Double Information Preserving Kernel Canonical Correlation Analysis and Double Information Preserving Multiple Canonical Correlation Analysis, respectively. Experiments on an artificial dataset and three real multi-view datasets show our proposed methods have better performance than the traditional CCA-based methods.

Supervised and unsupervised machine learning of structural phases of polymers adsorbed to nanowires.

We identify configurational phases and structural transitions in a polymer nanotube composite by means of machine learning. We employ various unsupervised dimensionality reduction methods, conventional neural networks, as well as the confusion method, an unsupervised neural-network-based approach. We find neural networks are able to reliably recognize all configurational phases that have been found previously in experiment and simulation. Furthermore, we locate the boundaries between configurational phases in a way that removes human intuition or bias. This could be done before only by relying on preconceived, ad hoc order parameters.