Data In Low-dimensional Space Research Articles

The effect of data transformation on low-dimensional integration of single-cell RNA-seq

BackgroundRecent developments in single-cell RNA sequencing have opened up a multitude of possibilities to study tissues at the level of cellular populations. However, the heterogeneity in single-cell sequencing data necessitates appropriate procedures to adjust for technological limitations and various sources of noise when integrating datasets from different studies. While many analysis procedures employ various preprocessing steps, they often overlook the importance of selecting and optimizing the employed data transformation methods.ResultsThis work investigates data transformation approaches used in single-cell clustering analysis tools and their effects on batch integration analysis. In particular, we compare 16 transformations and their impact on the low-dimensional representations, aiming to reduce the batch effect and integrate multiple single-cell sequencing data. Our results show that data transformations strongly influence the results of single-cell clustering on low-dimensional data space, such as those generated by UMAP or PCA. Moreover, these changes in low-dimensional space significantly affect trajectory analysis using multiple datasets, as well. However, the performance of the data transformations greatly varies across datasets, and the optimal method was different for each dataset. Additionally, we explored how data transformation impacts the analysis of deep feature encodings using deep neural network-based models, including autoencoder-based models and proto-typical networks. Data transformation also strongly affects the outcome of deep neural network models.ConclusionsOur findings suggest that the batch effect and noise in integrative analysis are highly influenced by data transformation. Low-dimensional features can integrate different batches well when proper data transformation is applied. Furthermore, we found that the batch mixing score on low-dimensional space can guide the selection of the optimal data transformation. In conclusion, data preprocessing is one of the most crucial analysis steps and needs to be cautiously considered in the integrative analysis of multiple scRNA-seq datasets.

Laplacian-based Cluster-Contractive t-SNE for High-Dimensional Data Visualization

Dimensionality reduction techniques aim at representing high-dimensional data in low-dimensional spaces to extract hidden and useful information or facilitate visual understanding and interpretation of the data. However, few of them take into consideration the potential cluster information contained implicitly in the high-dimensional data. In this article, we propose Lap tSNE, a new graph-layout nonlinear dimensionality reduction method based on t-SNE, one of the best techniques for visualizing high-dimensional data as 2D scatter plots. Specifically, Lap tSNE leverages the eigenvalue information of the graph Laplacian to shrink the potential clusters in the low-dimensional embedding when learning to preserve the local and global structure from high-dimensional space to low-dimensional space. It is nontrivial to solve the proposed model because the eigenvalues of normalized symmetric Laplacian are functions of the decision variable. We provide a majorization-minimization algorithm with convergence guarantee to solve the optimization problem of Lap tSNE and show how to calculate the gradient analytically, which may be of broad interest when considering optimization with Laplacian-composited objective. We evaluate our method by a formal comparison with state-of-the-art methods on seven benchmark datasets, both visually and via established quantitative measurements. The results demonstrate the superiority of our method over baselines such as t-SNE and UMAP. We also provide out-of-sample extension, large-scale extension, and mini-batch extension for our Lap tSNE to facilitate dimensionality reduction in various scenarios.

Dynamic heterogeneous graph representation learning with neighborhood type modeling

Graph representation learning aims to learn the representations of graph structured data in low-dimensional space, and has a wide range of applications in graph analysis tasks. Real-world networks are generally heterogeneous and dynamic, which contain multiple types of nodes and edges, and the graph may evolve at a high speed over time. The complex heterogeneous properties and rapidly evolving graph structures make it difficult to learn high-quality graph representations for dynamic heterogeneous graphs. Currently, studies concentrated on representation learning of temporal heterogeneous networks are insufficient. Existing methods either rely on meta-paths where the embedding quality heavily depending on experts’ selection, or use network snapshots where the fine-grained temporal information cannot be captured. In this paper, we propose a novel graph neural network model–node signature based Temporal Heterogeneous Graph Attention Network, termed as THGAT, for learning the representations of dynamic heterogeneous networks. THGAT improves the aggregation way of neighborhood information, and pays attention to the enlightenment of the importance of neighbor nodes by heterogeneous information and temporal information that cannot be ignored in the network. We also innovatively propose three node signature methods for encoding the heterogeneous information of the nodes and use the time encoding technique suitable for real-time networks to directly represent the temporal information, so as to overcome the limitations of existing methods. We conduct experiments on four real-world datasets, and the results demonstrate that THGAT improves the representation learning quality significantly, in aspects of link prediction, node classification, and node clustering, compared to the state-of-the-art methods. To make the work more complete, we also analyze the applicable scenarios of the three node signature methods through experiments, respectively.

Visualizing hierarchies in scRNA-seq data using a density tree-biased autoencoder.

MotivationSingle-cell RNA sequencing (scRNA-seq) allows studying the development of cells in unprecedented detail. Given that many cellular differentiation processes are hierarchical, their scRNA-seq data are expected to be approximately tree-shaped in gene expression space. Inference and representation of this tree structure in two dimensions is highly desirable for biological interpretation and exploratory analysis.ResultsOur two contributions are an approach for identifying a meaningful tree structure from high-dimensional scRNA-seq data, and a visualization method respecting the tree structure. We extract the tree structure by means of a density-based maximum spanning tree on a vector quantization of the data and show that it captures biological information well. We then introduce density-tree biased autoencoder (DTAE), a tree-biased autoencoder that emphasizes the tree structure of the data in low dimensional space. We compare to other dimension reduction methods and demonstrate the success of our method both qualitatively and quantitatively on real and toy data.Availability and implementationOur implementation relying on PyTorch and Higra is available at github.com/hci-unihd/DTAE.Supplementary information Supplementary data are available at Bioinformatics online.

From Anomaly Detection to Novel Fault Discrimination for Wind Turbine Gearboxes With a Sparse Isolation Encoding Forest

As vital renewable energy devices, wind turbines suffer from gearbox failures due to harsh speed increasing operations. Therefore, the gearbox fault diagnosis is crucial for wind turbine maintenance with reducing economic costs. However, obtaining faulty data is rather challenging, especially at the early fault stage. For this reason, a sparse isolation encoding forest (SIEF) is proposed aiming at both anomaly detection and novel fault discrimination for wind turbine gearboxes. In the present SIEF method, a sparse autoencoder is first trained with only normal data to obtain an optimized and robust weight structure. Newly acquired data corresponding to faulty or healthy conditions are sent to this encoder for feature extraction by encoding to its low dimensional space. All the data in low-dimensional space are fed to an isolation forest for anomaly detection and novel fault discrimination. In the addressed SIEF approach, only normal data are required to train the model for fault detection and further discrimination. It is consistent with the actual operations of the wind turbines, with much less dependence on the fault data for the model training. The proposed method was evaluated by fault diagnosis tests on the wind turbine gearboxes. Results show good performances of the proposal compared to peers.

Deep Learning and Machine Learning for Early Detection of Stroke and Haemorrhage

Stroke and cerebral haemorrhage are the second leading causes of death in the world after ischaemic heart disease. In this work, a dataset containing medical, physiological and environmental tests for stroke was used to evaluate the efficacy of machine learning, deep learning and a hybrid technique between deep learning and machine learning on the Magnetic Resonance Imaging (MRI) dataset for cerebral haemorrhage. In the first dataset (medical records), two features, namely, diabetes and obesity, were created on the basis of the values of the corresponding features. The t-Distributed Stochastic Neighbour Embedding algorithm was applied to represent the high-dimensional dataset in a low-dimensional data space. Meanwhile,the Recursive Feature Elimination algorithm (RFE) was applied to rank the features according to priority and their correlation to the target feature and to remove the unimportant features. The features are fed into the various classification algorithms, namely, Support Vector Machine (SVM), K Nearest Neighbours (KNN), Decision Tree, Random Forest, and Multilayer Perceptron. All algorithms achieved superior results. The Random Forest algorithm achieved the best performance amongst the algorithms; it reached an overall accuracy of 99%. This algorithm classified stroke cases with Precision, Recall and F1 score of 98%, 100% and 99%, respectively. In the second dataset, the MRI image dataset was evaluated by using the AlexNet model and AlexNet + SVM hybrid technique. The hybrid model AlexNet + SVM performed is better than the AlexNet model; it reached accuracy, sensitivity, specificity and Area Under the Curve (AUC) of 99.9%, 100%, 99.80% and 99.86%, respectively.

Manifold regularization ensemble clustering with many objectives using unsupervised extreme learning machines

Spectral clustering has been an effective clustering method, in last decades, because it can get an optimal solution without any assumptions on data’s structure. The basic key in spectral clustering is its similarity matrix. Despite many empirical successes in similarity matrix construction, almost all previous methods suffer from handling just one objective. To address the multi-objective ensemble clustering, we introduce a new ensemble manifold regularization (MR) method based on stacking framework. In our Manifold Regularization Ensemble Clustering (MREC) method, several objective functions are considered simultaneously, as a robust method for constructing the similarity matrix. Using it, the unsupervised extreme learning machine (UELM) is employed to find the generalized eigenvectors to embed the data in low-dimensional space. These eigenvectors are then used as the base point in spectral clustering to find the best partitioning of the data. The aims of this paper are to find robust partitioning that satisfy multiple objectives, handling noisy data, keeping diversity-based goals, and dimension reduction. Experiments on some real-world datasets besides to three benchmark protein datasets demonstrate the superiority of MREC over some state-of-the-art single and ensemble methods.

Dimensionality Reduction for Tensor Data Based on Local Decision Margin Maximization.

In machine learning, the idea of maximizing the margin between two classes is widely used in classifier design. Enlighted by the idea, this paper proposes a novel supervised dimensionality reduction method for tensor data based on local decision margin maximization. The proposed method seeks to preserve and protect the local discriminant information of the original data in the low-dimensional data space. Firstly, we depart the original tensor dataset into overlapped localities with discriminant information. Then, we extract the similarity and anti-similarity coefficients of each high-dimensional locality and preserve these coefficients in the embedding data space via the multilinear projection scheme. Under the combined effect of these coefficients, each dimension-reduced locality tends to be a convex set where strongly correlated intraclass points gather. Simultaneously, the local decision margin, which is defined as the shortest distance from the boundary of each locality to the nearest point of each side, will be maximized. Therefore, the local discriminant structure of the original data could be well maintained in the low-dimensional data space. Moreover, a simple iterative scheme is proposed to solve the final optimization problem. Finally, the experiment results on 6 real-world datasets demonstrate the effectiveness of the proposed method.

Root cause analysis approach based on reverse cascading decomposition in QFD and fuzzy weight ARM for quality accidents

Quality accidents (QAs) of high frequencies in various fields have caused large economic and reputational losses to manufacturers, and identification of the root causes of vicious QAs is a top priority and a major challenge for manufacturers. Especially in the era of big data, the large number of data could be collected from the product life cycle easily, these high-dimensional big data always bear so many un-correlation noise information, which has caused serious problem. The accurate and heuristic root cause analysis for QAs is an important and challenging task in exploring this mechanism due to the fuzzy and vague nature of the collected big quality data. Thus, in this study, a heuristic root cause identification solution based on the fuzzy weighted association rule mining (FWARM) for QAs is proposed. First, the formation mechanism of QAs and big quality accident data is expounded, and a big data driven root cause analysis framework of QAs is presented with the aid of reverse cascading decomposition in Quality Function Deployment (QFD). Second, principal component analysis (PCA) is adopted to eliminate redundancy and reduce data dimension of original process feature parameters from raw data in low-dimensional space so that the key variables as the potential root cause candidates can be extracted. Third, considering the fuzzy mechanism and vague nature of big data, a heuristic root cause identification approach based on FWARM is established, and the weight of nodes on the accident-relevance tree is computed by fuzzy weight coefficient. Finally, the proposed approach is verified with a case study of a quality accident analysis of a washing machine. Results shows that the proposed approach is conducive to heuristically identify the root causes of QAs in the context of big data.

Toward Making Unsupervised Graph Hashing Discriminative

Recently, hashing has attracted much attention in visual information retrieval due to its low storage cost and fast query speed. The goal of hashing is to map original high-dimensional data into a low-dimensional binary-code space where the similar data points are assigned similar hash codes and dissimilar points are far away from each other. Existing unsupervised hashing methods mainly focus on recovering the pairwise similarity of the original data in hash space, but do not take specific measures to make the generated binary codes to be discriminative. To address this problem, this paper proposes a novel unsupervised hashing method, named “Discriminative Unsupervised Graph Hashing” (DUGH), which takes both similarity and dissimilarity of original data into consideration to learn discriminative binary codes. In particular, a probabilistic model is utilized to learn the encoding of original data in low-dimensional space, which models the original neighbor structure through both positive and negative edges in the KNN graph and then maximizes the likelihood of observing these edges. To efficiently and accurately measure the neighbor structure for large-scale datasets, we propose an effective KNN graph construction algorithm based on the random projection tree and neighbor exploring techniques. The experimental results on one synthetic dataset and four typical real-world image datasets demonstrate that the proposed method significantly outperforms the state-of-the-art unsupervised hashing methods.

Projection Analysis Optimization for Human Transition Motion Estimation

It is a difficult task to estimate the human transition motion without the specialized software. The 3-dimensional (3D) human motion animation is widely used in video game, movie, and so on. When making the animation, human transition motion is necessary. If there is a method that can generate the transition motion, the making time will cost less and the working efficiency will be improved. Thus a new method called latent space optimization based on projection analysis (LSOPA) is proposed to estimate the human transition motion. LSOPA is carried out under the assistance of Gaussian process dynamical models (GPDM); it builds the object function to optimize the data in the low dimensional (LD) space, and the optimized data in LD space will be obtained to generate the human transition motion. The LSOPA can make the GPDM learn the high dimensional (HD) data to estimate the needed transition motion. The excellent performance of LSOPA will be tested by the experiments.

A New Method for Emulating Self-Organizing Maps for Visualization of Datasets

Several time-critical problems relying on large amount of data, e.g., business trends, disaster response and disease outbreak, require cost-effective, timely and accurate data summary and visualization, in order to come up with an efficient and effective decision. Self-organizing map (SOM) is a very effective data clustering and visualization tool as it provides intuitive display of data in lower-dimensional space. However, with [Formula: see text] complexity, SOM becomes inappropriate for large datasets. In this paper, we propose a force-directed visualization method that emulates SOMs capability to display the data clusters with [Formula: see text] complexity. The main idea is to perform a force-directed fine-tuning of the 2D representation of data. To demonstrate the efficiency and the vast potential of the proposed method as a fast visualization tool, the methodology is used to do a 2D-projection of the MNIST handwritten digits dataset.

Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering

Abstract Non-negative matrix factorization (NMF) can map high-dimensional data into a low-dimensional data space. Feature selection can eliminate the redundant and irrelevant features from the alternative features. In this paper, we propose a feature selection based dual-graph sparse non-negative matrix factorization (DSNMF) which can find an appropriate low dimensional representation of data by NMF and then select more discriminative features to further reduce the dimension of the low dimensional space by feature selection rather than reduce the dimension by only NMF or feature selection in many previous methods. DSNMF combines dual-graph model with non-negative matrix factorization, which can not only simultaneously preserve the geometric structures in both the data space and the feature space, but also make the two non-negative matrix factors update iteratively and interactively. In addition, DSNMF exerts L2,1-norm constraint on the non-negative matrix factor of the feature space to make full use of the sparse self-representation information. What's more, we propose a new local discriminative feature selection clustering called feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering (DSNMF-LDC) whose clustering effects are better. We give the objective function, the iterative updating rules and the convergence proof. Our empirical study shows that DSNMF-LDC is robust and excellent in comparison to 9 feature selection algorithms and 7 clustering algorithms in clustering accuracy (ACC) and normalized mutual information (NMI).

Gene selection for microarray data classification via subspace learning and manifold regularization.

With the rapid development of DNA microarray technology, large amount of genomic data has been generated. Classification of these microarray data is a challenge task since gene expression data are often with thousands of genes but a small number of samples. In this paper, an effective gene selection method is proposed to select the best subset of genes for microarray data with the irrelevant and redundant genes removed. Compared with original data, the selected gene subset can benefit the classification task. We formulate the gene selection task as a manifold regularized subspace learning problem. In detail, a projection matrix is used to project the original high dimensional microarray data into a lower dimensional subspace, with the constraint that the original genes can be well represented by the selected genes. Meanwhile, the local manifold structure of original data is preserved by a Laplacian graph regularization term on the low-dimensional data space. The projection matrix can serve as an importance indicator of different genes. An iterative update algorithm is developed for solving the problem. Experimental results on six publicly available microarray datasets and one clinical dataset demonstrate that the proposed method performs better when compared with other state-of-the-art methods in terms of microarray data classification. Graphical Abstract The graphical abstract of this work.

Band selection for hyperspectral image classification with spatial–spectral regularized sparse graph

Sparsity preserving projection is a well-known dimensionality reduction method that preserves the sparse representation relationship among data in low-dimensional space, which is beneficial for classification. The idea of sparsity preserving is applied to band selection for hyperspectral classification. Considering the spatial distribution characteristic of hyperspectral image (HSI), a spatial–spectral regularized sparse graph (ssRSG), which could utilize the spatial–spectral information in HSI to promote the discriminability of extracted local structure, is proposed. For band selection, the L 2,1 norm is applied to restrain the projection matrix and make a few bands with high importance scores, which are computed by the contribution of bands in a projection matrix. According to the importance score, more important bands are selected. Two real hyperspectral images are used to validate the performance of the proposed method.

An application of PART to the Football Manager data for players clusters analyses to inform club team formation

We aim to show how a neural network based machine learning projective clustering algorithm, Projective Adaptive Resonance Theory (PART), can be effectively used to provide data-informed sports decisions. We illustrate this data-driven decision recommendation for AS Roma player market in the Summer 2018 season, using the two separate databases of fourty-seven attributes taken from Football Manager 2018 for each of the twenty-four soccer player, with the first including players of the AS Roma squad 2017-18, and the second consisting of all players linked with transfer moves to AS Roma. This is high dimensional data as players should be grouped only in terms of their performance with respect to a small subset of attributes. Projective clustering analyses provide a purely data-driven analysis to identify critical attributes and attribute characteristics for a group of players to form a natural cluster (in lower dimensional data space) in an unsupervised way. By merging the two databases, our unsupervised clustering analysis provides evidence-based recommendations about the club team formation, and in particular, the decision to buy and sell players within the same clusters, under different scenarios including financial constraints.

Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian

Non-negative matrix factorization (NMF) is a very effective method for high dimensional data analysis, which has been widely used in information retrieval, computer vision, and pattern recognition. NMF aims to find two non-negative matrices whose product approximates the original matrix well. It can capture the underlying structure of data in the low dimensional data space using its parts-based representations. However, NMF is actually an unsupervised method without making use of prior information of data. In this paper, we propose a novel pairwise constrained non-negative matrix factorization with graph Laplacian method, which not only utilizes the local structure of the data by graph Laplacian, but also incorporates pairwise constraints generated among all labeled data into NMF framework. More specifically, we expect that data points which have the same class label will have very similar representations in the low dimensional space as much as possible, while data points with different class labels will have dissimilar representations as much as possible. Consequently, all data points are represented with more discriminating power in the lower dimensional space. We compare our approach with other typical methods and experimental results for image clustering show that this novel algorithm achieves the state-of-the-art performance.

Extensional Neighborhood Preserving Embedding

In this paper, a modified construction method of neighbor-graph is proposed for Neighborhood Preserving Embedding (NPE). NPE aims to find the local structure of the data set manifold and it is a linear approximation to Locally Linear Embedding (LLE). Although NPE gets an optimal embedding which can preserve the neighborhood structure for the original data set, but the construction method of neighborgraph is not the best, which suffers from the following issue: the data set is vectorized to compute the k-nearest neighbor graph (adjacency graph), which leads to the lost of the correlative columns information. So we modify the neighbor-graph construction method to preserve the corresponding columns information. In our method, we can describe the intrinsic structure of original data in lowdimensional space much better. Extensive experiments are performed on the well-known face databases: AR, ORL and Yale face databases to test and evaluate our method’s performance. The result demonstrates the effectiveness of our new method.

Performance Analysis of Hybrid (supervised and unsupervised) method for multiclass data set

Due to the increasing demand for multivariate data analysis from the various application the dimensionality reduction becomes an important task to represent the data in low dimensional space for the robust data representation. In this paper, multivariate data analyzed by using a new approach SVM and ICA to enhance the classification accuracy in a way that data can be present in more condensed form. Traditional methods are classified into two types namely standalone and hybrid method. Standalone method uses either supervised or unsupervised approach, whereas hybrid method uses both approaches. This paper consists of SVM (support vector machine) as supervised and ICA (Independent component analysis) as a unsupervised approach for the improvement of the classification on the basis of dimensionality reduction. SVM uses SRM (structural risk minimization) principle which is very effective over ERM (empirical risk minimization) which minimizes an upper bound on the expected risk, as opposed to ERM that minimizes the error on the training data, whereas ICA uses maximum independence maximization to improve performance. The perpendicular or right angel projection is used to avoid the redundancy and to improve the dimensionality reduction. At last step a classification algorithm is used to classify the data samples and classification accuracy is measured. Experiments are performed for various two classes as well as multiclass dataset and performance of hybrid, standalone approaches are compared.

Graph-based local concept coordinate factorization

Ubiquitous data are increasingly expanding in large volumes due to human activities, and grouping them into appropriate clusters is an important and yet challenging problem. Existing matrix factorization techniques have shown their significant power in solving this problem, e.g., nonnegative matrix factorization, concept factorization. Recently, one state-of-the-art method called locality-constrained concept factorization is put forward, but its locality constraint does not well reveal the intrinsic data structure since it only requires the concept to be as close to the original data points as possible. To address this issue, we present a graph-based local concept coordinate factorization (GLCF) method, which respects the intrinsic structure of the data through manifold kernel learning in the warped Reproducing Kernel Hilbert Space. Besides, a generalized update algorithm is developed to handle data matrices containing both positive and negative entries. Since GLCF is essentially based on the local coordinate coding and concept factorization, it inherits many advantageous properties, such as the locality and sparsity of the data representation. Moreover, it can better encode the locally geometrical structure via graph Laplacian in the manifold adaptive kernel. Therefore, a more compact and better structured representation can be obtained in the low-dimensional data space. Extensive experiments on several image and gene expression databases suggest the superiority of the proposed method in comparison with some alternatives.