Structure Of High-dimensional Data Research Articles

Calibration Transfer for Near-Infrared (NIR) Spectroscopy Based on Neighborhood Preserving Embedding

Calibration transfer is a subtle issue in the practical application of near-infrared (NIR) spectroscopy technique. In this paper, a novel method to calibration transfer based on neighborhood preserving embedding (CTNPE) for correcting spectral differences was proposed. As a manifold learning method, neighborhood preserving embedding (NPE) can not only capture the nonlinear manifold structure, but also retain the linearity and show good generalization ability. Since this approach can reveal low dimensional manifold structure in high dimensional spectroscopic data, it is beneficial to construct the transform relationship between source and target spectra. The performance of CTNPE was assessed and compared to that of piecewise direct standardization (PDS) and other four dimensionality reduction-based methods, including transfer based on target factor analysis (TTFA), spectral space transformation (SST), calibration transfer based on canonical correlation analysis (CTCCA) and based on independent component analysis (CTICA), in two real cases. The results indicated that CTNPE was able to successfully transfer spectra between instruments and samples in different physical states. Furthermore, CTNPE provided lower prediction errors than PDS, TTFA, CTCCA, SST, CTICA and direct prediction without a transfer function. Therefore, the comprehensive investigation carried out in the presented work demonstrates that CTNPE is a promising calibration transfer method for NIR, especially for correcting the variations for samples in different physical states.

Detecting changes in the second moment structure of high-dimensional sensor-type data in a K-sample setting

The K sample problem for high-dimensional vector time series is studied, especially focusing on sensor data streams, in order to analyze the second moment structure and detect changes across samples and/or across variables cumulated sum (CUSUM) statistics of bilinear forms of the sample covariance matrix. In this model, K independent vector time series are observed over a time span which may correspond to K sensors (locations) yielding d-dimensional data as well as K locations where d sensors emit univariate data. Unequal sample sizes are considered as arising when the sampling rate of the sensors differs. We provide large-sample approximations and two related change point statistics, a sum of squares and a pooled variance statistic. The resulting procedures are investigated by simulations and illustrated by analyzing a real data set.

Collaborative representation‐based locality preserving projections for image classification

Graph embedding has attracted much more research interests in dimensionality reduction. In this study, based on collaborative representation and graph embedding, the authors propose a new linear dimensionality reduction method called collaborative representation-based locality preserving projection (CRLPP). In the CRLPP, they assume that the similar samples should have similar reconstructions by collaborative representation and the similar reconstructions should also have the similar low-dimensional representations in the projected subspace. CRLPP first reconstructs each training sample using the collaborative representation of the other remaining training samples, and then designs the graph construction of all training samples, finally establishes the objective function of graph embedding using the collaborative reconstructions and the constructed graph. The proposed CRLPP can well preserve the intrinsic geometrical and discriminant structures of high-dimensional data in low-dimensional subspace. The effectiveness of the proposed is verified on several image datasets. The experimental results show that the proposed method outperforms the state-of-art dimensionality reduction.

Deep Double Supervised Embedding Neural Network Enhancing Class Separation for Visual High-Dimensional Industrial Process Monitoring

Visual process monitoring is the application of a visualization method to map the real-time operating information of an industrial process to a 2-D map, followed by process monitoring. However, owing to the complexity of industrial production processes and the complex correlations among industrial process variables, the structure and distribution of high-dimensional industrial data are very complicated. Therefore, a general visualization method cannot effectively separate the different fault data in a 2-D map for process monitoring. Accordingly, in this article, a deep double supervised embedding neural network (DDSE) is proposed for visualizing high-dimensional industrial data. The DDSE consists of two supervised deep neural networks: a deep class centres uniform distribution neural network (DCCUD), and a deep supervised t-stochastic neighbor embedding neural network (DSSNE). The DCCUD maps the high-dimensional industrial data to a new feature space in which the class centres obey a uniform distribution, promoting a good and separable situation for subsequent visualization procedures. The DSSNE then maps these high-dimensional features into a 2-D space. The training of the DDSE can be conducted through pre-training and fine-tuning. A proposed visual process monitoring approach combines the DDSE with the local outlier factor and k-nearest neighbor approaches. The proposed approach is tested on a Tennessee Eastman process, and the results illustrate that the proposed approach outperforms traditional methods in terms of visualization and visual process monitoring.

Stereo3D: using stereo images to enrich 3D visualization.

Visualization in 3D space is a standard but critical process for examining the complex structure of high-dimensional data. Stereoscopic imaging technology can be adopted to enhance 3D representation of many complex data, especially those consisting of points and lines. We illustrate the simple steps that are involved and strongly recommend others to implement it in designing visualization software. To facilitate its application, we created a new software that can convert a regular 3D scatterplot or network figure to a pair of stereo images. Stereo3D is freely available as an open source R package released under an MIT license at https://github.com/bioinfoDZ/Stereo3D. Others can integrate the codes and implement the method in academic software. deyou.zheng@einsteinmed.org. Supplementary data are available at Bioinformatics online.

Utilizing radar graphs in the visualization of simulation and estimation results in network meta-analysis.

Traditional visualization in meta-analysis uses forest plots to illustrate the combined treatment effect, along with the respective results from primary trials. While the purpose of visualization is clear in the pairwise setting, additional treatments broaden the focus and extend the results to be illustrated in network meta-analysis. The complexity increases further in situations where all potential contrasts in the network are compared to a predefined fixed value of interest, such as the 95% coverage evaluated against the nominal value of 95% in simulation studies. We propose utilizing radar graphs to illustrate results from network meta-analysis in cases where the interest lies in the comparison of estimated results (after fitting a network meta-analysis in a specific data set) or a performance measure (simulation study) to a pre-defined fixed reference value. Accounting for the complex high-dimensional data structure, the general picture of the full network is captured at once without increasing the space needed for visualization. Especially in large simulation studies, where multiple scenarios need to be visually combined to gain an overview on different scenarios, this type of illustration facilitates the discussion of results. Further properties, such as the expected variation due to the Monte-Carlo error or the differentiation between directly and indirectly estimated treatment contrasts in simulation studies, as well as the indication of well-connected and sparsely connected treatments in an applied network meta-analysis, can additionally be included in the visualization. While we used the radar-graph mainly for a simulation study, other applications are suitable whenever relative contributions of treatment (contrasts) are of interest.

Automated Detection of Surface Cracks and Numerical Correlation with Thermal-Structural Behaviors of Fire Damaged Concrete Beams

There are two specific aims in this study; first is to develop and validate an automated crack detection technique for the fire damaged beam. Second is to investigate whether the detected crack information and thermal-structural behaviors can be numerically related. To fulfill the aims, fire tests and residual strength tests are conducted on RC beams having different fire exposure time periods and sustained load levels. To detect the automated cracks, surface images of the fire damaged beam surfaces are taken with digital cameras and an automatic crack detection method is developed using a convolutional neural network (CNN) which is a deep learning technique primarily used for analyzing intricate structures of high-dimensional data [such as high definition (HD) images and videos]. The quantity of cracks detected using the proposed CNN changes depending on the test variables, and the changing trends are similar to those of the crack lengths obtained from the optical observation. Additionally, it is found that the quantity of the automatically detected cracks is numerically related to the temperatures inside the beams as well as the stiffnesses obtained from the residual strength tests.

Local Manifold-Based Sparse Discriminant Learning for Feature Extraction of Hyperspectral Image.

Hyperspectral image (HSI) generally contains a complex manifold structure and strong sparse correlation in its nonlinear high-dimensional data space. However, the existing manifold learning and sparse learning methods usually consider the manifold structure and sparse relationship separately rather than combining manifold and sparse properties to discover the intrinsic information in the original data. To simultaneously reveal the complex sparse relation and manifold structure of HSI, a novel feature extraction (FE) method, called local manifold-based sparse discriminant learning (LMSDL), has been proposed on the basis of manifold learning and sparse representation (SR). The LMSDL method first designs a new sparse optimization model called local manifold-based SR (LMSR) to reveal the local manifold-based sparse structure of data. Then, two geometrical sparse graphs are constructed to represent the discriminant relationship between samples and the geometrical and sparse neighbors. An objective function is constructed via geometrical sparse graphs and reconstruction points to learn a projection matrix for FE. The LMSDL effectively reveals the complex sparse relation and manifold structure in high-dimensional data, and it enhances the representation ability of extracted features for HSI classification significantly. The experimental results on the three real HSI datasets show that the proposed LMSDL algorithm possesses better performance in comparison with some state-of-the-art FE methods.

Vibration‐based detection and classification of structural changes using principal component analysis and ‐distributed stochastic neighbor embedding

This paper describes a structural health monitoring strategy to detect and classify structural changes in structures that can be equipped with sensors. The proposed approach is based on the t-distributed stochastic neighbor embedding (t-SNE), a nonlinear technique that can represent the local structure of high-dimensional data collected from multiple sensors in a plane or spatial representation. We propose the following basic steps for the detection and classification. First, the raw data are preprocessed: We scale the data using the mean-centered group scaling and apply principal component analysis to reduce the dimensionality of the scaled data. Second, t-SNE is applied to represent the scaled and reduced data as points in a plane, defining a cluster for each structural state. Finally, the current structure to be diagnosed is associated with a cluster (or structural state) using three different strategies: (a) the smallest point-centroid distance; (b) the majority voting; and (c) the sum of the inverse distances. The combination of t-SNE with our preprocessing and the three proposed classification strategies signif- icantly improves the quality of the clusters that represent different structural states. We evaluate the performance of our method using experimental data from an aluminum plate instrumented with piezoelectric transducers. Results are presented in the time domain, and they reveal the high classification accuracy and strong performance of this method, with a percentage of correct decisions close to 100% in several scenarios.

Swarm intelligence for self-organized clustering

Algorithms implementing populations of agents which interact with one another and sense their environment may exhibit emergent behavior such as self-organization and swarm intelligence. Here a swarm system, called Databionic swarm (DBS), is introduced which is able to adapt itself to structures of high-dimensional data characterized by distance and/or density-based structures in the data space. By exploiting the interrelations of swarm intelligence, self-organization and emergence, DBS serves as an alternative approach to the optimization of a global objective function in the task of clustering. The swarm omits the usage of a global objective function and is parameter-free because it searches for the Nash equilibrium during its annealing process.To our knowledge, DBS is the first swarm combining these approaches. Its clustering can outperform common clustering methods such as K-means, PAM, single linkage, spectral clustering, model-based clustering, and Ward, if no prior knowledge about the data is available. A central problem in clustering is the correct estimation of the number of clusters. This is addressed by a DBS visualization called topographic map which allows assessing the number of clusters. It is known that all clustering algorithms construct clusters, irrespective of the data set contains clusters or not. In contrast to most other clustering algorithms, the topographic map identifies, that clustering of the data is meaningless if the data contains no (natural) clusters. The performance of DBS is demonstrated on a set of benchmark data, which are constructed to pose difficult clustering problems and in two real-world applications.

Community Detection in Complex Networks Using Nonnegative Matrix Factorization and Density-Based Clustering Algorithm

Community detection is a critical issue in the field of complex networks. Capable of extracting inherent patterns and structures in high dimensional data, the non-negative matrix factorization (NMF) method has become one of the hottest research topics in community detection recently. However, this method has a significant drawback; most community detection methods using NMF require the number of communities to be preassigned or determined by searching for the best community structure among all candidates. To address the problem, in this paper, we use an improved density peak clustering to obtain the number of cores as the pre-defined parameter of nonnegative matrix factorization. Then we adopt nonnegative double singular value decomposition initialization which can rapidly reduce the approximation error of nonnegative matrix factorization. Finally, we compare and analyze the performance of different algorithms on artificial networks and real-world networks. Experimental results indicate that the proposed method is superior to the state-of-the-art methods.

On function-on-function regression: partial least squares approach

Functional data analysis tools, such as function-on-function regression models, have received considerable attention in various scientific fields because of their observed high-dimensional and complex data structures. Several statistical procedures, including least squares, maximum likelihood, and maximum penalized likelihood, have been proposed to estimate such function-on-function regression models. However, these estimation techniques produce unstable estimates in the case of degenerate functional data or are computationally intensive. To overcome these issues, we proposed a partial least squares approach to estimate the model parameters in the function-on-function regression model. In the proposed method, the B-spline basis functions are utilized to convert discretely observed data into their functional forms. Generalized cross-validation is used to control the degrees of roughness. The finite-sample performance of the proposed method was evaluated using several Monte-Carlo simulations and an empirical data analysis. The results reveal that the proposed method competes favorably with existing estimation techniques and some other available function-on-function regression models, with significantly shorter computational time.

Uncovering High-dimensional Structures of Projections from Dimensionality Reduction Methods

Projections are conventional methods of dimensionality reduction for information visualization used to transform high-dimensional data into low dimensional space. If the projection method restricts the output space to two dimensions, the result is a scatter plot. The goal of this scatter plot is to visualize the relative relationships between high-dimensional data points that build up distance and density-based structures. However, the Johnson–Lindenstrauss lemma states that the two-dimensional similarities in the scatter plot cannot coercively represent high-dimensional structures. Here, a simplified emergent self-organizing map uses the projected points of such a scatter plot in combination with the dataset in order to compute the generalized U-matrix. The generalized U-matrix defines the visualization of a topographic map depicting the misrepresentations of projected points with regards to a given dimensionality reduction method and the dataset.•The topographic map provides accurate information about the high-dimensional distance and density based structures of high-dimensional data if an appropriate dimensionality reduction method is selected.•The topographic map can uncover the absence of distance-based structures.•The topographic map reveals the number of clusters in a dataset as the number of valleys.

Interpretable Recommender System With Heterogeneous Information: A Geometric Deep Learning Perspective

Recommender systems (RS) are ubiquitous in digital space. This paper develops a deep learning-based approach to address three practical challenges in RS: complex structures of high-dimensional data, noise in relational information, and the black-box nature of machine learning algorithms. Our method—Multi-GraphGraph Attention Network (MG-GAT)—learns latent user and business representations by aggregating a diverse set of information from neighbors of each user (business) on a neighbor importance graph. MG-GAT out-performs state-of-the-art deep learning models in the recommendation task using two large-scale datasets collected from Yelp and four other standard datasets in RS. The improved performance highlights MG-GAT’s advantage in incorporating multi-modal features in a principled manner. The features importance, neighbor importance graph, and latent representations reveal business insights on predictive features and explainable characteristics of business and users. Moreover, the learned neighbor importance graph can be used in a variety of management applications, such as targeting customers, promoting new businesses, and designing information acquisition strategies. Our paper presents a quintessential big data application of deep learning models in management while providing interpretability essential for real-world decision-making.

Improved Low-Rank Filtering of MR Spectroscopic Imaging Data With Pre-Learnt Subspace and Spatial Constraints.

To investigate the use of pre-learnt subspace and spatial constraints for denoising magnetic resonance spectroscopic imaging (MRSI) data. We exploit the partial separability or subspace structures of high-dimensional MRSI data for denoising. More specifically, we incorporate a subspace model with pre-learnt spectral basis into the low-rank approximation (LORA) method. Spectral basis is determined based on empirical prior distributions of the spectral parameters variations learnt from auxiliary training data; spatial priors are also incorporated as is done in LORA to further improve denoising performance. The effects of the explicit subspace and spatial constraints in reducing estimation bias and variance have been analyzed using Cramér-Rao Lower bound analysis, Monte-Carlo study, and experimental study. The denoising effectiveness of LORA can be significantly improved by incorporating pre-learnt spectral basis and spatial priors into LORA. This study provides an effective method for denoising MRSI data along with comprehensive analyses of its performance. The proposed method is expected to be useful for a wide range of studies using MRSI.

Visualizing structure and transitions in high-dimensional biological data.

The high-dimensional data created by high-throughput technologies require visualization tools that reveal data structure and patterns in an intuitive form. We present PHATE, a visualization method that captures both local and global nonlinear structure using an information-geometric distance between data points. We compare PHATE to other tools on a variety of artificial and biological datasets, and find that it consistently preserves a range of patterns in data, including continual progressions, branches and clusters, better than other tools. We define a manifold preservation metric, which we call denoised embedding manifold preservation (DEMaP), and show that PHATE produces lower-dimensional embeddings that are quantitatively better denoised as compared to existing visualization methods. An analysis of a newly generated single-cell RNA sequencing dataset on human germ-layer differentiation demonstrates how PHATE reveals unique biological insight into the main developmental branches, including identification of three previously undescribed subpopulations. We also show that PHATE is applicable to a wide variety of data types, including mass cytometry, single-cell RNA sequencing, Hi-C and gut microbiome data.

The art of using t-SNE for single-cell transcriptomics

Single-cell transcriptomics yields ever growing data sets containing RNA expression levels for thousands of genes from up to millions of cells. Common data analysis pipelines include a dimensionality reduction step for visualising the data in two dimensions, most frequently performed using t-distributed stochastic neighbour embedding (t-SNE). It excels at revealing local structure in high-dimensional data, but naive applications often suffer from severe shortcomings, e.g. the global structure of the data is not represented accurately. Here we describe how to circumvent such pitfalls, and develop a protocol for creating more faithful t-SNE visualisations. It includes PCA initialisation, a high learning rate, and multi-scale similarity kernels; for very large data sets, we additionally use exaggeration and downsampling-based initialisation. We use published single-cell RNA-seq data sets to demonstrate that this protocol yields superior results compared to the naive application of t-SNE.

A Frequency-Based Approach for the Detection and Classification of Structural Changes Using t-SNE †.

This work presents a structural health monitoring (SHM) approach for the detection and classification of structural changes. The proposed strategy is based on t-distributed stochastic neighbor embedding (t-SNE), a nonlinear procedure that is able to represent the local structure of high-dimensional data in a low-dimensional space. The steps of the detection and classification procedure are: (i) the data collected are scaled using mean-centered group scaling (MCGS); (ii) then principal component analysis (PCA) is applied to reduce the dimensionality of the data set; (iii) t-SNE is applied to represent the scaled and reduced data as points in a plane defining as many clusters as different structural states; and (iv) the current structure to be diagnosed will be associated with a cluster or structural state based on three strategies: (a) the smallest point-centroid distance; (b) majority voting; and (c) the sum of the inverse distances. The combination of PCA and t-SNE improves the quality of the clusters related to the structural states. The method is evaluated using experimental data from an aluminum plate with four piezoelectric transducers (PZTs). Results are illustrated in frequency domain, and they manifest the high classification accuracy and the strong performance of this method.

Robust subspace clustering based on non-convex low-rank approximation and adaptive kernel

As a relatively advanced method, the low-rank kernel space clustering method shows good performance in dealing with nonlinear structure of high-dimensional data. Unfortunately, this method is sensitive to large corruptions and doesn’t balance the contribution of all singular values. To solve the above problems, the low-rank kernel method is modified, and a robust subspace clustering method (LAKRSC) based on non-convex low-rank approximation and adaptive kernel is proposed. In our model, the weighted Schatten p-norm is introduced to balance the importance of different singular values, which can more accurately approximate the rank function and be more flexible in practical applications. Therefore, applying weighted Schatten p-norm to adaptive kernel can approximate the original low rank hypothesis better when the data is mapped into the feature space. In addition, our model uses correntropy to handle complex noise which enhances the robustness of the model. A new algorithm HQ&ADMM, combined by Half-Quadratic technique (HQ) and ADMM, is studied to solve our model. Experiments on four real-world datasets show that the clustering performance of LAKRSC is significantly better than that of several more advanced methods.

Enhancing Personalized Recommendation by Implicit Preference Communities Modeling

Recommender systems aim to capture user preferences and provide accurate recommendations to users accordingly. For each user, there usually exist others with similar preferences, and a collection of users may also have similar preferences with each other, thus forming a community. However, such communities may not necessarily be explicitly given, and the users inside the same communities may not know each other; they are formally defined and named Implicit Preference Communities (IPCs) in this article. By enriching user preferences with the information of other users in the communities, the performance of recommender systems can also be enhanced. Historical explicit ratings are a good resource to construct the IPCs of users but is usually sparse. Meanwhile, user preferences are easily affected by their social connections, which can be jointly used for IPC modeling with the ratings. However, this imposes two challenges for model design. First, the rating and social domains are heterogeneous; thus, it is challenging to coordinate social information and rating behaviors for a same learning task. Therefore, transfer learning is a good strategy for IPC modeling. Second, the communities are not explicitly labeled, and existing supervised learning approaches do not fit the requirement of IPC modeling. As co-clustering is an effective unsupervised learning approach for discovering block structures in high-dimensional data, it is a cornerstone for discovering the structure of IPCs. In this article, we propose a recommendation model with Implicit Preference Communities from user ratings and social connections. To tackle the unsupervised learning limitation, we design a Bayesian probabilistic graphical model to capture the IPC structure for recommendation. Meanwhile, following the spirit of transfer learning, both rating behaviors and social connections are introduced into the model by parameter sharing. Moreover, Gibbs sampling-based algorithms are proposed for parameter inferences of the models. Furthermore, to meet the need for online scenarios when the data arrive sequentially as a stream, a novel online sampling-based parameter inference algorithm for recommendation is proposed. To the best of our knowledge, this is the first attempt to propose and formally define the concept of IPC.