Structure Of High-dimensional Data Research Articles

Statistical Embedding: Beyond Principal Components

There has been an intense recent activity in embedding of very high-dimensional and nonlinear data structures, much of it in the data science and machine learning literature. We survey this activity in four parts. In the first part, we cover nonlinear methods such as principal curves, multidimensional scaling, local linear methods, ISOMAP, graph-based methods and diffusion mapping, kernel based methods and random projections. The second part is concerned with topological embedding methods, in particular mapping topological properties into persistence diagrams and the Mapper algorithm. Another type of data sets with a tremendous growth is very high-dimensional network data. The task considered in part three is how to embed such data in a vector space of moderate dimension to make the data amenable to traditional techniques such as cluster and classification techniques. Arguably, this is the part where the contrast between algorithmic machine learning methods and statistical modeling, represented by the so-called stochastic block model, is at its greatest. In the paper, we discuss the pros and cons for the two approaches. The final part of the survey deals with embedding in R2, that is, visualization. Three methods are presented: t-SNE, UMAP and LargeVis based on methods in parts one, two and three, respectively. The methods are illustrated and compared on two simulated data sets; one consisting of a triplet of noisy Ranunculoid curves, and one consisting of networks of increasing complexity generated with stochastic block models and with two types of nodes.

Stability and machine learning applications of persistent homology using the Delaunay-Rips complex

Persistent homology (PH) is a robust method to compute multi-dimensional geometric and topological features of a dataset. Because these features are often stable under certain perturbations of the underlying data, are often discriminating, and can be used for visualization of structure in high-dimensional data and in statistical and machine learning modeling, PH has attracted the interest of researchers across scientific disciplines and in many industry applications. However, computational costs may present challenges to effectively using PH in certain data contexts, and theoretical stability results may not hold in practice. In this paper, we define, implement, and investigate a simplicial complex construction for computing persistent homology of Euclidean point cloud data, which we call the Delaunay-Rips complex (DR). By only considering simplices that appear in the Delaunay triangulation of the point cloud and assigning the Vietoris-Rips weights to simplices, DR avoids potentially costly computations in the persistence calculations. We document and compare a Python implementation of DR with other simplicial complex constructions for generating persistence diagrams. By imposing sufficient conditions on point cloud data, we are able to theoretically justify the stability of the persistence diagrams produced using DR. When the Delaunay triangulation of the point cloud changes under perturbations of the points, we prove that DR-produced persistence diagrams exhibit instability. Since we cannot guarantee that real-world data will satisfy our stability conditions, we demonstrate the practical robustness of DR for persistent homology in comparison with other simplicial complexes in machine learning applications. We find in our experiments that using DR in an ML-TDA pipeline performs comparatively well as using other simplicial complex constructions.

Enhancing Feature Selection for Imbalanced Alzheimer’s Disease Brain MRI Images by Random Forest

Imbalanced learning problems often occur in application scenarios and are additionally an important research direction in the field of machine learning. Traditional classifiers are substantially less effective for datasets with an imbalanced distribution, especially for high-dimensional longitudinal data structures. In the medical field, the imbalance of data problem is more common, and correctly identifying samples of the minority class can obtain important information. Moreover, class imbalance in imbalanced AD (Alzheimer’s disease) data presents a significant challenge for machine learning algorithms that assume the data are evenly distributed within the classes. In this paper, we propose a random forest-based feature selection algorithm for imbalanced neuroimaging data classification. The algorithm employs random forest to evaluate the value of each feature and combines the correlation matrix to choose the optimal feature subset, which is applied to imbalanced MRI (magnetic resonance imaging) AD data to identify AD, MCI (mild cognitive impairment), and NC (normal individuals). In addition, we extract multiple features from AD images that can represent 2D and 3D brain information. The effectiveness of the proposed method is verified by the experimental evaluation using the public ADNI (Alzheimer’s neuroimaging initiative) dataset, and results demonstrate that the proposed method has a higher prediction accuracy and AUC (area under the receiver operating characteristic curve) value in NC-AD, MCI-AD, and NC-MCI group data, with the highest accuracy and AUC value for the NC-AD group data.

Unsupervised Subspace Learning With Flexible Neighboring.

Graph-based subspace learning has been widely used in various applications as the rapid growth of data dimension, while the graph is constructed by affinity matrix of input data. However, it is difficult for these subspace learning methods to preserve the intrinsic local structure of data with the high-dimensional noise. To address this problem, we proposed a novel unsupervised dimensionality reduction approach named unsupervised subspace learning with flexible neighboring (USFN). We learn a similarity graph by adaptive probabilistic neighborhood learning process to preserve the manifold structure of high-dimensional data. In addition, we utilize the flexible neighboring to learn projection and latent representation of manifold structure of high-dimensional data to remove the impact of noise. The adaptive similarity graph and latent representation are jointly learned by integrating adaptive probabilistic neighborhood learning and manifold residue term into a unified objection function. The experimental results on synthetic and real-world datasets demonstrate the performance of the proposed unsupervised subspace learning USFN method.

Frequency-domain physical constrained neural network for nonlinear system dynamic prediction

The current use of physical constraints as prior knowledge of neural network models is a new frontier for artificial intelligence to solve complex nonlinear system predictions. Aiming at the problems of limited feature expression and low time-domain computational efficiency of existing physical-constrained neural networks, a physical informed neural network (PINN) modeling approach in frequency domain for nonlinear system dynamic prediction is proposed. First, the frequency-domain physics-constrained neural network (FPNN) framework is constructed, which mainly consists of CliqueNet and Fourier spectral method (FSM). Then, the CliqueNet with convolution feature cyclic structure for high dimensional sequential data is designed. There are forward and backward connections between any two layers of the network to improve the cross-level information flow of the deep network. Finally, FSM is utilized to inform the neural network of physical states by learning specific initial conditions without training data. Simulation results show that FPNN can effectively achieve Kuramoto–Sivashinsky complex system state prediction, with higher computational efficiency and faster learning efficiency compared with SOTA approaches.

Target Parameter Estimation Algorithm Based on Real-Valued HOSVD for Bistatic FDA-MIMO Radar

Since there is a frequency offset between each adjacent antenna of FDA radar, there exists angle-range two-dimensional dependence in the transmitter. For bistatic FDA-multiple input multiple output (MIMO) radar, range-direction of departure (DOD)-direction of arrival (DOA) information is coupled in transmitting the steering vector. How to decouple the three information has become the focus of research. Aiming at the issue of target parameter estimation of bistatic FDA-MIMO radar, a real-valued parameter estimation algorithm based on high-order-singular value decomposition (HOSVD) is developed. Firstly, for decoupling DOD and range in transmitter, it is necessary to divide the transmitter into subarrays. Then, the forward–backward averaging and unitary transformation techniques are utilized to convert complex-valued data into real-valued data. The signal subspace is obtained by HOSVD, and the two-dimensional spatial spectral function is constructed. Secondly, the dimension of spatial spectrum is reduced by the Lagrange algorithm, so that it is only related to DOA, and the DOA estimation is obtained. Then the frequency increment between subarrays is used to decouple the DOD and range information, and eliminate the phase ambiguity at the same time. Finally, the DOD and range estimation automatically matched with DOA estimation are obtained. The proposed algorithm uses the multidimensional structure of high-dimensional data to promote performance. Meanwhile, the proposed real-valued tensor-based method can effectively cut down the computing time. Simulation results verify the high efficiency of the developed method.

Proteomics biomarker discovery for individualized prevention of familial pancreatic cancer using statistical learning.

The low five-year survival rate of pancreatic ductal adenocarcinoma (PDAC) and the low diagnostic rate of early-stage PDAC via imaging highlight the need to discover novel biomarkers and improve the current screening procedures for early diagnosis. Familial pancreatic cancer (FPC) describes the cases of PDAC that are present in two or more individuals within a circle of first-degree relatives. Using innovative high-throughput proteomics, we were able to quantify the protein profiles of individuals at risk from FPC families in different potential pre-cancer stages. However, the high-dimensional proteomics data structure challenges the use of traditional statistical analysis tools. Hence, we applied advanced statistical learning methods to enhance the analysis and improve the results' interpretability. We applied model-based gradient boosting and adaptive lasso to deal with the small, unbalanced study design via simultaneous variable selection and model fitting. In addition, we used stability selection to identify a stable subset of selected biomarkers and, as a result, obtain even more interpretable results. In each step, we compared the performance of the different analytical pipelines and validated our approaches via simulation scenarios. In the simulation study, model-based gradient boosting showed a more accurate prediction performance in the small, unbalanced, and high-dimensional datasets than adaptive lasso and could identify more relevant variables. Furthermore, using model-based gradient boosting, we discovered a subset of promising serum biomarkers that may potentially improve the current screening procedure of FPC. Advanced statistical learning methods helped us overcome the shortcomings of an unbalanced study design in a valuable clinical dataset. The discovered serum biomarkers provide us with a clear direction for further investigations and more precise clinical hypotheses regarding the development of FPC and optimal strategies for its early detection.

KODAMA exploratory analysis in metabolic phenotyping.

KODAMA is a valuable tool in metabolomics research to perform exploratory analysis. The advanced analytical technologies commonly used for metabolic phenotyping, mass spectrometry, and nuclear magnetic resonance spectroscopy push out a bunch of high-dimensional data. These complex datasets necessitate tailored statistical analysis able to highlight potentially interesting patterns from a noisy background. Hence, the visualization of metabolomics data for exploratory analysis revolves around dimensionality reduction. KODAMA excels at revealing local structures in high-dimensional data, such as metabolomics data. KODAMA has a high capacity to detect different underlying relationships in experimental datasets and correlate extracted features with accompanying metadata. Here, we describe the main application of KODAMA exploratory analysis in metabolomics research.

Long-Range Dependence Involutional Network for Logo Detection

Logo detection is one of the crucial branches in computer vision due to various real-world applications, such as automatic logo detection and recognition, intelligent transportation, and trademark infringement detection. Compared with traditional handcrafted-feature-based methods, deep learning-based convolutional neural networks (CNNs) can learn both low-level and high-level image features. Recent decades have witnessed the great feature representation capabilities of deep CNNs and their variants, which have been very good at discovering intricate structures in high-dimensional data and are thereby applicable to many domains including logo detection. However, logo detection remains challenging, as existing detection methods cannot solve well the problems of a multiscale and large aspect ratios. In this paper, we tackle these challenges by developing a novel long-range dependence involutional network (LDI-Net). Specifically, we designed a strategy that combines a new operator and a self-attention mechanism via rethinking the intrinsic principle of convolution called long-range dependence involution (LD involution) to alleviate the detection difficulties caused by large aspect ratios. We also introduce a multilevel representation neural architecture search (MRNAS) to detect multiscale logo objects by constructing a novel multipath topology. In addition, we implemented an adaptive RoI pooling module (ARM) to improve detection efficiency by addressing the problem of logo deformation. Comprehensive experiments on four benchmark logo datasets demonstrate the effectiveness and efficiency of the proposed approach.

Cglasso: An R Package for Conditional Graphical Lasso Inference with Censored and Missing Values

Sparse graphical models have revolutionized multivariate inference. With the advent of high-dimensional multivariate data in many applied fields, these methods are able to detect a much lower-dimensional structure, often represented via a sparse conditional independence graph. There have been numerous extensions of such methods in the past decade. Many practical applications have additional covariates or suffer from missing or censored data. Despite the development of these extensions of sparse inference methods for graphical models, there have been so far no implementations for, e.g., conditional graphical models. Here we present the general-purpose package cglasso for estimating sparse conditional Gaussian graphical models with potentially missing or censored data. The method employs an efficient expectation-maximization estimation of an ℓ₁ -penalized likelihood via a block-coordinate descent algorithm. The package has a user-friendly data manipulation interface. It estimates a solution path and includes various automatic selection algorithms for the two ℓ₁ tuning parameters, associated with the sparse precision matrix and sparse regression coefficients, respectively. The package pays particular attention to the visualization of the results, both by means of marginal tables and figures, and of the inferred conditional independence graphs. This package provides a unique and computational efficient implementation of a conditional Gaussian graphical model that is able to deal with the additional complications of missing and censored data. As such it constitutes an important contribution for empirical scientists wishing to detect sparse structures in high-dimensional data.

Information Retrieval With Chessboard-Shaped Topology for Hyperspectral Target Detection

Given a priori knowledge, hyperspectral target detection aims to locate objects of interest within specific scenes by utilizing differences in spectral characteristics among various land covers. However, for those traditional model-driven detectors with monotonic analytical mode, they perform mediocrely in the disassembly of hyperspectral image (HSI) data, failing to cope with real scenes with complexity. The discrepancy between fixed model assumptions and HSI data severely reduces detection effects, leading to the inability of such methods to mine deep-level features and adapt to the variability of imaging scenes. To overcome the limitations of traditional methods, we propose a chessboard-shaped topological framework for high-dimensional data structures to disassemble an HSI from both spatial and spectral dimensions adaptively. With hyperspectral target detection is refined into an information retrieval task in a topological space, a target detection method based on chessboard-shaped topology (CTTD) is proposed. In the topological space, latent and hidden data features of original images are presented in an intuitive way. Therefore, the differences in both spatial and spectral dimensions between the two classes of objects, namely target and background, are specifically amplified and exploited to perform the information retrieval task with superior performance. Extensive experimental results on benchmark HSI data sets demonstrate that CTTD can efficiently adapt to the variability of real scenes while extracting abundant and detailed information for accurate target localization. Moreover, both detection effects and computational efficiency exhibited by the proposed method provide a strong support for its popularization in practical applications.

MULTIVARIATE MULTILEVEL MODELLING TO ASSESS FACTORS AFFECTING THE QUALITY OF VOCATIONAL HIGH SCHOOLS IN SOUTH SULAWESI PROVINCE

This study analyzes the quality of Vocational High Schools (VHS), which have a hierarchical data structure and have more than one response variable. Data gathered for this study is from the Basic Education Data (DAPODIK) in the form of raw data variables of several variables that characterize the quality of VHS and other independent variables in South Sulawesi for four years (2018 to 2021) from the Ministry of Finance Republic of Indonesia (KEMENKEU), and Statistics Indonesia (BPS). The explanatory variable at the regency level consists of four years (2018 to 2021), a multi-year and high-dimensional data structure. Therefore, Principal Component Analysis (PCA) is used to overcome this. The modelling is done by using multivariate multilevel modelling (MVMM) on one-level and two-level structures. This study aims to model the average National Examination and Accreditation scores of Vocational High School in South Sulawesi using MVMM modelling that considers the regency/city level and identifies the factors that influence the average National Examination and Accreditation scores. The results showed that the two-level multivariate model with a random intercept as a hierarchical component was better than the one-level multilevel model based on a minor Deviance Information Criterion (DIC) value. Simultaneously, at the 5% level of significance, variables that contribute significantly to the quality of Vocational High Schools in South Sulawesi Province are produced. The variables that have a significant effect on the quality of Vocational High Schools at the school level are the ratio of the number of students/pupils per study group, the percentage of certified teachers to the number of teachers, the ratio of the number of students/pupils per number of toilets, the ratio of laboratory availability, and the ratio of the availability of supporting rooms. Meanwhile, at the regency level, it was found that the percentage of poverty and Gross Regional Domestic Product (GRDP) had a significant effect on the quality of Vocational High Schools.

Bayesian learners in gradient boosting for linear mixed models.

Selection of relevant fixed and random effects without prior choices made from possibly insufficient theory is important in mixed models. Inference with current boosting techniques suffers from biased estimates of random effects and the inflexibility of random effects selection. This paper proposes a new inference method "BayesBoost" that integrates a Bayesian learner into gradient boosting with simultaneous estimation and selection of fixed and random effects in linear mixed models. The method introduces a novel selection strategy for random effects, which allows for computationally fast selection of random slopes even in high-dimensional data structures. Additionally, the new method not only overcomes the shortcomings of Bayesian inference in giving precise and unambiguous guidelines for the selection of covariates by benefiting from boosting techniques, but also provides Bayesian ways to construct estimators for the precision of parameters such as variance components or credible intervals, which are not available in conventional boosting frameworks. The effectiveness of the new approach can be observed via simulation and in a real-world application.

Hyperspectral image classification using meta-heuristics and artificial neural network

Hyperspectral images usually comprise several continuous spectral bands that represent the category of similar objects or material within the captured scene. These high-dimensional data structures have a high level of correlation and possess unique information that can be used for precise image classification. The precise selection of useful features from these high dimensional band information is very important to reduce the challenge of hyper spectral image classification approaches. Nowadays, metaheuristic algorithms are immensely utilized as a promising tool for hyperspectral image classification. In the present research work, hyperspectral images are classified with the various combinations of meta-heuristic approaches and the neural network including the mostly used Cuckoo Search (CS) optimization algorithm to resolve the global optimization search problems considering the improvement needed in image classification. Further, the strength of CS is improved using the integration of the Genetic Algorithm (GA) fitness function within the CS. The feature selection is performed by the hybrid CS and GA algorithm and the optimized features are then fed to ANN for training and classification. The paper has shown a comparative analysis of various meta heuristics techniques with ANN on parameters like kappa coefficient, Class accuracy and overall Accuracy and the designed algorithms are tested on the Indian Pines dataset. The proposed CS and GA with ANN outperformed the two already existing works with an overall average accuracy of 97.30% and a kappa coefficient of 0.9760.

Unsupervised machine learning methods and emerging applications in healthcare.

Unsupervised machine learning methods are important analytical tools that can facilitate the analysis and interpretation of high-dimensional data. Unsupervised machine learning methods identify latent patterns and hidden structures in high-dimensional data and can help simplify complex datasets. This article provides an overview of key unsupervised machine learning techniques including K-means clustering, hierarchical clustering, principal component analysis, and factor analysis. With a deeper understanding of these analytical tools, unsupervised machine learning methods can be incorporated into health sciences research to identify novel risk factors, improve prevention strategies, and facilitate delivery of personalized therapies and targeted patient care.Level of evidence: I.

Shape-aware stochastic neighbor embedding for robust data visualisations

BackgroundThe t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm has emerged as one of the leading methods for visualising high-dimensional (HD) data in a wide variety of fields, especially for revealing cluster structure in HD single-cell transcriptomics data. However, t-SNE often fails to correctly represent hierarchical relationships between clusters and creates spurious patterns in the embedding. In this work we generalised t-SNE using shape-aware graph distances to mitigate some of the limitations of the t-SNE. Although many methods have been recently proposed to circumvent the shortcomings of t-SNE, notably Uniform manifold approximation (UMAP) and Potential of heat diffusion for affinity-based transition embedding (PHATE), we see a clear advantage of the proposed graph-based method.ResultsThe superior performance of the proposed method is first demonstrated on simulated data, where a significant improvement compared to t-SNE, UMAP and PHATE, based on quantitative validation indices, is observed when visualising imbalanced, nonlinear, continuous and hierarchically structured data. Thereafter the ability of the proposed method compared to the competing methods to create faithfully low-dimensional embeddings is shown on two real-world data sets, the single-cell transcriptomics data and the MNIST image data. In addition, the only hyper-parameter of the method can be automatically chosen in a data-driven way, which is consistently optimal across all test cases in this study.ConclusionsIn this work we show that the proposed shape-aware stochastic neighbor embedding method creates low-dimensional visualisations that robustly and accurately reveal key structures of high-dimensional data.

The Intrinsic Structure of High-Dimensional Data According to the Uniqueness of Constant Mean Curvature Hypersurfaces

In this paper, we study the intrinsic structures of high-dimensional data sets for analyzing their geometrical properties, where the core message of the high-dimensional data is hiding on some nonlinear manifolds. Using the manifold learning technique with a particular focus on the mean curvature, we develop new methods to investigate the uniqueness of constant mean curvature spacelike hypersurfaces in the Lorentzian warped product manifolds. Furthermore, we extend the uniqueness of stochastically complete hypersurfaces using the weak maximum principle. For the more general cases, we propose some non-existence results and a priori estimates for the constant higher-order mean curvature spacelike hypersurface.

Robust Covariance Matrix Estimation for High-Dimensional Compositional Data with Application to Sales Data Analysis

Compositional data arises in a wide variety of research areas when some form of standardization and composition is necessary. Estimating covariance matrices is of fundamental importance for high-dimensional compositional data analysis. However, existing methods require the restrictive Gaussian or sub-Gaussian assumption, which may not hold in practice. We propose a robust composition adjusted thresholding covariance procedure based on Huber-type M-estimation to estimate the sparse covariance structure of high-dimensional compositional data. We introduce a cross-validation procedure to choose the tuning parameters of the proposed method. Theoretically, by assuming a bounded fourth moment condition, we obtain the rates of convergence and signal recovery property for the proposed method and provide the theoretical guarantees for the cross-validation procedure under the high-dimensional setting. Numerically, we demonstrate the effectiveness of the proposed method in simulation studies and also a real application to sales data analysis.

Visualization of emergency department clinical data for interpretable patient phenotyping

Visual summarization of clinical data collected on patients contained within the electronic health record (EHR) may enable precise and rapid triage at the time of patient presentation to an emergency department (ED). The triage process is critical in the appropriate allocation of resources and in anticipating eventual patient disposition, typically admission to the hospital or discharge home. EHR data are high-dimensional and complex, but offer the opportunity to discover and characterize underlying data-driven patient phenotypes. Data-driven phenotypes are intended to relieve reliance on weak labels like diagnosis codes and to aid in identifying populations of existing patients that are most similar to a specific patient. These phenotypes will enable improved, personalized therapeutic decision making and prognostication. In this work, we focus on the challenge of two-dimensional patient projections. A low dimensional embedding offers visual interpretability lost in higher dimensions. While linear dimensionality reduction techniques such as principal component analysis are often used towards this aim, they are insufficient to describe the variance of patient data. This linear reduction does not account for higher order, non-linear interactions of variables. In this work, we employ the newly-described non-linear embedding technique called uniform manifold approximation and projection (UMAP). UMAP seeks to capture both local and global structures in high-dimensional data. We then use Gaussian mixture models to identify clusters in the embedded data and use the adjusted Rand index (ARI) to establish stability in the discovery of these clusters. This technique is applied to five common clinical chief complaints from a real-world ED EHR dataset, describing the emergent properties of discovered clusters. We observe clinically-relevant cluster attributes, suggesting that visual embeddings of EHR data using non-linear dimensionality reduction is a promising approach to reveal data-driven patient phenotypes. In the five chief complaints, we find between 2 and 6 clusters, with the peak mean pairwise ARI between subsequent training iterations to range from 0.35 to 0.74.

Large-Scale Subspace Clustering by Independent Distributed and Parallel Coding.

Subspace clustering is a popular method to discover underlying low-dimensional structures of high-dimensional multimedia data (e.g., images, videos, and texts). In this article, we consider a large-scale subspace clustering (LS2C) problem, that is, partitioning million data points with a millon dimensions. To address this, we explore an independent distributed and parallel framework by dividing big data/variable matrices and regularization by both columns and rows. Specifically, LS2C is independently decomposed into many subproblems by distributing those matrices into different machines by columns since the regularization of the code matrix is equal to a sum of that of its submatrices (e.g., square-of-Frobenius/ l1 -norm). Consensus optimization is designed to solve these subproblems in a parallel way for saving communication costs. Moreover, we provide theoretical guarantees that LS2C can recover consensus subspace representations of high-dimensional data points under broad conditions. Compared with the state-of-the-art LS2C methods, our approach achieves better clustering results in public datasets, including a million images and videos.