High-dimensional Data Analysis Research Articles

Nearly optimal Bayesian shrinkage for high-dimensional regression

During the past decade, shrinkage priors have received much attention in Bayesian analysis of high-dimensional data. This paper establishes the posterior consistency for high-dimensional linear regression with a class of shrinkage priors, which has a heavy and flat tail and allocates a sufficiently large probability mass in a very small neighborhood of zero. While enjoying its efficiency in posterior simulations, the shrinkage prior can lead to a nearly optimal posterior contraction rate and the variable selection consistency as the spike-and-slab prior. Our numerical results show that under the posterior consistency, Bayesian methods can yield much better results in variable selection than the regularization methods such as LASSO and SCAD. This paper also establishes a BvM-type result, which leads to a convenient way of uncertainty quantification for regression coefficient estimates.

Holistic Integration of Omics Tools for Precision Nutrition in Health and Disease.

The combination of multiple omics approaches has emerged as an innovative holistic scope to provide a more comprehensive view of the molecular and physiological events underlying human diseases (including obesity, dyslipidemias, fatty liver, insulin resistance, and inflammation), as well as for elucidating unique and specific metabolic phenotypes. These omics technologies include genomics (polymorphisms and other structural genetic variants), epigenomics (DNA methylation, histone modifications, long non-coding RNA, telomere length), metagenomics (gut microbiota composition, enterotypes), transcriptomics (RNA expression patterns), proteomics (protein quantities), and metabolomics (metabolite profiles), as well as interactions with dietary/nutritional factors. Although more evidence is still necessary, it is expected that the incorporation of integrative omics could be useful not only for risk prediction and early diagnosis but also for guiding tailored dietary treatments and prognosis schemes. Some challenges include ethical and regulatory issues, the lack of robust and reproducible results due to methodological aspects, the high cost of omics methodologies, and high-dimensional data analyses and interpretation. In this review, we provide examples of system biology studies using multi-omics methodologies to unravel novel insights into the mechanisms and pathways connecting the genotype to clinically relevant traits and therapy outcomes for precision nutrition applications in health and disease.

What drives performance in machine learning models for predicting heart failure outcome?

The development of acute heart failure (AHF) is a critical decision point in the natural history of the disease and carries a dismal prognosis. The lack of appropriate risk-stratification tools at hospital discharge of AHF patients significantly limits clinical ability to precisely tailor patient-specific therapeutic regimen at this pivotal juncture. Machine learning-based strategies may improve risk stratification by incorporating analysis of high-dimensional patient data with multiple covariates and novel prediction methodologies. In the current study, we aimed at evaluating the drivers for success in prediction models and establishing an institute-tailored artificial Intelligence-based prediction model for real-time decision support. We used a cohort of all 10 868 patients AHF patients admitted to a tertiary hospital during a 12 years period. A total of 372 covariates were collected from admission to the end of the hospitalization. We assessed model performance across two axes: (i) type of prediction method and (ii) type and number of covariates. The primary outcome was 1-year survival from hospital discharge. For the model-type axis, we experimented with seven different methods: logistic regression (LR) with either L1 or L2 regularization, random forest (RF), Cox proportional hazards model (Cox), extreme gradient boosting (XGBoost), a deep neural-net (NeuralNet) and an ensemble classifier of all the above methods. We were able to achieve an area under receiver operator curve (AUROC) prediction accuracy of more than 80% with most prediction models including L1/L2-LR (80.4%/80.3%), Cox (80.2%), XGBoost (80.5%), NeuralNet (80.4%). RF was inferior to other methods (78.8%), and the ensemble model was slightly superior (81.2%). The number of covariates was a significant modifier (P < 0.001) of prediction success, the use of multiplex-covariates preformed significantly better (AUROC 80.4% for L1-LR) compared with a set of known clinical covariates (AUROC 77.8%). Demographics followed by lab-tests and administrative data resulted in the largest gain in model performance. The choice of the predictive modelling method is secondary to the multiplicity and type of covariates for predicting AHF prognosis. The application of a structured data pre-processing combined with the use of multiple-covariates results in an accurate, institute-tailored risk prediction in AHF.

GeneNetTools: tests for Gaussian graphical models with shrinkage.

Gaussian graphical models (GGMs) are network representations of random variables (as nodes) and their partial correlations (as edges). GGMs overcome the challenges of high-dimensional data analysis by using shrinkage methodologies. Therefore, they have become useful to reconstruct gene regulatory networks from gene-expression profiles. However, it is often ignored that the partial correlations are 'shrunk' and that they cannot be compared/assessed directly. Therefore, accurate (differential) network analyses need to account for the number of variables, the sample size, and also the shrinkage value, otherwise, the analysis and its biological interpretation would turn biased. To date, there are no appropriate methods to account for these factors and address these issues. We derive the statistical properties of the partial correlation obtained with the Ledoit-Wolf shrinkage. Our result provides a toolbox for (differential) network analyses as (i) confidence intervals, (ii) a test for zero partial correlation (null-effects) and (iii) a test to compare partial correlations. Our novel (parametric) methods account for the number of variables, the sample size and the shrinkage values. Additionally, they are computationally fast, simple to implement and require only basic statistical knowledge. Our simulations show that the novel tests perform better than DiffNetFDR-a recently published alternative-in terms of the trade-off between true and false positives. The methods are demonstrated on synthetic data and two gene-expression datasets from Escherichia coli and Mus musculus. The R package with the methods and the R script with the analysis are available in https://github.com/V-Bernal/GeneNetTools. Supplementary data are available at Bioinformatics online.

High-dimensional randomization-based inference capitalizing on classical design and modern computing

A common complication that can arise with analyses of high-dimensional data is the repeated use of hypothesis tests. A second complication, especially with small samples, is the reliance on asymptotic p-values. Our proposed approach for addressing both complications uses a scientifically motivated scalar summary statistic, and although not entirely novel, seems rarely used. The method is illustrated using a crossover study of seventeen participants examining the effect of exposure to ozone versus clean air on the DNA methylome, where the multivariate outcome involved 484,531 genomic locations. Our proposed test yields a single null randomization distribution, and thus a single Fisher-exact p-value that is statistically valid whatever the structure of the data. However, the relevance and power of the resultant test requires the careful a priori selection of a single test statistic. The common practice using asymptotic p-values or meaningless thresholds for “significance” is inapposite in general.

Machine Learning Algorithms for understanding the determinants of under-five Mortality

BackgroundUnder-five mortality is a matter of serious concern for child health as well as the social development of any country. The paper aimed to find the accuracy of machine learning models in predicting under-five mortality and identify the most significant factors associated with under-five mortality.MethodThe data was taken from the National Family Health Survey (NFHS-IV) of Uttar Pradesh. First, we used multivariate logistic regression due to its capability for predicting the important factors, then we used machine learning techniques such as decision tree, random forest, Naïve Bayes, K- nearest neighbor (KNN), logistic regression, support vector machine (SVM), neural network, and ridge classifier. Each model’s accuracy was checked by a confusion matrix, accuracy, precision, recall, F1 score, Cohen’s Kappa, and area under the receiver operating characteristics curve (AUROC). Information gain rank was used to find the important factors for under-five mortality. Data analysis was performed using, STATA-16.0, Python 3.3, and IBM SPSS Statistics for Windows, Version 27.0 software.ResultBy applying the machine learning models, results showed that the neural network model was the best predictive model for under-five mortality when compared with other predictive models, with model accuracy of (95.29% to 95.96%), recall (71.51% to 81.03%), precision (36.64% to 51.83%), F1 score (50.46% to 62.68%), Cohen’s Kappa value (0.48 to 0.60), AUROC range (93.51% to 96.22%) and precision-recall curve range (99.52% to 99.73%). The neural network was the most efficient model, but logistic regression also shows well for predicting under-five mortality with accuracy (94% to 95%)., AUROC range (93.4% to 94.8%), and precision-recall curve (99.5% to 99.6%). The number of living children, survival time, wealth index, child size at birth, birth in the last five years, the total number of children ever born, mother’s education level, and birth order were identified as important factors influencing under-five mortality.ConclusionThe neural network model was a better predictive model compared to other machine learning models in predicting under-five mortality, but logistic regression analysis also shows good results. These models may be helpful for the analysis of high-dimensional data for health research.

Robust Covariance Matrix Estimation for High-Dimensional Compositional Data with Application to Sales Data Analysis

Compositional data arises in a wide variety of research areas when some form of standardization and composition is necessary. Estimating covariance matrices is of fundamental importance for high-dimensional compositional data analysis. However, existing methods require the restrictive Gaussian or sub-Gaussian assumption, which may not hold in practice. We propose a robust composition adjusted thresholding covariance procedure based on Huber-type M-estimation to estimate the sparse covariance structure of high-dimensional compositional data. We introduce a cross-validation procedure to choose the tuning parameters of the proposed method. Theoretically, by assuming a bounded fourth moment condition, we obtain the rates of convergence and signal recovery property for the proposed method and provide the theoretical guarantees for the cross-validation procedure under the high-dimensional setting. Numerically, we demonstrate the effectiveness of the proposed method in simulation studies and also a real application to sales data analysis.

Development Model of College English Education Service Talents Based on Nonlinear Random Matrix

The development of artificial intelligence makes people’s life and work easier and more effective, and computer-based online exams and marking not only improve students’ learning efficiency but also reduce the pressure of teachers’ marking work. For objective questions, marking has gone from manual marking to cursor reader marking to computerized character matching, and the correct rate of marking has soared to 100%; for subjective questions, foreign systems such as PEG and E-rater have been used, and domestic systems such as those using English large corpus similarity matching and those based on natural language understanding using intelligent algorithms have been used for marking. Most of these systems are based on some shallow linguistic features such as rules and LSAs for marking, and there is no deep perception of English language sense. Although the current intelligent marking systems have made a lot of achievements, they do not fundamentally solve the problem of the rationality of intelligent marking of subjective questions. In this article, we propose a regularized discriminant analysis algorithm with good estimation of the mean, and a dimensionality reduction algorithm for high-dimensional missing data by using the relevant research results of random matrix theory to address the problems of traditional machine learning methods in high-dimensional data analysis. Although the linear discriminant analysis algorithm performs well in solving many practical problems, it works poorly in dealing with high-dimensional data. The specific analysis is as follows: in terms of age characteristics, the mobile population under the age of 35 has a significant preference for urban consumer comfort, and it increases with age, peaking at the stage of 30–35 years old and then decreasing rapidly. For this reason, a regularized discriminant analysis algorithm based on random matrix theory is proposed. First, a good estimate of the high-dimensional covariance matrix is made by the nonlinear shrinkage method or the eigenvalue interception method, respectively; then, the estimated high-dimensional covariance matrix is used to calculate the discriminant function values and perform the classification. The classification experiments conducted on simulated and real datasets show that the proposed algorithm is not only more widely applicable but also has a high correct classification rate.

Network Comparison with Interpretable Contrastive Network Representation Learning.

Identifying unique characteristics in a network through comparison with another network is an essential network analysis task. For example, with networks of protein interactions obtained from normal and cancer tissues, we can discover unique types of interactions in cancer tissues. This analysis task could be greatly assisted by contrastive learning, which is an emerging analysis approach to discover salient patterns in one dataset relative to another. However, existing contrastive learning methods cannot be directly applied to networks as they are designed only for high-dimensional data analysis. To address this problem, we introduce a new analysis approach called contrastive network representation learning (cNRL). By integrating two machine learning schemes, network representation learning and contrastive learning, cNRL enables embedding of network nodes into a low-dimensional representation that reveals the uniqueness of one network compared to another. Within this approach, we also design a method, named i-cNRL, which offers interpretability in the learned results, allowing for understanding which specific patterns are only found in one network. We demonstrate the effectiveness of i-cNRL for network comparison with multiple network models and real-world datasets. Furthermore, we compare i-cNRL and other potential cNRL algorithm designs through quantitative and qualitative evaluations.

A simple tuning parameter selection method for high dimensional regression

The penalized regression is an important technique for high-dimensional data analysis, but penalized estimation method hinge on finding a suitable choice of tuning parameter. In this paper, a simple modified L curve method is proposed to select the tuning parameter for penalized estimation including Lasso, SCAD and MCP in linear regression models. Through data simulation and actual data analysis, we find that the modified L curve method can be simpler and more accurate than traditional tuning parameter selection schemes such as CV and BIC. Furthermore, the method is able to identify the true model consistently and has the less model error, especially for the cases where there is a high correlation between predictors.

Dimensionality reduction for visualizing high-dimensional biological data

High throughput technologies used in experimental biological sciences produce data with a vast number of variables at a rapid pace, making large volumes of high-dimensional data available. The exploratory analysis of such high-dimensional data can be aided by human interpretable low-dimensional visualizations. This work investigates how both discrete and continuous structures in biological data can be captured using the recently proposed dimensionality reduction method SONG, and compares the results with commonly used methods UMAP and PHATE. Using simulated and real-world datasets, we observe that SONG produces insightful visualizations by preserving various patterns, including discrete clusters, continuums, and branching structures in all considered datasets. More importantly, for datasets containing both discrete and continuous structures, SONG performs better at preserving both the structures compared to UMAP and PHATE. Furthermore, our quantitative evaluation of the three methods using downstream analysis validates the on par quality of the SONG’s low-dimensional embeddings compared to the other methods.

Graphia: A platform for the graph-based visualisation and analysis of high dimensional data.

Graphia is an open-source platform created for the graph-based analysis of the huge amounts of quantitative and qualitative data currently being generated from the study of genomes, genes, proteins metabolites and cells. Core to Graphia’s functionality is support for the calculation of correlation matrices from any tabular matrix of continuous or discrete values, whereupon the software is designed to rapidly visualise the often very large graphs that result in 2D or 3D space. Following graph construction, an extensive range of measurement algorithms, routines for graph transformation, and options for the visualisation of node and edge attributes are available, for graph exploration and analysis. Combined, these provide a powerful solution for the interpretation of high-dimensional data from many sources, or data already in the form of a network or equivalent adjacency matrix. Several use cases of Graphia are described, to showcase its wide range of applications in the analysis biological data. Graphia runs on all major desktop operating systems, is extensible through the deployment of plugins and is freely available to download from https://graphia.app/.

A global two-stage algorithm for non-convex penalized high-dimensional linear regression problems

By the asymptotic oracle property, non-convex penalties represented by minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD) have attracted much attentions in high-dimensional data analysis, and have been widely used in signal processing, image restoration, matrix estimation, etc. However, in view of their non-convex and non-smooth characteristics, they are computationally challenging. Almost all existing algorithms converge locally, and the proper selection of initial values is crucial. Therefore, in actual operation, they often combine a warm-starting technique to meet the rigid requirement that the initial value must be sufficiently close to the optimal solution of the corresponding problem. In this paper, based on the DC (difference of convex functions) property of MCP and SCAD penalties, we aim to design a global two-stage algorithm for the high-dimensional least squares linear regression problems. A key idea for making the proposed algorithm to be efficient is to use the primal dual active set with continuation (PDASC) method to solve the corresponding sub-problems. Theoretically, we not only prove the global convergence of the proposed algorithm, but also verify that the generated iterative sequence converges to a d-stationary point. In terms of computational performance, the abundant research of simulation and real data show that the algorithm in this paper is superior to the latest SSN method and the classic coordinate descent (CD) algorithm for solving non-convex penalized high-dimensional linear regression problems.

Preserved anatomical bypasses predict variance in language functions after stroke

The severity of post-stroke aphasia is related to damage to white matter connections. However, neural signaling can route not only through direct connections, but also along multi-step network paths. When brain networks are damaged by stroke, paths can bypass around the damage to restore communication. The shortest network paths between regions could be the most efficient routes for mediating bypasses. We examined how shortest-path bypasses after left hemisphere strokes were related to language performance. Regions within and outside of the canonical language network could be important in aphasia recovery. Therefore, we innovated methods to measure the influence of bypasses in the whole brain.Distinguishing bypasses from all residual shortest paths is difficult without pre-stroke imaging. We identified bypasses by finding shortest paths in subjects with stroke that were longer than the most reliably observed connections in age-matched control networks. We tested whether features of those bypasses predicted scores in four orthogonal dimensions of language performance derived from a principal components analysis of a battery of language tasks. The features were the length of each bypass in steps, and how many bypasses overlapped on each individual direct connection. We related these bypass features to language factors using support vector regression, a technique that extracts robust relationships in high-dimensional data analysis. The support vector regression parameters were tuned using grid-search cross-validation.We discovered that the length of bypasses reliably predicted variance in lexical production (R2 = .576) and auditory comprehension scores (R2 = .164). Bypass overlaps reliably predicted variance in Lexical Production scores (R2 = .247). The predictive elongation features revealed that bypass efficiency along the dorsal stream and ventral stream were most related to Lexical Production and Auditory Comprehension, respectively. Among the predictive bypass overlaps, increased bypass routing through the right hemisphere putamen was negatively related to lexical production ability.

Panoramic Manifold Projection (Panoramap) for Single-Cell Data Dimensionality Reduction and Visualization.

Nonlinear dimensionality reduction (NLDR) methods such as t-Distributed Stochastic Neighbour Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) have been widely used for biological data exploration, especially in single-cell analysis. However, the existing methods have drawbacks in preserving data’s geometric and topological structures. A high-dimensional data analysis method, called Panoramic manifold projection (Panoramap), was developed as an enhanced deep learning framework for structure-preserving NLDR. Panoramap enhances deep neural networks by using cross-layer geometry-preserving constraints. The constraints constitute the loss for deep manifold learning and serve as geometric regularizers for NLDR network training. Therefore, Panoramap has better performance in preserving global structures of the original data. Here, we apply Panoramap to single-cell datasets and show that Panoramap excels at delineating the cell type lineage/hierarchy and can reveal rare cell types. Panoramap can facilitate trajectory inference and has the potential to aid in the early diagnosis of tumors. Panoramap gives improved and more biologically plausible visualization and interpretation of single-cell data. Panoramap can be readily used in single-cell research domains and other research fields that involve high dimensional data analysis.

Introduction to the Special Issue in Honor of Professor C. R. Rao

The vast array of scientific contributions by Professor C. R. Rao has inspired considerable research in all fields of empirical sciences. The special issue of the Journal of Quantitative Economics (JQE), in honor of Professor C. R. Rao, is intended to be forward-looking, hoping to encourage young economists and econometricians to understand the influence of Rao’s contributions to economics and econometrics, to dig deeper into various gems among Professor Rao’s contributions which remain to be explored further and to review the recent advances in Bayesian analysis, high-dimensional data analysis and other areas with possible applications to economics and econometrics. Economic data are often truncated and/or censored, and yet Rao’s weighted distribution remains to see more applications.

Using the Kriging Correlation for unsupervised feature selection problems

This paper proposes a KC Score to measure feature importance in clustering analysis of high-dimensional data. The KC Score evaluates the contribution of features based on the correlation between the original features and the reconstructed features in the low dimensional latent space. A KC Score-based feature selection strategy is further developed for clustering analysis. We investigate the performance of the proposed strategy by conducting a study of four single-cell RNA sequencing (scRNA-seq) datasets. The results show that our strategy effectively selects important features for clustering. In particular, in three datasets, our proposed strategy selected less than 5% of the features and achieved the same or better clustering performance than when using all of the features.

Nonparametric locality reforming projection for linear discriminative dimensionality reduction

Discriminative dimensionality reduction (DDR) makes significant sense in high-dimensional data analysis. Over the decades, research has demonstrated that modeling the intrinsic local structure can well represent the data. Using pairwise neighboring data points to represent the between-class and within-class differences, local structure-based DDR methods are effective. However, how to set the parameters, e.g., the parameter of k-nearest neighbors, is still an open question, which makes the algorithms perform unstably. To address this issue, we propose a nonparametric local structure-based DDR method, named nonparametric locality reforming projection (NLRP). Also, we propose an effective analytical method for effectiveness analysis in both cases of sufficient and insufficient training samples for DDR methods. The experiment results on the AR, Carnegie Mellon University pose, illumination, and expression, and ORL databases are consistent with the theoretical analysis and verified the effectiveness of the proposed NLRP method. Especially, NLRP performs well in the case of very insufficient samples.

Multi-Channel High-Dimensional Data Analysis with PARAFAC-GA-BP for Nonstationary Mechanical Fault Diagnosis

Conventional signal processing methods such as Principle Component Analysis (PCA) focus on the decomposition of signals in the 2D time–frequency domain. Parallel factor analysis (PARAFAC) is a novel method used to decompose multi-dimensional arrays, which focuses on analyzing the relevant feature information by deleting the duplicated information among the multiple measurement points. In the paper, a novel hybrid intelligent algorithm for the fault diagnosis of a mechanical system was proposed to analyze the multiple vibration signals of the centrifugal pump system and multi-dimensional complex signals created by pressure and flow information. The continuous wavelet transform was applied to analyze the high-dimensional multi-channel signals to construct the 3D tensor, which makes use of the advantages of the parallel factor decomposition to extract feature information of the complex system. The method was validated by diagnosing the nonstationary failure modes under the faulty conditions with impeller blade damage, impeller perforation damage and impeller edge damage. The correspondence between different fault characteristics of a centrifugal pump in a time and frequency information matrix was established. The characteristic frequency ranges of the fault modes are effectively presented. The optimization method for a PARAFAC-BP neural network is proposed using a genetic algorithm (GA) to significantly improve the accuracy of the centrifugal pump fault diagnosis.

Innovation Model of College Education Based on Nonlinear Random Matrix Organization and Management

At present, the scientific research and innovation team in colleges and universities has become an important carrier for the development of science and technology and personnel training in China, but in terms of its development status, the role of the scientific research and innovation team has not been really brought into play. Most colleges and universities only have the organizational form of scientific research and innovation teams. Too many scientific research tasks still rely on a small number of backbones. The cohesion of the team is not enough, and the combined force of the team is not manifested. In production practice, more and more high-dimensional data with dimension p approaching or even exceeding the number of samples n are collected and stored. Unlike traditional data analysis, high-dimensional data analysis is more complex and difficult. Layer of scientific research and innovation teams in colleges and universities is often ignored by managers, which is manifested by ignoring the cultural construction of the team. In the eyes of team managers, the time spent on building a team culture is far less than the time it takes to do a few more scientific experiments. University scientific research and innovation team is a multi-level network system. The asymptotic and nonasymptotic theories of random matrices break the framework of classical multivariate statistical analysis and are very suitable for the study of statistical characteristics of high-dimensional data, which can help machine learning algorithms to complete the analysis of high-dimensional data and expand its application range. The research and innovation team in colleges and universities is deeply analyzed from the perspective of nonlinearity. It is believed that the scientific research and innovation team in colleges and universities is a complex nonlinear system in terms of scientific research objects-interdisciplinary research, or from the perspective of personnel composition. Correspondingly, the construction should be considered from a nonlinear perspective in the process, following the systematic, integral, and multi-dimensional characteristics of nonlinear systems. Specifically applied to the construction of the team, first, from a macro perspective, three principles should be paid attention to at the beginning of the creation of the team, and then the specific application should be explained in detail. It is hoped that a new perspective will be put forward in the construction of scientific research innovation teams in colleges and universities to better promote the development of teams.