Ultra-high Dimensions Research Articles

Uncertainty quantification for structural response field with ultra-high dimensions

The structural response field is crucial for understanding mechanical behavior, especially under uncertain conditions. However, current uncertainty quantification predominantly address one-dimensional output under multi-dimensional input. For the structural full field response with ultra-high dimension, quantifying corresponding uncertainties becomes a formidable task. This paper introduces an efficient uncertainty quantification method for structural response field. Initially, the manifold learning is introduced to transform uncertainty quantification of ultra-high-dimensional responses into the quantification of low-dimensional manifold projections. Subsequently, polynomial chaos expansions facilitate uncertainty propagation, allowing for the effective evaluation of statistical moments for each projection. Leveraging the established manifold structure enables obtaining analytical solutions for statistical moments of full field responses without additional computational costs. In the absence of prior information, the derivative λ-PDF is utilized to model uncertainty of response at any position, so as to realize the visualization of uncertainty. The proposed uncertainty quantification for structural response field provides a non-intrusive analysis framework, which can conveniently deal with structural multi-field coupling problems. Finally, three engineering examples involving multi field coupling are presented to demonstrate the effectiveness and practicability of proposed uncertainty quantification of structural response field.

High-dimensional sparse vine copula regression with application to genomic prediction.

High-dimensional data sets are often available in genome-enabled predictions. Such data sets include nonlinear relationships with complex dependence structures. For such situations, vine copula-based (quantile) regression is an important tool. However, the current vine copula-based regression approaches do not scale up to high and ultra-high dimensions. To perform high-dimensional sparse vine copula-based regression, we propose 2 methods. First, we show their superiority regarding computational complexity over the existing methods. Second, we define relevant, irrelevant, and redundant explanatory variables for quantile regression. Then, we show our method's power in selecting relevant variables and prediction accuracy in high-dimensional sparse data sets via simulation studies. Next, we apply the proposed methods to the high-dimensional real data, aiming at the genomic prediction of maize traits. Some data processing and feature extraction steps for the real data are further discussed. Finally, we show the advantage of our methods over linear models and quantile regression forests in simulation studies and real data applications.

Optimality in high-dimensional tensor discriminant analysis

Tensor discriminant analysis is an important topic in tensor data analysis. However, given the many proposals for tensor discriminant analysis methods, there lacks a systematic theoretical study, especially concerning optimality. We fill this gap by providing the minimax lower bounds for the estimation and prediction errors under the tensor discriminant analysis model coupled with the sparsity assumption. We further show that one existing high-dimensional tensor discriminant analysis estimator has matching upper bounds, and is thus optimal. Our results apply to tensors with arbitrary orders and ultra-high dimensions. If one focuses on one-way tensors (i.e., vectors), our results further provide strong theoretical justifications for several popular sparse linear discriminant analysis methods. Numerical studies are also presented to support our theoretical results.

Semiparametric penalized quadratic inference functions for longitudinal data in ultra-high dimensions

In many biomedical and health studies, multivariate data arise from repeated measurements on a sample of subjects over time. In order to analyze such longitudinal data, we need to consider the correlations from the same subject, and it is inappropriate to use a simple multivariate model assuming independence structure. Motivated by a large scale longitudinal public health study that requires longitudinal data analysis with correlated multivariate discrete responses from repeated measurements and very high dimensional covariates, we adopt a flexible semiparametric approach for simultaneous variable selection and estimation without the requirement of specifying the full likelihood. Specifically, we propose generalized partially linear single-index models using penalized quadratic inference functions for longitudinal data in ultra-high dimensions. A key feature is that we allow the number of single-index covariates in the nonparametric term to diverge and even to be in ultra-high dimensions. The penalized quadratic inference functions easily incorporate within-subject correlation and pursue efficient estimation, and the single-index models can incorporate nonlinearity and some interactions while avoiding the curse of dimensionality. In this challenging setting, we contribute both an efficient algorithm and new asymptotic theory for our proposed approach for diverging and even ultra-dimensional covariates and a multivariate correlated response in longitudinal data. We apply our method to investigate diabetes status within a continuing longitudinal public health study with very high-dimensional genetic variables and phenotype variables.

Regularized Linear Programming Discriminant Rule with Folded Concave Penalty for Ultrahigh-Dimensional Data

We propose the regularized linear programming discriminant (LPD) rule with folded concave penalty in the ultrahigh-dimensional regime. We use the local linear approximation (LLA) algorithm to redirect the model with folded concave penalty to a weighted ℓ 1 model. The strong oracle property of the solution constructed by the one-step local linear approximation (LLA) algorithm is verified. In addition, we propose efficient and parallelizable algorithms based on feature space split to address the computational challenges due to ultrahigh dimensionality. The proposed feature-split algorithm is compared to existing methods by both numerical simulations and applications to real data examples. The numerical comparisons suggest that the proposed method works well for ultrahigh dimensions, while the linear programming solver and alternating direction method of multiplier (ADMM) algorithm may fail for such high dimensions. Supplementary materials for this article are available online.

High-dimensional variable screening through kernel-based conditional mean dependence

This article proposes a model-free feature screening method for ultrahigh-dimensional data. The proposed method is built on the called kernel-based conditional mean dependence, which is defined based on the reproducing kernel embedding and the Hilbert–Schmidt norm of a tensor operator. Theoretically, both sure screening and rank consistency properties are established under weak assumptions. Through an extensive simulation study as well as a real data analysis, it is illustrated that the statistics generated by other kernels in distance kernel family are more sensitive to feature screening in ultra-high dimensions.

Copula shrinkage and portfolio allocation in ultra-high dimensions

Copulas prove to be a convenient tool in modeling joint distributions. As the data dimensionality grows, obtaining precise and well-conditioned estimates of copula-based distributions becomes a challenge. Currently, copula-based high dimensional settings are typically used for as many as a few hundred variables and require large data samples for estimation to be precise. In this paper, we handle the problem of estimation of Gaussian and t copulas in ultra-high dimensions, up to thousands of variables that use up to 30 times shorter sample lengths. Specifically, we employ recently developed large covariance matrix shrinkage tools to obtain precise and well-conditioned estimates of copula matrix parameters. Simulations show that shrinkage copulas significantly outperform traditional estimators, especially in high dimensions. We also illustrate benefits of this approach for the problem of allocation of large portfolios of stocks. Our experiments show that the shrinkage estimators applied to t copula-based dynamic models deliver better portfolios in terms of cumulative return and maximum downfall over portfolio lifetime than traditional benchmarks.

A large-dimensional test for cross-sectional anomalies:Efficient sorting revisited

Many researchers seek factors that predict the cross-section of stock returns. In finance, the key is to replicate anomalies by long–short portfolios based on their firm characteristics, with microcap biases alleviated via New York Stock Exchange (NYSE) breakpoints and value-weighted returns. In econometrics, the key is to include a covariance matrix estimator of stock returns for (mimicking) the portfolio construction. This paper marries these two strands of literature in order to test the zoo of cross-sectional anomalies by injecting size controls, basically NYSE breakpoints and value-weighted returns, into efficient sorting. We propose to use a covariance matrix estimator for ultra-high dimensions (up to 5,000) taking into account large, small and microcap stocks. We demonstrate that using a nonlinear shrinkage estimator of the covariance matrix substantially enhances the power of tests for cross-sectional anomalies: On average, t-statistics more than double. Furthermore, the proposed revisited efficient sorting method computes even highly significant factor portfolios net of transaction costs.

Petrochemical Production Big Data and its Four Typical Application Paradigms

In recent years, the big data has attracted more and more attention. It can bring us more information and broader perspective to analyse and deal with problems than the conventional situation. However, so far, there is no widely acceptable and measurable definition for the term “big data”. For example, what significant features a data set needs to have can be called big data, and how large a data set is can be called big data, and so on. Although the "5V" description widely used in textbooks has been tried to solve the above problems in many big data literatures, "5V" still has significant shortcomings and limitations, and is not suitable for completely describing big data problems in practical fields such as industrial production. Therefore, this paper creatively puts forward the new concept of data cloud and the data cloud-based "3M" descriptive definition of big data, which refers to a wide range of data sources (Multisource), ultra-high dimensions (Multi-dimensional) and a long enough time span (Multi-spatiotemporal). Based on the 3M description of big data, this paper sets up four typical application paradigms for the production big data, analyses the typical application of four paradigms of big data, and lays the foundation for applications of big data from petrochemical industry.

Large-scale model selection in misspecified generalized linear models

Summary Model selection is crucial both to high-dimensional learning and to inference for contemporary big data applications in pinpointing the best set of covariates among a sequence of candidate interpretable models. Most existing work implicitly assumes that the models are correctly specified or have fixed dimensionality, yet both model misspecification and high dimensionality are prevalent in practice. In this paper, we exploit the framework of model selection principles under the misspecified generalized linear models presented in Lv & Liu (2014), and investigate the asymptotic expansion of the posterior model probability in the setting of high-dimensional misspecified models. With a natural choice of prior probabilities that encourages interpretability and incorporates the Kullback–Leibler divergence, we suggest using the high-dimensional generalized Bayesian information criterion with prior probability for large-scale model selection with misspecification. Our new information criterion characterizes the impacts of both model misspecification and high dimensionality on model selection. We further establish the consistency of covariance contrast matrix estimation and the model selection consistency of the new information criterion in ultrahigh dimensions under some mild regularity conditions. Our numerical studies demonstrate that the proposed method enjoys improved model selection consistency over its main competitors.

Shrinkage for Gaussian and t Copulas in Ultra-High Dimensions

Copulas are a convenient framework to synthesize joint distributions, particularly in higher dimensions. Currently, copula-based high dimensional settings are used for as many as a few hundred variables and require large data samples for estimation to be precise. In this paper, we employ shrinkage techniques for large covariance matrices in the problem of estimation of Gaussian and t copulas whose dimensionality goes well beyond that typical in the literature. Specifically, we use the covariance matrix shrinkage of Ledoit and Wolf to estimate large matrix parameters of Gaussian and t copulas for up to thousands of variables, using up to 20 times lower sample sizes. The simulation study shows that the shrinkage estimation significantly outperforms traditional estimators, both in low and especially high dimensions. We also apply this approach to the problem of allocation of large portfolios.

A New Big Data Benchmark for OLAP Cube Design Using Data Pre-Aggregation Techniques

In recent years, several new technologies have enabled OLAP processing over Big Data sources. Among these technologies, we highlight those that allow data pre-aggregation because of their demonstrated performance in data querying. This is the case of Apache Kylin, a Hadoop based technology that supports sub-second queries over fact tables with billions of rows combined with ultra high cardinality dimensions. However, taking advantage of data pre-aggregation techniques to designing analytic models for Big Data OLAP is not a trivial task. It requires very advanced knowledge of the underlying technologies and user querying patterns. A wrong design of the OLAP cube alters significantly several key performance metrics, including: (i) the analytic capabilities of the cube (time and ability to provide an answer to a query), (ii) size of the OLAP cube, and (iii) time required to build the OLAP cube. Therefore, in this paper we (i) propose a benchmark to aid Big Data OLAP designers to choose the most suitable cube design for their goals, (ii) we identify and describe the main requirements and trade-offs for effectively designing a Big Data OLAP cube taking advantage of data pre-aggregation techniques, and (iii) we validate our benchmark in a case study.

A Large-Dimensional Test for Cross-Sectional Anomalies: Efficient Sorting Revisited

Many researchers seek factors that predict the cross-section of stock returns. In finance, the key is to replicate anomalies by long-short portfolios based on their factor scores, with microcaps alleviated via New York Stock Exchange (NYSE) breakpoints and value-weighted returns. In econometrics, the key is to include a covariance matrix estimator of stock returns for the (mimicking) portfolio construction. This paper marries these two strands of literature in order to test the zoo of cross-sectional anomalies by injecting size controls, basically NYSE breakpoints and value-weighted returns, into efficient sorting. Thus, we propose to use a covariance matrix estimator for ultra-high dimensions (up to 5,000) taking into account large, small and microcap stocks. We demonstrate that using a nonlinear shrinkage estimator of the covariance matrix substantially enhances the power of tests for cross-sectional anomalies: On average, ‘Student’ t-statistics more than double.

Dependence modelling in ultra high dimensions with vine copulas and the Graphical Lasso

To model high dimensional data, Gaussian methods are widely used since they remain tractable and yield parsimonious models by imposing strong assumptions on the data. Vine copulas are more flexible by combining arbitrary marginal distributions and (conditional) bivariate copulas. Yet, this adaptability is accompanied by sharply increasing computational effort as the dimension increases. The proposed approach overcomes this burden and makes the first step into ultra high dimensional non-Gaussian dependence modelling by using a divide-and-conquer approach. First, Gaussian methods are applied to split datasets into feasibly small subsets and second, parsimonious and flexible vine copulas are applied thereon. Finally, these sub-models are reconciled into one joint model. Numerical results demonstrating the feasibility of the novel approach in moderate dimensions are provided. The ability of the approach to estimate ultra high dimensional non-Gaussian dependence models in thousands of dimensions is presented.

An iterative penalized least squares approach to sparse canonical correlation analysis.

It is increasingly interesting to model the relationship between two sets of high-dimensional measurements with potentially high correlations. Canonical correlation analysis (CCA) is a classical tool that explores the dependency of two multivariate random variables and extracts canonical pairs of highly correlated linear combinations. Driven by applications in genomics, text mining, and imaging research, among others, many recent studies generalize CCA to high-dimensional settings. However, most of them either rely on strong assumptions on covariance matrices, or do not produce nested solutions. We propose a new sparse CCA (SCCA) method that recasts high-dimensional CCA as an iterative penalized least squares problem. Thanks to the new iterative penalized least squares formulation, our method directly estimates the sparse CCA directions with efficient algorithms. Therefore, in contrast to some existing methods, the new SCCA does not impose any sparsity assumptions on the covariance matrices. The proposed SCCA is also very flexible in the sense that it can be easily combined with properly chosen penalty functions to perform structured variable selection and incorporate prior information. Moreover, our proposal of SCCA produces nested solutions and thus provides great convenient in practice. Theoretical results show that SCCA can consistently estimate the true canonical pairs with an overwhelming probability in ultra-high dimensions. Numerical results also demonstrate the competitive performance of SCCA.

Quantile regression for functional partially linear model in ultra-high dimensions

Quantile regression for functional partially linear model in ultra-high dimensions is proposed and studied. By focusing on the conditional quantiles, where conditioning is on both multiple random processes and high-dimensional scalar covariates, the proposed model can lead to a comprehensive description of the scalar response. To select and estimate important variables, a double penalized functional quantile objective function with two nonconvex penalties is developed, and the optimal tuning parameters involved can be chosen by a two-step technique. Based on the difference convex analysis (DCA), the asymptotic properties of the resulting estimators are established, and the convergence rate of the prediction of the conditional quantile function can be obtained. Simulation studies demonstrate a competitive performance against the existing approach. A real application to Alzheimer’s Disease Neuroimaging Initiative (ADNI) data is used to illustrate the practicality of the proposed model.

Collision-free poisson motion planning in ultra high-dimensional molecular conformation spaces.

The function of protein, RNA, and DNA is modulated by fast, dynamic exchanges between three-dimensional conformations. Conformational sampling of biomolecules with exact and nullspace inverse kinematics, using rotatable bonds as revolute joints and noncovalent interactions as holonomic constraints, can accurately characterize these native ensembles. However, sampling biomolecules remains challenging owing to their ultra-high dimensional configuration spaces, and the requirement to avoid (self-) collisions, which results in low acceptance rates. Here, we present two novel mechanisms to overcome these limitations. First, we introduce temporary constraints between near-colliding links. The resulting constraint varieties instantaneously redirect the search for collision-free conformations, and couple motions between distant parts of the linkage. Second, we adapt a randomized Poisson-disk motion planner, which prevents local oversampling and widens the search, to ultra-high dimensions. Tests on several model systems show that the sampling acceptance rate can increase from 16% to 70%, and that the conformational coverage in loop modeling measured as average closeness to existing loop conformations doubled. Correlated protein motions identified with our algorithm agree with those from MD simulations. © 2018 Wiley Periodicals, Inc.

Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression

ABSTRACTWe propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates a regression coefficient and its corresponding subgradient simultaneously in each iteration. It combines the strengths of the coordinate descent and the semismooth Newton algorithm, and effectively solves the computational challenges posed by dimensionality and nonsmoothness. We establish the convergence properties of the algorithm. In addition, we present an adaptive version of the “strong rule” for screening predictors to gain extra efficiency. Through numerical experiments, we demonstrate that the proposed algorithm is very efficient and scalable to ultrahigh dimensions. We illustrate the application via a real data example. Supplementary materials for this article are available online.

Likelihood ratio test for partial sphericity in high and ultra-high dimensions

We consider, in the setting of p and n large, sample covariance matrices whose population counterparts follow a spiked population model, i.e., with the exception of the first (largest) few, all the population eigenvalues are equal. We study the asymptotic distribution of the partial maximum likelihood ratio statistic and use it to test for the dimension of the population spike subspace. Furthermore, we extend this to the ultra-high-dimensional case, i.e., p > n . A thorough study of the power of the test gives a correction that allows us to test for the dimension of the population spike subspace even for values of the limit of p / n close to 1 , a setting where other approaches have proved to be deficient.

Making decision trees feasible in ultrahigh feature and label dimensions

Due to the non-linear but highly interpretable representations, decision tree (DT) models have significantly attracted a lot of attention of researchers. However, it is difficult to understand and interpret DT models in ultrahigh dimensions and DT models usually suffer from the curse of dimensionality and achieve degenerated performance when there are many noisy features. To address these issues, this paper first presents a novel data-dependent generalization error bound for the perceptron decision tree (PDT), which provides the theoretical justification to learn a sparse linear hyperplane in each decision node and to prune the tree. Following our analysis, we introduce the notion of budget-aware classifier (BAC) with a budget constraint on the weight coefficients, and propose a supervised budgeted tree (SBT) algorithm to achieve non-linear prediction performance. To avoid generating an unstable and complicated decision tree and improve the generalization of the SBT, we present a pruning strategy by learning classifiers to minimize cross-validation errors on each BAC. To deal with ultrahigh label dimensions, based on three important phenomena of real-world data sets from a variety of application domains, we develop a sparse coding tree framework for multi-label annotation problems and provide the theoretical analysis. Extensive empirical studies verify that 1) SBT is easy to understand and interpret in ultrahigh dimensions and is more resilient to noisy features. 2) Compared with state-of-the-art algorithms, our proposed sparse coding tree framework is more efficient, yet accurate in ultrahigh label and feature dimensions.