Analysis To Higher Dimensions Research Articles

Gaussian process regression based on deep neural network for reliability analysis in high dimensions

Gaussian process regression based on deep neural network for reliability analysis in high dimensions

An Efficient Convex Formulation for Reduced-Rank Linear Discriminant Analysis in High Dimensions

An Efficient Convex Formulation for Reduced-Rank Linear Discriminant Analysis in High Dimensions

Dynamic Principal Component Analysis in High Dimensions

Principal component analysis is a versatile tool to reduce dimensionality which has wide applications in statistics and machine learning. It is particularly useful for modeling data in high-dimensional scenarios where the number of variables p is comparable to, or much larger than the sample size n. Despite an extensive literature on this topic, researchers have focused on modeling static principal eigenvectors, which are not suitable for stochastic processes that are dynamic in nature. To characterize the change in the entire course of high-dimensional data collection, we propose a unified framework to directly estimate dynamic eigenvectors of covariance matrices. Specifically, we formulate an optimization problem by combining the local linear smoothing and regularization penalty together with the orthogonality constraint, which can be effectively solved by manifold optimization algorithms. We show that our method is suitable for high-dimensional data observed under both common and irregular designs, and theoretical properties of the estimators are investigated under l q ( 0 ≤ q ≤ 1 ) sparsity. Extensive experiments demonstrate the effectiveness of the proposed method in both simulated and real data examples.

Sharpe Ratio analysis in high dimensions: Residual-based nodewise regression in factor models

We provide a new theory for nodewise regression when the residuals from a fitted factor model are used. We apply our results to the analysis of the consistency of Sharpe Ratio estimators when there are many assets in a portfolio. We allow for an increasing number of assets as well as time observations of the portfolio. Since the nodewise regression is not feasible due to the unknown nature of idiosyncratic errors, we provide a feasible-residual-based nodewise regression to estimate the precision matrix of errors which is consistent even when number of assets, p, exceeds the time span of the portfolio, n. In another new development, we also show that the precision matrix of returns can be estimated consistently, even with an increasing number of factors and p>n. We show that: (1) with p>n, the Sharpe Ratio estimators are consistent in global minimum-variance and mean–variance portfolios; and (2) with p>n, the maximum Sharpe Ratio estimator is consistent when the portfolio weights sum to one; and (3) with p<<n, the maximum-out-of-sample Sharpe Ratio estimator is consistent.

Significance testing for canonical correlation analysis in high dimensions.

We consider the problem of testing for the presence of linear relationships between large sets of random variables based on a post-selection inference approach to canonical correlation analysis. The challenge is to adjust for the selection of subsets of variables having linear combinations with maximal sample correlation. To this end, we construct a stabilized one-step estimator of the euclidean-norm of the canonical correlations maximized over subsets of variables of pre-specified cardinality. This estimator is shown to be consistent for its target parameter and asymptotically normal, provided the dimensions of the variables do not grow too quickly with sample size. We also develop a greedy search algorithm to accurately compute the estimator, leading to a computationally tractable omnibus test for the global null hypothesis that there are no linear relationships between any subsets of variables having the pre-specified cardinality. We further develop a confidence interval that takes the variable selection into account.

Canonical correlation analysis in high dimensions with structured regularization.

Canonical correlation analysis (CCA) is a technique for measuring the association between two multivariate data matrices. A regularized modification of canonical correlation analysis (RCCA) which imposes an ℓ2 penalty on the CCA coefficients is widely used in applications with high-dimensional data. One limitation of such regularization is that it ignores any data structure, treating all the features equally, which can be ill-suited for some applications. In this article we introduce several approaches to regularizing CCA that take the underlying data structure into account. In particular, the proposed group regularized canonical correlation analysis (GRCCA) is useful when the variables are correlated in groups. We illustrate some computational strategies to avoid excessive computations with regularized CCA in high dimensions. We demonstrate the application of these methods in our motivating application from neuroscience, as well as in a small simulation example.

Conditional reliability analysis in high dimensions based on controlled mixture importance sampling and information reuse

In many contexts, it is of interest to assess the impact of selected parameters on the failure probability of a physical system. To this end, one can perform conditional reliability analysis, in which the probability of failure becomes a function of these parameters. Computing conditional reliability requires recomputing failure probabilities for a sample sequence of the parameters, which strongly increases the already high computational cost of conventional reliability analysis. We alleviate these costs by reusing information from previous reliability computations in each subsequent reliability analysis of the sequence. The method is designed using two variants of importance sampling and performs information transfer by reusing importance densities from previous reliability analyses in the current one. We put forward a criterion for selecting the most informative importance densities, which is robust with respect to the input space dimension, and use a recently proposed density mixture model for constructing effective importance densities in high dimensions. The method controls the estimator coefficient of variation to achieve a prescribed accuracy. We demonstrate its performance by means of two engineering examples featuring a number of pitfall features such as strong non-linearity, high dimensionality and small failure probabilities (10−5to10−9).

Banks’ business models in the euro area: a cluster analysis in high dimensions

In this paper, we investigate the credit risk in the loan portfolio of banks following different business models. We develop a data-driven methodology for identifying the business models of the 365 largest European banks that is suitable for very granular harmonised supervisory data. Our dataset allows us to take into account the full range of the activities in which banks are involved. The proposed method combines in an optimal way data clustering, dimensionality reduction and outlier detection. We identify four business models and exclude as 'outliers' banks that follow idiosyncratic business models. Furthermore, empirical evidence is provided that banks following different business models differ significantly with respect to the credit risk they undertake in their loan portfolios. Traditional commercial banks are characterized by the lowest levels of credit risk while the loan portfolios of securities holding banks are riskier compared to the other banks.

Profile Analysis in High Dimensions

The three tests in profile analysis: test of parallelism, test of level and test of flatness are modified so that high-dimensional data can be analysed. Using specific scores, dimension reduction is performed and the exact null distributions are derived for the three hypotheses.

Surrogate modeling immersed probability density evolution method for structural reliability analysis in high dimensions

In conjunction with advanced surrogate modeling methods, an improved scheme of probability density evolution method (PDEM) is presented to tackle with the challenge inherent in high-dimensional structural reliability analysis. In this method, the KPCA-GPR model is developed, where the kernel principal component analysis (KPCA)-based nonlinear dimension reduction and the Gaussian process regression (GPR) surrogate model are combined via a joint-training scheme. In this regard, the identified KPCA-based subspace is optimal to the approximation accuracy of the resultant GPR model. Then, the KPCA-GPR model is constructed using the active learning (AL)-based sampling strategy, so as to accurately approximate the equivalent extreme-value (EEV) of structural responses at the whole representative point set involved in the PDEM with as fewer samples as possible. Finally, the reliability is readily evaluated by the one-dimensional integral of the EEVs’ probability density function derived from the PDEM. To illustrate the effectiveness of the proposed AL-KPCA-GPR-PDEM, two numerical examples are studied, involving the reliability analysis of both nonlinear analytical functions with different dimensions and shear-frame structures under earthquake ground motions. Numerical results indicate that massive computational cost savings and desirable accuracy enhancement are achieved by the AL-KPCA-GPR-PDEM when dealing with the reliability problems in high dimensions.

Active learning and active subspace enhancement for PDEM-based high-dimensional reliability analysis

An efficient reliability method, termed as the AL-AS-GPR-PDEM, is proposed in this study, which effectively combines the probability density evolution method (PDEM) and the Gaussian process regression (GPR) surrogate model enhanced by both the active subspace (AS)-based dimension reduction and the active learning (AL)-based sampling strategy. In this method, the AS-GPR model is developed, where the AS and the GPR are jointly calibrated so as to bypass the tricky prerequisite of gradient information to discover the latent AS of the original parameter space; then, the AL-based sampling strategy is employed to construct a satisfactorily accurate AS-GPR model using as fewer training samples as possible. This treatment allows that the newly-added training sample at each iteration can be simultaneously used to both update the discovered low-dimensional AS and refine the constructed GPR model within the AS. Lastly, the finalized AS-GPR model is employed to predict the equivalent extreme values (EEVs) of structural responses at the representative point set, thereby the PDEM-based reliability analysis can be readily performed based on the estimated EEVs. To testify the effectiveness of the proposed method, four numerical examples are addressed, involving both surrogate modelling of analytical functions with different characteristics and dynamic reliability analysis of linear/nonlinear engineering structures under seismic excitations. It is demonstrated that the proposed method is of satisfactory accuracy and efficiency for structural reliability analysis in high dimensions.

A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions

A self-calibrated direct estimation algorithm based on ℓ1-regularized quadratic programming is proposed. The self-calibration is achieved by an iterative algorithm for finding the regularization parameter simultaneously with the estimation target. The proposed algorithm is free of cross-validation. Two applications of this algorithm are proposed, namely precision matrix estimation and linear discriminant analysis. It is proven that the proposed estimators are consistent under different matrix norm errors and misclassification rate. Moreover, extensive simulation and empirical studies are conducted to evaluate the finite-sample performance and examine the support recovery ability of the proposed estimators. With the theoretical and empirical evidence, it is shown that the proposed estimator is better than its competitors in statistical accuracy and has clear computational advantages.

Global sensitivity analysis in high dimensions with PLS-PCE

Global sensitivity analysis is a central part of uncertainty quantification with engineering models. Variance-based sensitivity measures such as Sobol’ and total-effect indices are amongst the most popular and commonly used tools for global sensitivity analysis. Multiple sampling-based estimators of these measures are available, but they often come at considerable computational cost due to the large number of required model evaluations. If the computational model is expensive to evaluate, these approaches are quickly rendered infeasible. An alternative is the use of surrogate models, which reduce the computational cost per sample significantly. This contribution focuses on a recently introduced latent-variable-based polynomial chaos expansion (PCE) based on partial least squares (PLS) analysis, which is particularly suitable for high-dimensional problems. We develop an efficient way of computing variance-based sensitivities with the PLS-PCE surrogate. By back-transforming the surrogate model from its latent variable space-basis to the original input variable space-basis, we derive analytical expressions for the sought sensitivities. These expressions depend on the surrogate model coefficients exclusively. Thus, once the surrogate model is built, the variance-based sensitivities can be computed at negligible computational cost and no additional sampling is required. The accuracy of the method is demonstrated with two numerical experiments of an elastic truss and a thin steel plate.

Self-organizing mappings on the Grassmannian with applications to data analysis in high dimensions

We propose a method for extending Kohonen’s self-organizing mapping to the geometric framework of the Grassmannian. The resulting algorithm serves as a prototype of the extension of the SOM to the setting of abstract manifolds. The ingredients required for this are a means to measure distance between two points, and a method to move one point in the direction of another. In practice, the data are not required to have a representation in Euclidean space. We discuss in detail how a point on a Grassmannian is moved in the direction of another along a geodesic path. We demonstrate the implementation of the algorithm on several illustrative data sets, hyperspectral images and gene expression data sets.

Statistical inference of genetic pathway analysis in high dimensions.

Genetic pathway analysis has become an important tool for investigating the association between a group of genetic variants and traits. With dense genotyping and extensive imputation, the number of genetic variants in biological pathways has increased considerably and sometimes exceeds the sample size [Formula: see text]. Conducting genetic pathway analysis and statistical inference in such settings is challenging. We introduce an approach that can handle pathways whose dimension [Formula: see text] could be greater than [Formula: see text]. Our method can be used to detect pathways that have nonsparse weak signals, as well as pathways that have sparse but stronger signals. We establish the asymptotic distribution for the proposed statistic and conduct theoretical analysis on its power. Simulation studies show that our test has correct Type I error control and is more powerful than existing approaches. An application to a genome-wide association study of high-density lipoproteins demonstrates the proposed approach.

A Self-Calibrated Direct Approach to Precision Matrix Estimation and Linear Discriminant Analysis in High Dimensions

This paper proposes a self-calibrated direct estimation algorithm based on ell1-regularized quadratic programming. The self-calibration is achieved by an iterative algorithm for finding the regularization parameter simultaneously with the estimation target. The proposed algorithm is free of cross-validation. We consider two applications of this algorithm in this paper, namely precision matrix estimation and linear discriminant analysis. We prove the consistency results for the proposed estimators under different matrix norm errors and misclassification rate. Moreover, we conduct extensive simulation and empirical studies to evaluate the finite-sample performance and examine the support recovery ability of the proposed estimators. With the theoretical and empirical evidence, we show that the proposed estimator is better than its competitors in statistical accuracy and has clear computational advantages.

Counterfactual Analysis With Artificial Controls: Inference, High Dimensions and Nonstationarity

Recently, there has been growing interest in developing econometric tools to conduct counterfactual analysis with aggregate data when a “treated” unit suffers an intervention, such as a policy change, and there is no obvious control group. Usually, the proposed methods are based on the construction of an artificial counterfactual from a pool of “untreated” peers, organized in a panel data structure. In this paper, we consider a general framework for counterfactual analysis in high dimensions with potentially non-stationary data and either deterministic and/or stochastic trends, which nests well-established methods, such as the synthetic control. Furthermore, we propose a resampling procedure to test intervention effects that does not rely on post-intervention asymptotics and that can be used even if there is only a single observation after the intervention. A simulation study is provided as well as an empirical application where the effects of price changes on the sales of a product is measured.

Sliced inverse regression-based sparse polynomial chaos expansions for reliability analysis in high dimensions

Reliability analysis requires a large number of original model evaluations, especially for high-nonlinear and high-dimensional problems. This could be computationally expensive when a single evaluation is time-consuming, finite element models for example. Metamodeling techniques have been developed for reliability assessments in order to enhance computational efficiency. Polynomial chaos expansions are widely used for metamodel buildings, but suffering from the “curse of dimensionality”. This work proposes an efficient reliability method which combines the sliced inverse regression (SIR) with sparse polynomial chaos expansions (SPCE). The SIR technique is firstly adopted to achieve a dimension reduction by finding a new input vector which reduces the dimension of the original input vector without losing the essential information of model responses. Then a SPCE metamodel is built with respect to the reduced dimensionality by means of the stepwise regression technique. An iteration algorithm is employed to select the optimal metamodel for a given size of design of experiments. Three representative examples with random variables ranging from 6 to 300 are provided for validation, which show great effectiveness and efficiency of the proposed approach, particularly for high-dimensional problems.

Local linear smoothing for sparse high dimensional varying coefficient models

Varying coefficient models are useful generalizations of parametric linear models. They allow for parameters that depend on a covariate or that develop in time. They have a wide range of applications in time series analysis and regression. In time series analysis they have turned out to be a powerful approach to infer on behavioral and structural changes over time. In this paper, we are concerned with high dimensional varying coefficient models including the time varying coefficient model. Most studies in high dimensional nonparametric models treat penalization of series estimators. On the other side, kernel smoothing is a well established, well understood and successful approach in nonparametric estimation, in particular in the time varying coefficient model. But not much has been done for kernel smoothing in high-dimensional models. In this paper we will close this gap and we develop a penalized kernel smoothing approach for sparse high-dimensional models. The proposed estimators make use of a novel penalization scheme working with kernel smoothing. We establish a general and systematic theoretical analysis in high dimensions. This complements recent alternative approaches that are based on basis approximations and that allow more direct arguments to carry over insights from high-dimensional linear models. Furthermore, we develop theory not only for regression with independent observations but also for local stationary time series in high-dimensional sparse varying coefficient models. The development of theory for local stationary processes in a high-dimensional setting creates technical challenges. We also address issues of numerical implementation and of data adaptive selection of tuning parameters for penalization.The finite sample performance of the proposed methods is studied by simulations and it is illustrated by an empirical analysis of NASDAQ composite index data.

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.