Sufficient Dimension Reduction Research Articles

The Alternating Direction Method of Multipliers for Sufficient Dimension Reduction

The minimum average variance estimation (MAVE) method has proven to be an effective approach to sufficient dimension reduction. In this study, we apply the computationally efficient optimization algorithm named alternating direction method of multipliers (ADMM) to a particular approach (MAVE or minimum average variance estimation) to the problem of sufficient dimension reduction (SDR). Under some assumptions, we prove that the iterative sequence generated by ADMM converges to some point of the associated augmented Lagrangian function. Moreover, that point is stationary. It also presents some numerical simulations on synthetic data to demonstrate the computational efficiency of the algorithm.

Using noise filtering and sufficient dimension reduction method on unstructured economic data

Using noise filtering and sufficient dimension reduction method on unstructured economic data

An active learning method using deep adversarial autoencoder-based sufficient dimension reduction neural network for high-dimensional reliability analysis

Reliability analysis often requires time-consuming evaluations, especially when dealing with high-dimensional and nonlinear problems. To address this challenge, surrogate model methods are frequently employed. One way to improve the efficiency of surrogate model methods involves selecting informative samples that significantly enhance the accuracy of the surrogate model. This paper introduces a novel approach to facilitate the construction of surrogate models and selection of informative samples in high-dimensional reliability analysis, through an active learning method based on a deep adversarial autoencoder-based sufficient dimension reduction (AAE-SDR) neural network. The AAE-SDR neural network serves as a surrogate model, transforming complex high-dimensional variables into tractable, low-dimensional embeddings relevant to the target. These embeddings are Gaussian-distributed with a distinct latent limit state boundary. A new sampling strategy is proposed to select informative misclassified samples by iteratively identifying candidate samples near the latent limit state boundary and uniformly sampling from the candidate sample dataset based on the latent Gaussian distribution. The effectiveness of the proposed approach is demonstrated through two high-dimensional numerical examples and a cable-stayed bridge case study. Results show that the proposed method simplifies complex high-dimensional reliability problems and provides a relatively accurate estimated failure probability with a limited number of samples.

Variable selection using data splitting and projection for principal fitted component models in high dimension

Sufficient dimension reduction (SDR) is such an effective way to detect nonlinear relationship between response variable and covariates by reducing the dimensionality of covariates without information loss. The principal fitted component (PFC) model is a way to implement SDR using some class of basis functions, however the PFC model is not efficient when there are many irrelevant or noisy covariates. There have been a few studies on the selection of variables in the PFC model via penalized regression or sequential likelihood ratio test. A novel variable selection technique in the PFC model has been proposed by incorporating a recent development in multiple testing such as mirror statistics and random data splitting. It is highlighted how we construct a mirror statistic in the PFC model using the idea of projection of coefficients to the other space generated from data splitting. The proposed method is superior to some existing methods in terms of false discovery rate (FDR) control and applicability to high-dimensional cases. In particular, the proposed method outperforms other methods as the number of covariates tends to be getting larger, which would be appealing in high dimensional data analysis. Simulation studies and analyses of real data sets have been conducted to show the finite sample performance and the gain that it yields compared to existing methods.

Clustering of lifestyle risk factors in relation to suicidal thoughts and behaviors in young adolescents: a cross-national study of 45 low- and middle-income countries

BackgroundPrior research has reaffirmed lifestyle risk behaviors to cluster among adolescents. However, the lifestyle cluster effect on suicidal thoughts and behaviors (STBs) was unclear among adolescents in low- and middle-income countries (LMICs). No comparison of such associations was conducted across nations.MethodsData from 45 LMICs were obtained from the Global School-based Student Health Survey (GSHS) between 2009 and 2019. Lifestyle behavior factors were collected through a structured questionnaire. Suicidal ideation, plan, and attempt were ascertained by three single-item questions. Lifestyle risk scores were calculated via a sufficient dimension reduction technique, and lifestyle risk clusters were constructed using a latent class analysis. Generalized linear mixed models with odds ratio (OR) and 95% confidence interval (CI) were used to estimate the lifestyle-STB associations.ResultsA total of 229,041 adolescents were included in the final analysis. The weighted prevalence of suicidal ideation, plan, and attempt was 7.37%, 5.81%, and 4.59%, respectively. Compared with the favorable lifestyle group, the unfavorable group had 1.48-, 1.53-, and 3.11-fold greater odds of suicidal ideation (OR = 1.48, 95%CI: 1.30–1.69), plan (OR = 1.53, 95%CI 1.34–1.75), and attempt (OR = 3.11, 95%CI 2.64–3.65). Four clusters of lifestyle risk behaviors were identified, namely healthy lifestyles (H–L), insufficient intake of vegetables and fruit (V-F), frequent consumption of soft drinks and fast food (D-F), and tobacco smoking and alcohol drinking (S-A) clusters. Compared with H–L cluster, V-F cluster was associated with 43% and 42% higher odds of suicidal ideation and plan, followed by S-A cluster (26% for ideation and 20% for plan), but not significant in D-F cluster (P > 0.05). D-F cluster was associated with 2.85-fold increased odds of suicidal attempt, followed by V-F cluster (2.43-fold) and S-A cluster (1.18-fold).ConclusionsClustering of lifestyle risk behaviors is informative for risk stratification of STBs in resource-poor settings. Lifestyle-oriented suicide prevention efforts should be initiated among school-attending adolescents in LMICs.

Functional sufficient dimension reduction through information maximization with application to classification

Considering the case where the response variable is a categorical variable and the predictor is a random function, two novel functional sufficient dimensional reduction (FSDR) methods are proposed based on mutual information and square loss mutual information. Compared to the classical FSDR methods, such as functional sliced inverse regression and functional sliced average variance estimation, the proposed methods are appealing because they are capable of estimating multiple effective dimension reduction directions in the case of a relatively small number of categories, especially for the binary response. Moreover, the proposed methods do not require the restrictive linear conditional mean assumption and the constant covariance assumption. They avoid the inverse problem of the covariance operator which is often encountered in the functional sufficient dimension reduction. The functional principal component analysis with truncation be used as a regularization mechanism. Under some mild conditions, the statistical consistency of the proposed methods is established. Simulation studies and real data analyzes are used to evaluate the finite sample properties of our methods.

A selective review of nonlinear sufficient dimension reduction

A selective review of nonlinear sufficient dimension reduction

Beta regression for double‐bounded response with correlated high‐dimensional covariates

Continuous responses measured on a standard unit interval are ubiquitous in many scientific disciplines. Statistical models built upon a normal error structure do not generally work because they can produce biassed estimates or result in predictions outside either bound. In real‐life applications, data are often high‐dimensional, correlated and consist of a mixture of various data types. Little literature is available to address the unique data challenge. We propose a semiparametric approach to analyse the association between a double‐bounded response and high‐dimensional correlated covariates of mixed types. The proposed method makes full use of all available data through one or several linear combinations of the covariates without losing information from the data. The only assumption we make is that the response variable follows a Beta distribution; no additional assumption is required. The resulting estimators are consistent and efficient. We illustrate the proposed method in simulation studies and demonstrate it in a real‐life data application. The semiparametric approach contributes to the sufficient dimension reduction literature for its novelty in investigating double‐bounded response which is absent in the current literature. This work also provides a new tool for data practitioners to analyse the association between a popular unit interval response and mixed types of high‐dimensional correlated covariates.

Model-Free Statistical Inference on High-Dimensional Data

This article aims to develop an effective model-free inference procedure for high-dimensional data. We first reformulate the hypothesis testing problem via sufficient dimension reduction framework. With the aid of new reformulation, we propose a new test statistic and show that its asymptotic distribution is χ 2 distribution whose degree of freedom does not depend on the unknown population distribution. We further conduct power analysis under local alternative hypotheses. In addition, we study how to control the false discovery rate of the proposed χ 2 tests, which are correlated, to identify important predictors under a model-free framework. To this end, we propose a multiple testing procedure and establish its theoretical guarantees. Monte Carlo simulation studies are conducted to assess the performance of the proposed tests and an empirical analysis of a real-world dataset is used to illustrate the proposed methodology. Supplementary materials for this article are available online including a standardized description of the materials available for reproducing the work.

A Slicing‐Free Perspective to Sufficient Dimension Reduction: Selective Review and Recent Developments

SummarySince the pioneering work of sliced inverse regression, sufficient dimension reduction has been growing into a mature field in statistics and it has broad applications to regression diagnostics, data visualisation, image processing and machine learning. In this paper, we provide a review of several popular inverse regression methods, including sliced inverse regression (SIR) method and principal hessian directions (PHD) method. In addition, we adopt a conditional characteristic function approach and develop a new class of slicing‐free methods, which are parallel to the classical SIR and PHD, and are named weighted inverse regression ensemble (WIRE) and weighted PHD (WPHD), respectively. Relationship with recently developed martingale difference divergence matrix is also revealed. Numerical studies and a real data example show that the proposed slicing‐free alternatives have superior performance than SIR and PHD.

대용량 자료의 이항 분류에서의 충분 차원 축소를 위한 전향적 접근 방법

Sufficient dimension reduction, aimed at finding a lower-dimensional subspace in explanatory variables that contains response variable information, typically relies on inverse-based methodologies. These methods are easy to implement but often require linear or constant variance conditions. To address these limitations, techniques for forwardly estimating the central subspace have been developed. In particular, methods utilizing the Reproducing Kernel Hilbert Space have gained attention, but their use in analyzing large datasets is limited due to the characteristics of the kernel space. In this paper, we study a novel forward approach for sufficient dimension reduction in binomial classification of large-scale data. We propose a method that employs a divide-and-conquer technique to split data into subsets, then independently perform dimension reduction on each subset before synthesizing them into a final model. It was shown that when the number of partitions of data was appropriately selected, the loss in prediction accuracy was not significant compared to the existing method, while being efficient in terms of storage space and calculation cost. In addition, simulations in various models showed superior prediction accuracy than other inverse-based techniques. The utility of the proposed method was confirmed through the real data analysis.

Nonlinear sufficient dimension reduction for distribution-on-distribution regression

We introduce a new approach to nonlinear sufficient dimension reduction in cases where both the predictor and the response are distributional data, modeled as members of a metric space. Our key step is to build universal kernels (cc-universal) on the metric spaces, which results in reproducing kernel Hilbert spaces for the predictor and response that are rich enough to characterize the conditional independence that determines sufficient dimension reduction. For univariate distributions, we construct the universal kernel using the Wasserstein distance, while for multivariate distributions, we resort to the sliced Wasserstein distance. The sliced Wasserstein distance ensures that the metric space possesses similar topological properties to the Wasserstein space, while also offering significant computation benefits. Numerical results based on synthetic data show that our method outperforms possible competing methods. The method is also applied to several data sets, including fertility and mortality data and Calgary temperature data.

Robust Sparse Sufficient Dimension Reduction via Adaptive Lasso Penalty

In some multi-index models applications, there is an important role for dimension reduction and variable selection (VS) methods. The ALMAVE is a method for variables selection under sufficient dimension reduction theory settings. It combines the adaptive Lasso and MAVE (minimum average variance estimation) to produce sparse and accurate solutions. The ALMAVE is a very sensitive method to outliers in the response y due to use the least-squares criterion. In this article, we proposed robust ALMAVE. Also, an efficient estimation algorithm was proposed. The simulation studies and Logo design data analysis were employed to check the effectiveness of ALMAVE.

Sliced Inverse Regression in Metric Spaces

In this article, we propose a general nonlinear sufficient dimension reduction (SDR) framework when both the predictor and response lie in some general metric spaces. We construct reproducing kernel Hilbert spaces whose kernels are fully determined by the distance functions of the metric spaces, then leverage the inherent structures of these spaces to define a nonlinear SDR framework. We adapt the classical sliced inverse regression of \citet{Li:1991} within this framework for the metric space data. We build the estimator based on the corresponding linear operators, and show it recovers the regression information unbiasedly. We derive the estimator at both the operator level and under a coordinate system, and also establish its convergence rate. We illustrate the proposed method with both synthetic and real datasets exhibiting non-Euclidean geometry.

Research on Stock Price Prediction Model Based on Weighted Sufficient Dimension Reduction and Bagging Framework

Research on Stock Price Prediction Model Based on Weighted Sufficient Dimension Reduction and Bagging Framework

A note on sufficient dimension reduction with post dimension reduction statistical inference

A note on sufficient dimension reduction with post dimension reduction statistical inference

Likelihood-based surrogate dimension reduction

We consider the problem of surrogate sufficient dimension reduction, that is, estimating the central subspace of a regression model, when the covariates are contaminated by measurement error. When no measurement error is present, a likelihood-based dimension reduction method that relies on maximizing the likelihood of a Gaussian inverse regression model on the Grassmann manifold is well-known to have superior performance to traditional inverse moment methods. We propose two likelihood-based estimators for the central subspace in measurement error settings, which make different adjustments to the observed surrogates. Both estimators are computed based on maximizing objective functions on the Grassmann manifold and are shown to consistently recover the true central subspace. When the central subspace is assumed to depend on only a few covariates, we further propose to augment the likelihood function with a penalty term that induces sparsity on the Grassmann manifold to obtain sparse estimators. The resulting objective function has a closed-form Riemann gradient which facilitates efficient computation of the penalized estimator. We leverage the state-of-the-art trust region algorithm on the Grassmann manifold to compute the proposed estimators efficiently. Simulation studies and a data application demonstrate the proposed likelihood-based estimators perform better than inverse moment-based estimators in terms of both estimation and variable selection accuracy.

Quantitative trading models based on Sufficient Dimension Reduction and Ensemble Learning

With the continuous development of the securities market, constructing quantitative trading strategies with strong generalization ability the adaption to dynamic and changeable market environment is becoming more and more momentous in the field of quantitative finance. Under such circumstances, this paper proposes a stock prediction model based on sufficient dimension reduction theory and ensemble learning, which can be deployed on quantitative trading strategy. The proposed model on the one hand alleviates the curse of dimension by using the sufficient dimension reduction method and maximally retains variation of stock factors, on the other hand employs ensemble learning technique which can weigh model bias and variance to address the overfitting or underfitting problems. Furthermore, since ensemble learning are model free, different sub-models can be applied to improve the generalization ability of the algorithm on predicting occasions. In the section of accuracy comparison and quantitative trading strategy experiments based on real stock datasets, the proposed model demonstrates the best prediction results and adequate robustness compared to other existed methods.

An improved sufficient dimension reduction-based Kriging modeling method for high-dimensional evaluation-expensive problems

Kriging is a powerful surrogate modeling method for the analysis and optimization of computationally expensive problems. However, the efficient construction of high-precision models for high-dimensional expensive problems is an important challenge for Kriging method and has received much attention recently. To address this challenge, we propose an improved sufficient dimension reduction-based Kriging modeling method (KISDR) for high-dimensional evaluation-expensive problems. First, the martingale difference divergence is introduced into the sufficient dimension reduction to obtain a more accurate and stable estimate of the projection matrix, which can project the high-dimensional inputs into a low-dimensional latent space while maintaining sufficient information for response prediction. Second, we propose to utilize the ladle estimator to identify the dimension of latent space. The ladle estimator combines both eigenvalues and eigenvectors of the matrix and can identify the latent dimensionality more precisely. After that, a new Kriging correlation function is constructed by integrating the information of dimension reduction into the correlation structure, which significantly reduces the number of hyperparameters to be estimated. Finally, we devise a local optimization approach to fine-tune the Kriging hyperparameters to further improve the modeling accuracy. In this study, six mathematical examples and three engineering examples with dimensions varying from 30-D to 100-D are employed for performance analysis and comparison. The results indicate that the proposed KISDR can precisely identify and exploit the low-dimensional linear structure in the data to improve modeling accuracy and efficiency for high-dimensional evaluation-expensive problems.

Semiparametric recovery of central dimension reduction space with nonignorable nonresponse

Sufficient dimension reduction (SDR) methods are effective tools for handling high dimensional data. Classical SDR methods are developed under the assumption that the data are completely observed. When the data are incomplete due to missing values, SDR has only been considered when the data are randomly missing, but not when they are nonignorably missing, which is arguably more difficult to handle due to the missing values' dependence on the reasons they are missing. The purpose of this paper is to fill this void. We propose an intuitive, easy‐to‐implement SDR estimator based on a semiparametric propensity score function for response data with non‐ignorable missing values. We refer to it as the dimension reduction‐based imputed estimator. We establish the theoretical properties of this estimator and examine its empirical performance via an extensive numerical study on real and simulated data. As well, we compare the performance of our proposed dimension reduction‐based imputed estimator with two competing estimators, including the fusion refined estimator and cumulative slicing estimator. A distinguishing feature of our method is that it requires no validation sample. The SDR theory developed in this paper is a non‐trivial extension of the existing literature, due to the technical challenges posed by nonignorable missingness. All the technical proofs of the theorems are given in the Appendix .