High-dimensional Quantile Regression Research Articles

Uncertainty quantification in epigenetic clocks via conformalized quantile regression.

DNA methylation (DNAm) is a chemical modification of DNA that can be influenced by various factors, including age, environment, and lifestyle. An epigenetic clock is a predictive tool that measures biological age based on DNAm levels. It can provide insights into an individual's biological age, which may differ from their chronological age. This difference, known as the epigenetic age acceleration, may indicate the state of one's health and risk for age-related diseases. Moreover, epigenetic clocks are used in studies of aging to assess the effectiveness of anti-aging interventions and to understand the underlying mechanisms of aging and disease. Various epigenetic clocks have been developed using samples from different populations, tissues, and cell types, typically by training high-dimensional linear regression models with an elastic net penalty. While these models can predict mean biological age with high precision, there is a lack of uncertainty quantification which is important for interpreting the precision of age estimations and for clinical decision-making. To understand the distribution of a biological age clock beyond its mean, we propose a general pipeline for training epigenetic clocks, based on an integration of high-dimensional quantile regression and conformal prediction, to effectively reveal population heterogeneity and construct prediction intervals. Our approach produces adaptive prediction intervals not only achieving nominal coverage but also accounting for the inherent variability across individuals. By using the data collected from 728 blood samples in 11 DNAm datasets from children, we find that our quantile regression-based prediction intervals are narrower than those derived from conventional mean regression-based epigenetic clocks. This observation demonstrates an improved statistical efficiency over the existing pipeline for training epigenetic clocks. In addition, the resulting intervals have a synchronized varying pattern to age acceleration, effectively revealing cellular evolutionary heterogeneity in age patterns in different developmental stages during individual childhoods and adolescent cohort. Our findings suggest that conformalized high-dimensional quantile regression can produce valid prediction intervals and uncover underlying population heterogeneity. Although our methodology focuses on the distribution of aging in children, it is applicable to a broader range of populations to improve understanding of epigenetic age beyond the mean. This inference-based toolbox could provide valuable insights for future applications of epigenetic interventions for age-related diseases.

Revealing accuracy in climate dynamics: enhancing evapotranspiration estimation using advanced quantile regression and machine learning models

This study examines the effectiveness of various quantile regression (QR) and machine learning (ML) methodologies developed for analyzing the relationship between meteorological parameters and daily reference evapotranspiration (ETref) across diverse climates in Iran spanning from 1987 to 2022. The analyzed models include D-vine copula-based quantile regression (DVQR), multivariate linear quantile regression (MLQR), Bayesian model averaging quantile regression (BMAQR), as well as machine learning algorithms such as extreme learning machine (ELM), random forest (RF), M5 model Tree (M5Tree), least squares support vector regression algorithm (LSSVR), and extreme gradient boosting (XGBoost). Additionally, empirical equations (EEs) such as Baier and Robertson (BARO), Jensen and Haise (JEHA), and Penman (PENM) models were considered. While the EEs demonstrated acceptable performance, the QR and ML models exhibited superior accuracy. Among these, the MLQR model displayed the highest accuracy compared to DVQR and BMAQR models. Moreover, LSSVR, XGBoost, and M5Tree models outperformed ELM and RF models. Notably, LSSVR, XGBoost, and MLQR models exhibited comparable performance (R2 and NSE > 0.92, MBE and RMSE < 0.5, and SI > 0.05) to M5Tree and BMAQR models across all climates. Importantly, these models significantly outperformed EEs, DVQR, ELM, and RF models in all climates. In conclusion, high-dimensional QR and ML models are recommended as promising alternatives for accurately estimating daily ETref in diverse global climate conditions.

Screen then select: a strategy for correlated predictors in high-dimensional quantile regression

Screen then select: a strategy for correlated predictors in high-dimensional quantile regression

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Modern massive data with enormous sample size and tremendous dimensionality are usually impossible to process with a single machine. They are typically stored and processed in a distributed manner. In this paper, we propose a distributed bootstrap simultaneous inference for a high-dimensional quantile regression model using massive data. Meanwhile, a communication-efficient (CE) distributed learning algorithm is developed via the CE surrogate likelihood framework and ADMM procedure, which can handle the non-smoothness of the quantile regression loss and the Lasso penalty. We theoretically prove the convergence of the algorithm and establish a lower bound on the number of communication rounds ιmin that warrant statistical accuracy and efficiency. The distributed bootstrap validity and efficiency are corroborated by an extensive simulation study.

Reprint: Hypothesis testing on high dimensional quantile regression

Quantile regression has been an important analytical tool in econometrics since it was proposed in 1970s. Many advantages make it still popular in the era of big data. This paper focuses on the testing problems of high-dimensional quantile regression, supplementing to robust methods in the literature of high-dimensional hypothesis testing. We first construct a new test statistic based on the quantile regression score function. The new test statistic avoids the inverse operation of the covariance matrix, and hence becomes applicable to high-dimensional or even ultrahigh-dimensional settings. The proposed method retains robustness for non-Gaussian and heavy-tailed distributions. We then derive the limiting distributions of the proposed test statistic under both the null and the alternative hypotheses. We further investigate the case where the design matrix follows an elliptical distribution. We examined the finite sample performance of our proposed method through Monte Carlo simulations. The numerical comparisons exhibit that our proposed tests outperform some existing methods in terms of controlling Type I error rate and power, when the data deviate from the Gaussian assumptions or are heavy-tailed. We illustrate our proposed high-dimensional quantile testing in financial econometrics, through an empirical analysis of Chinese stock market data.

Robust transfer learning for high-dimensional quantile regression model with linear constraints

Robust transfer learning for high-dimensional quantile regression model with linear constraints

Bayesian variable selection and estimation in binary quantile regression using global-local shrinkage priors

In this paper, we construct a Bayesian hierarchical model with global-local shrinkage priors for the regression coefficients, which includes the horseshoe prior and normal-gamma prior. This model is used for high-dimensional quantile regression models with dichotomous response data. We have developed an efficient sampling algorithm to generate posterior samplings for making posterior inference. We use a location-scale mixture representation of the asymmetric Laplace distribution. We assess the performance of the proposed methods through Monte Carlo simulations and two real-data applications in terms of parameter estimation and variable selection. Numerical results demonstrate that the proposed methods perform comparably with existing Bayesian methods under a variety of scenarios.

Two-Stage Online Debiased Lasso Estimation and Inference for High-Dimensional Quantile Regression with Streaming Data

Two-Stage Online Debiased Lasso Estimation and Inference for High-Dimensional Quantile Regression with Streaming Data

Hypothesis testing on high dimensional quantile regression

Quantile regression has been an important analytical tool in econometrics since it was proposed in 1970s. Many advantages make it still popular in the era of big data. This paper focuses on the testing problems of high-dimensional quantile regression, supplementing to robust methods in the literature of high-dimensional hypothesis testing. We first construct a new test statistic based on the quantile regression score function. The new test statistic avoids the inverse operation of the covariance matrix, and hence becomes applicable to high-dimensional or even ultrahigh-dimensional settings. The proposed method retains robustness for non-Gaussian and heavy-tailed distributions. We then derive the limiting distributions of the proposed test statistic under both the null and the alternative hypotheses. We further investigate the case where the design matrix follows an elliptical distribution. We examined the finite sample performance of our proposed method through Monte Carlo simulations. The numerical comparisons exhibit that our proposed tests outperform some existing methods in terms of controlling Type I error rate and power, when the data deviate from the Gaussian assumptions or are heavy-tailed. We illustrate our proposed high-dimensional quantile testing in financial econometrics, through an empirical analysis of Chinese stock market data.

A tuning-free efficient test for marginal linear effects in high-dimensional quantile regression

A tuning-free efficient test for marginal linear effects in high-dimensional quantile regression

High-Dimensional Variable Selection for Quantile Regression Based on Variational Bayesian Method

The quantile regression model is widely used in variable relationship research of moderate sized data, due to its strong robustness and more comprehensive description of response variable characteristics. With the increase of data size and data dimensions, there have been some studies on high-dimensional quantile regression under the classical statistical framework, including a high-efficiency frequency perspective; however, this comes at the cost of randomness quantification, or the use of a lower efficiency Bayesian method based on MCMC sampling. To overcome these problems, we propose high-dimensional quantile regression with a spike-and-slab lasso penalty based on variational Bayesian (VBSSLQR), which can, not only improve the computational efficiency, but also measure the randomness via variational distributions. Simulation studies and real data analysis illustrated that the proposed VBSSLQR method was superior or equivalent to other quantile and nonquantile regression methods (including Bayesian and non-Bayesian methods), and its efficiency was higher than any other method.

Communication-efficient estimation and inference for high-dimensional quantile regression based on smoothed decorrelated score.

Distributed estimation based on different sources of observations has drawn attention in the modern statistical learning. In practice, due to the expensive cost or time-consuming process to collect data in some cases, the sample size on each local site can be small, but the dimension of covariates is large and may be far larger than the sample size on each site. In this article, we focus on the distributed estimation and inference for a preconceived low-dimensional parameter vector in the high-dimensional quantile regression model with small local sample size. Specifically, we consider that the data are inherently distributed and propose two communication-efficient estimators by generalizing the decorrelated score approach to conquer the slow convergence rate of nuisance parameter estimation and adopting the smoothing technique based on multiround algorithms. The risk bounds and limiting distributions of the proposed estimators are given. The finite sample performance of the proposed estimators is studied through simulations and an application to a gene expression dataset is also presented.

High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization

Abstractℓ 1 -penalized quantile regression (QR) is widely used for analysing high-dimensional data with heterogeneity. It is now recognized that the ℓ1-penalty introduces non-negligible estimation bias, while a proper use of concave regularization may lead to estimators with refined convergence rates and oracle properties as the signal strengthens. Although folded concave penalized M-estimation with strongly convex loss functions have been well studied, the extant literature on QR is relatively silent. The main difficulty is that the quantile loss is piecewise linear: it is non-smooth and has curvature concentrated at a single point. To overcome the lack of smoothness and strong convexity, we propose and study a convolution-type smoothed QR with iteratively reweighted ℓ1-regularization. The resulting smoothed empirical loss is twice continuously differentiable and (provably) locally strongly convex with high probability. We show that the iteratively reweighted ℓ1-penalized smoothed QR estimator, after a few iterations, achieves the optimal rate of convergence, and moreover, the oracle rate and the strong oracle property under an almost necessary and sufficient minimum signal strength condition. Extensive numerical studies corroborate our theoretical results.

Optimal time-varying tail risk network with a rolling window approach

We contribute to the literature on financial network contagion and systemic risk by developing a time-varying framework based on the rolling window technic and high dimensional quantile regression. The new selection criterion enables us to determine the optimal rolling width, which trades off the estimation accuracy and time variation of the tail risk network. Monte Carlo simulations show that our procedure significantly improves upon the estimation performance of the time-varying model. Using Chinese banking’s market data over 2009–2019, we measure the time-varying tail risk connectedness among banks using idiosyncratic returns. We find strong evidence of prominent risk dependence across banks during the stock crash. The network model using the optimal rolling window outperforms the traditional approaches in capturing structural changes.

Regularized projection score estimation of treatment effects in high-dimensional quantile regression

Regularized projection score estimation of treatment effects in high-dimensional quantile regression

Sparse signal shrinkage and outlier detection in high-dimensional quantile regression with variational Bayes

Sparse signal shrinkage and outlier detection in high-dimensional quantile regression with variational Bayes

Lack-of-Fit Tests for Quantile Regression Models

SummaryThe paper novelly transforms lack-of-fit tests for parametric quantile regression models into checking the equality of two conditional distributions of covariates. Accordingly, by applying some successful two-sample test statistics in the literature, two tests are constructed to check the lack of fit for low and high dimensional quantile regression models. The low dimensional test works well when the number of covariates is moderate, whereas the high dimensional test can maintain the power when the number of covariates exceeds the sample size. The null distribution of the high dimensional test has an explicit form, and the p-values or critical values can then be calculated directly. The finite sample performance of the tests proposed is examined by simulation studies, and their usefulness is further illustrated by two real examples.

Debiasing and Distributed Estimation for High-Dimensional Quantile Regression.

Distributed and parallel computing is becoming more important with the availability of extremely large data sets. In this article, we consider this problem for high-dimensional linear quantile regression. We work under the assumption that the coefficients in the regression model are sparse; therefore, a LASSO penalty is naturally used for estimation. We first extend the debiasing procedure, which is previously proposed for smooth parametric regression models to quantile regression. The technical challenges include dealing with the nondifferentiability of the loss function and the estimation of the unknown conditional density. In this article, the main objective is to derive a divide-and-conquer estimation approach using the debiased estimator which is useful under the big data setting. The effectiveness of distributed estimation is demonstrated using some numerical examples.

Oracle Estimation of a Change Point in High-Dimensional Quantile Regression

ABSTRACTIn this article, we consider a high-dimensional quantile regression model where the sparsity structure may differ between two sub-populations. We develop ℓ1-penalized estimators of both regression coefficients and the threshold parameter. Our penalized estimators not only select covariates but also discriminate between a model with homogenous sparsity and a model with a change point. As a result, it is not necessary to know or pretest whether the change point is present, or where it occurs. Our estimator of the change point achieves an oracle property in the sense that its asymptotic distribution is the same as if the unknown active sets of regression coefficients were known. Importantly, we establish this oracle property without a perfect covariate selection, thereby avoiding the need for the minimum level condition on the signals of active covariates. Dealing with high-dimensional quantile regression with an unknown change point calls for a new proof technique since the quantile loss function is nonsmooth and furthermore the corresponding objective function is nonconvex with respect to the change point. The technique developed in this article is applicable to a general M-estimation framework with a change point, which may be of independent interest. The proposed methods are then illustrated via Monte Carlo experiments and an application to tipping in the dynamics of racial segregation. Supplementary materials for this article are available online.

Model selection in high-dimensional quantile regression with seamless [formula omitted] penalty

We introduce and study the seamless L0 quantile estimator in a linear model when the number of parameters increases with sample size. For this estimator we derive the convergence rate and oracle properties. A consistent BIC criterion to select the tuning parameters is given.