SLOPE for Sparse Linear Regression: Asymptotics and Optimal Regularization

Abstract

In sparse linear regression, the SLOPE estimator generalizes LASSO by penalizing different coordinates of the estimate according to their magnitudes. In this paper, we present a precise performance characterization of SLOPE under the i.i.d. Gaussian design. We focus on the asymptotic regime where the number of unknown parameters grows in proportion to the number of observations. Our asymptotic characterization enables us to derive the fundamental limits of SLOPE in both estimation and variable selection settings. We also provide a computationally feasible way to optimally design the regularizing sequences to reach the aforementioned fundamental limits. In both settings, we show that the optimal design problem can be formulated as certain infinite-dimensional convex optimization problems that have efficient and accurate finite-dimensional approximations. Numerical simulations verify all our asymptotic predictions. They demonstrate the superiority of our optimal regularizing sequences over other designs used in the existing literature.