A data-driven line search rule for support recovery in high-dimensional data analysis

Abstract

For ℓ0 penalized (nonlinear) regression problems, most existing algorithms carried out theoretical analysis and numerical calculation with a fixed step size. However, the selection of an appropriate step size and the guarantee of good performance depend heavily on the parameters of the restricted strong convexity and smoothness of the loss function, which are difficult to calculate in practice. To overcome this problem, a novel and efficient data-driven line search rule is proposed to adaptively determine the appropriate step size based on the idea of support detection and root finding. For the step size by the line search, the ℓ2 error bound of iteration sequence and the target regression coefficient has be analyzed without any restrictions on the parameters of the loss function. A lot of numerical comparisons with state-of-the-art algorithms in linear and logistic regression problems show the stability, effectiveness and superiority of the proposed algorithms.