Efficient distributed optimization for large-scale high-dimensional sparse penalized Huber regression

Abstract

This paper provides a new perspective for robust estimation and inference methods based on Huber regression. An efficient distributed learning with sparsity is proposed, in which the observations are randomly divided among different machines. We modify the proximal alternating direction method of multiplies (ADMM) algorithm to calculate the sparse penalized Huber regression. The convergence property of the algorithm is established. Computationally, our proposed approach requires only a central machine to solve a SCAD or adaptive lasso penalized M-estimation problem, and other node machines to calculate gradients based on local datasets. In terms of communication, the estimation error of our method decreases to that of the centralized method in several rounds of communication. Simulation studies are conducted to evaluate the finite-sample performance of our proposed method. The empirical study shows that our method has good practicability in the practice of a Communities and Crime analysis.