Distributed statistical estimation in quantile regression over a network

Abstract

In this paper, we consider distributed quantile regression over a decentralized network, which is an example of a popularly used non-smooth statistical model. Somewhat different from the viewpoint of the optimization literature, we jointly consider numerical convergence and statistical convergence when the objective function is the loss function constructed from i.i.d. data, and the convergence to the true population parameter (instead of the minimizer of the empirical risk) is emphasized. Distributed sub-gradient methods with and without gradient tracking are considered, with an emphasis on the former. With gradient tracking, the estimate linearly converges to the true parameter up to the statistical precision based on all data, when the local sample size is sufficiently large. The distinction of our result compared to the existing optimization literature is that linear convergence can be established despite the fact that the function is not strongly convex. Numerical illustrations are given to compare with distributed descent algorithm without gradient tracking, and compare the ATC (adapt-then-combine) version with the non-ATC one.