A Newton Tracking Algorithm With Exact Linear Convergence for Decentralized Consensus Optimization

Abstract

This paper considers the problem of decentralized consensus optimization over a network, where each node holds a strongly convex and twice-differentiable local objective function. Our goal is to minimize the sum of the local objective functions and find the exact optimal solution using only local computation and neighboring communication. We propose a novel Newton tracking algorithm, which updates the local variable in each node along a local Newton direction modified with neighboring and historical information. We investigate the connections between the proposed Newton tracking algorithm and several existing methods, including gradient tracking and primal-dual methods. We prove that the proposed algorithm converges to the exact optimal solution at a linear rate. Furthermore, when the iterate is close to the optimal solution, we show that the proposed algorithm requires $O(\max \lbrace \kappa _f \sqrt{\kappa _g} + \kappa _f^2, \frac{\kappa _g^{3/2}}{\kappa _f} + \kappa _f\sqrt{\kappa _g} \rbrace \log {\frac{1}{\Delta }})$ iterations to find a $\Delta$ -optimal solution, where $\kappa _f$ and $\kappa _g$ are condition numbers of the objective function and the graph, respectively. Our numerical results demonstrate the efficacy of Newton tracking and validate the theoretical findings.