Convergence of first-order methods via the convex conjugate

Abstract

This paper gives a unified and succinct approach to the O(1∕k),O(1∕k), and O(1∕k2) convergence rates of the subgradient, gradient, and accelerated gradient methods for unconstrained convex minimization. In the three cases the proof of convergence follows from a generic bound defined by the convex conjugate of the objective function.