Regularized quasi-monotone method for stochastic optimization

Abstract

We adapt the quasi-monotone method, an algorithm characterized by uniquely having convergence quality guarantees for the last iterate, for composite convex minimization in the stochastic setting. For the proposed numerical scheme we derive the optimal convergence rate of $$\text{ O }\left( \frac{1}{\sqrt{k+1}}\right)$$ in terms of the last iterate, rather than on average as it is standard for subgradient methods. The theoretical guarantee for individual convergence of the regularized quasi-monotone method is confirmed by numerical experiments on $$\ell _1$$ -regularized robust linear regression.