Convex Finite-Sum and Distributed Optimization

Abstract

In this chapter, we will study an important class of convex optimization problems whose objective function is given by the summation of many components. With wide applications in machine learning and distributed optimization, these problems can be viewed either as deterministic optimization problems with a special finite-sum structure or stochastic optimization problems with a discrete distribution. Accordingly, we will study two typical classes of randomized algorithms for solving them. While the first class incorporates random block decomposition into the dual space of the primal–dual methods for deterministic convex optimization, the second one employs variance reduction techniques into stochastic gradient descent methods for stochastic optimization.