Incremental Subgradient Algorithm Research Articles

A Markovian Incremental Stochastic Subgradient Algorithm

In this article, a stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information, and the sequence of partial subgradients is determined by a general Markov chain. This makes it suitable to be used in networks, where the path of information flow is stochastically selected. We prove convergence of the algorithm to a weighted objective function, where the weights are given by the Cesàro limiting probability distribution of the Markov chain. Unlike previous works in the literature, the Cesàro limiting distribution is general (not necessarily uniform), allowing for general weighted objective functions and flexibility in the method.

Incremental subgradient algorithms with dynamic step sizes for separable convex optimizations

We consider the incremental subgradient algorithm employing dynamic step sizes for minimizing the sum of a large number of component convex functions. The dynamic step size rule was firstly introduced by Goffin and Kiwiel [Math. Program., 1999, 85(1): 207‐211] for the subgradient algorithm, soon later, for the incremental subgradient algorithm by Nedic and Bertsekas in [SIAM J. Optim., 2001, 12(1): 109‐138]. It was observed experimentally that the incremental approach has been very successful in solving large separable optimizations and that the dynamic step sizes generally have better computational performance than others in the literature. In the present paper, we propose two modified dynamic step size rules for the incremental subgradient algorithm and analyse the convergence and complexity properties of them. At last, the assignment problem is considered and the incremental subgradient algorithms employing different kinds of dynamic step sizes are applied to solve the problem. The computational experiments show that the two modified ones converges dramatically faster and more stable than the corresponding one in [SIAM J. Optim., 2001, 12(1): 109‐138]. Particularly, for solving large separable convex optimizations, we strongly recommend the second one (see Algorithm 3.3 in the paper) since it has interesting computational performance and is the simplest one.

Incremental and Parallel Machine Learning Algorithms With Automated Learning Rate Adjustments.

The existing machine learning algorithms for minimizing the convex function over a closed convex set suffer from slow convergence because their learning rates must be determined before running them. This paper proposes two machine learning algorithms incorporating the line search method, which automatically and algorithmically finds appropriate learning rates at run-time. One algorithm is based on the incremental subgradient algorithm, which sequentially and cyclically uses each of the parts of the objective function; the other is based on the parallel subgradient algorithm, which uses parts independently in parallel. These algorithms can be applied to constrained nonsmooth convex optimization problems appearing in tasks of learning support vector machines without adjusting the learning rates precisely. The proposed line search method can determine learning rates to satisfy weaker conditions than the ones used in the existing machine learning algorithms. This implies that the two algorithms are generalizations of the existing incremental and parallel subgradient algorithms for solving constrained nonsmooth convex optimization problems. We show that they generate sequences that converge to a solution of the constrained nonsmooth convex optimization problem under certain conditions. The main contribution of this paper is the provision of three kinds of experiment showing that the two algorithms can solve concrete experimental problems faster than the existing algorithms. First, we show that the proposed algorithms have performance advantages over the existing ones in solving a test problem. Second, we compare the proposed algorithms with a different algorithm Pegasos, which is designed to learn with a support vector machine efficiently, in terms of prediction accuracy, value of the objective function, and computational time. Finally, we use one of our algorithms to train a multilayer neural network and discuss its applicability to deep learning.

Normalized Incremental Subgradient Algorithm and Its Application

The problem of minimizing the sum of a number of component functions is of great importance in the real world. In this paper, a new incremental optimization algorithm, named normalized incremental subgradient (NIS) algorithm, is proposed for a class of such problems where the component functions have common local minima. The NIS algorithm is performed incrementally just as the general incremental subgradient (IS) algorithm and thus can be implemented in a distributed way. In the NIS algorithm, the update of each subiteration is based on a search direction obtained by individually normalizing each component of subgradients of component functions, resulting in much better convergence performance as compared to the IS algorithm and other traditional optimization methods (e.g., Gauss-Newton method). The convergence of the NIS algorithm with both diminishing stepsizes and constant stepsizes is proved and analyzed theoretically. Two important applications are presented. One is to solve a class of convex feasibility problems in a distributed way and the other is distributed maximum likelihood estimation. Numerical examples, arising from two important topics in the area of wireless sensor networks-source localization and node localization-demonstrate the effectiveness and efficiency of the NIS algorithm.

Incremental Stochastic Subgradient Algorithms for Convex Optimization

This paper studies the effect of stochastic errors on two constrained incremental subgradient algorithms. The incremental subgradient algorithms are viewed as decentralized network optimization algorithms as applied to minimize a sum of functions, when each component function is known only to a particular agent of a distributed network. First, the standard cyclic incremental subgradient algorithm is studied. In this, the agents form a ring structure and pass the iterate in a cycle. When there are stochastic errors in the subgradient evaluations, sufficient conditions on the moments of the stochastic errors are obtained that guarantee almost sure convergence when a diminishing step-size is used. In addition, almost sure bounds on the algorithm's performance with a constant step-size are also obtained. Next, the Markov randomized incremental subgradient method is studied. This is a noncyclic version of the incremental algorithm where the sequence of computing agents is modeled as a time nonhomogeneous Markov chain. Such a model is appropriate for mobile networks, as the network topology changes across time in these networks. Convergence results and error bounds for the Markov randomized method in the presence of stochastic errors for diminishing and constant step-sizes are obtained.