Diminishing Step Size Research Articles

Gossip-based distributed stochastic mirror descent for constrained optimization

This paper considers a distributed constrained optimization problem over a multi-agent network in the non-Euclidean sense. The gossip protocol is adopted to relieve the communication burden, which also adapts to the constantly changing topology of the network. Based on this idea, a gossip-based distributed stochastic mirror descent (GB-DSMD) algorithm is proposed to handle the problem under consideration. The performances of GB-DSMD algorithms with constant and diminishing step sizes are analyzed, respectively. When the step size is constant, the error bound between the optimal function value and the expected function value corresponding to the average iteration output of the algorithm is derived. While for the case of the diminishing step size, it is proved that the output of the algorithm uniformly approaches to the optimal value with probability 1. Finally, as a numerical example, the distributed logistic regression is reported to demonstrate the effectiveness of the GB-DSMD algorithm.

The continuous stochastic gradient method: part I–convergence theory

In this contribution, we present a full overview of the continuous stochastic gradient (CSG) method, including convergence results, step size rules and algorithmic insights. We consider optimization problems in which the objective function requires some form of integration, e.g., expected values. Since approximating the integration by a fixed quadrature rule can introduce artificial local solutions into the problem while simultaneously raising the computational effort, stochastic optimization schemes have become increasingly popular in such contexts. However, known stochastic gradient type methods are typically limited to expected risk functions and inherently require many iterations. The latter is particularly problematic, if the evaluation of the cost function involves solving multiple state equations, given, e.g., in form of partial differential equations. To overcome these drawbacks, a recent article introduced the CSG method, which reuses old gradient sample information via the calculation of design dependent integration weights to obtain a better approximation to the full gradient. While in the original CSG paper convergence of a subsequence was established for a diminishing step size, here, we provide a complete convergence analysis of CSG for constant step sizes and an Armijo-type line search. Moreover, new methods to obtain the integration weights are presented, extending the application range of CSG to problems involving higher dimensional integrals and distributed data.

A generalized conditional gradient method for multiobjective composite optimization problems

This article deals with multiobjective composite optimization problems that consist of simultaneously minimizing several objective functions, each of which is composed of a combination of smooth and non-smooth functions. To tackle these problems, we propose a generalized version of the conditional gradient method, also known as Frank-Wolfe method. The method is analysed with three step size strategies, including Armijo-type, adaptive, and diminishing step sizes. We establish asymptotic convergence properties and iteration-complexity bounds, with and without convexity assumptions on the objective functions. Numerical experiments illustrating the practical behaviour of the methods are presented.

Proximal Stochastic Recursive Momentum Methods for Nonconvex Composite Decentralized Optimization

Consider a network of N decentralized computing agents collaboratively solving a nonconvex stochastic composite problem. In this work, we propose a single-loop algorithm, called DEEPSTORM, that achieves optimal sample complexity for this setting. Unlike double-loop algorithms that require a large batch size to compute the (stochastic) gradient once in a while, DEEPSTORM uses a small batch size, creating advantages in occasions such as streaming data and online learning. This is the first method achieving optimal sample complexity for decentralized nonconvex stochastic composite problems, requiring O(1) batch size. We conduct convergence analysis for DEEPSTORM with both constant and diminishing step sizes. Additionally, under proper initialization and a small enough desired solution error, we show that DEEPSTORM with a constant step size achieves a network-independent sample complexity, with an additional linear speed-up with respect to N over centralized methods. All codes are made available at https://github.com/gmancino/DEEPSTORM.

Convergence of Random Reshuffling under the Kurdyka–Łojasiewicz Inequality

We study the random reshuffling method for smooth nonconvex optimization problems with a finite-sum structure. Though this method is widely utilized in practice, e.g., in the training of neural networks, its convergence behavior is only understood in several limited settings. In this paper, under the well-known Kurdyka–Łojasiewicz (KL) inequality, we establish strong limit-point convergence results for with appropriate diminishing step sizes; namely, the whole sequence of iterates generated by is convergent and converges to a single stationary point in an almost sure sense. In addition, we derive the corresponding rate of convergence, depending on the KL exponent and suitably selected diminishing step sizes. When the KL exponent lies in , the convergence is at a rate of with counting the number of iterations. When the KL exponent belongs to , our derived convergence rate is of the form with depending on the KL exponent. The standard KL inequality-based convergence analysis framework only applies to algorithms with a certain descent property. We conduct a novel convergence analysis for the nondescent method with diminishing step sizes based on the KL inequality, which generalizes the standard KL framework. We summarize our main steps and core ideas in an informal analysis framework, which is of independent interest. As a direct application of this framework, we establish similar strong limit-point convergence results for the reshuffled proximal point method.

Single-objective and multi-objective optimization for variance counterbalancing in stochastic learning

Artificial neural networks have proved to be useful in a host of demanding applications, therefore becoming increasingly important in science and engineering. Large-scale problems constitute a challenging task for training neural networks using the stochastic gradient descent method and variations, which are based on the random selection of mini-batches of training points at every iteration. The challenge lies on the mandatory use of diminishing search step sizes in order to retain mild error fluctuations throughout the training set preserving so the quality of the network’s generalization capability. Variance counterbalancing was recently proposed as a remedy for addressing the diminishing step sizes in neural network training using stochastic gradient methods. It is based on the concurrent minimization of the average mean squared error of the network along with the error variance over random sets of mini-batches. Also, it promotes the use of advanced optimization algorithms instead of the slowly convergent gradient descent. The present work aims at enriching our understanding of the original variance counterbalancing approach, as well as reformulating it as a multi-objective problem by taking advantage of its bi-objective nature. Experimental analysis reveals the performances of the studied approaches and their competitive edge over the established Adam method.

Distributed delayed dual averaging for distributed optimization over time-varying digraphs

In this paper, a push-sum based distributed delayed dual averaging algorithm (PS-DDDA) is proposed to solve the distributed constrained optimization problem over the time-varying unbalanced directed graph (digraph). It considers the scenario in which each agent has delays while calculating the gradients and communicating. Both delays are assumed to be time-varying but bounded, which are more common in practice. Furthermore, we demonstrate that the cost function at the local state average converges to the optimal value and the local state average converges to the consensus with a rate of O (1/T) by utilizing the diminishing step size, where T is the total number of iterations. Moreover, to solve the distributed online constrained optimization problem, we propose the online version of PS-DDDA, and prove that its regret bound increases sublinearly with a rate of O (T). Finally, the effectiveness of the proposed algorithms is corroborated by solving logistic regression problems.

An Optimization Framework for Federated Edge Learning

The optimal design of federated learning (FL) algorithms for solving general machine learning (ML) problems in practical edge computing systems with quantized message passing remains an open problem. This paper considers an edge computing system where the server and workers have possibly different computing and communication capabilities and employ quantization before transmitting messages. To explore the full potential of FL in such an edge computing system, we first present a general FL algorithm, namely GenQSGD, parameterized by the numbers of global and local iterations, mini-batch size, and step size sequence. Then, we analyze its convergence for an arbitrary step size sequence and specify the convergence results under three commonly adopted step size rules, namely the constant, exponential, and diminishing step size rules. Next, we optimize the algorithm parameters to minimize the energy cost under the time constraint and convergence error constraint, with the focus on the overall implementing process of FL. Specifically, for any given step size sequence under each considered step size rule, we optimize the numbers of global and local iterations and mini-batch size to optimally implement FL for applications with preset step size sequences. We also optimize the step size sequence along with these algorithm parameters to explore the full potential of FL. The resulting optimization problems are challenging non-convex problems with non-differentiable constraint functions. We propose iterative algorithms to obtain KKT points using general inner approximation (GIA) and tricks for solving complementary geometric programming (CGP). Finally, we numerically demonstrate the remarkable gains of GenQSGD with optimized algorithm parameters over existing FL algorithms and reveal the significance of optimally designing general FL algorithms.

Zeroth-Order Non-Convex Optimization for Cooperative Multi-Agent Systems with Diminishing Step Size and Smoothing Radius

We study a class of zeroth-order distributed optimization problems, where each agent can control a partial vector and observe a local cost that depends on the joint vector of all agents, and the agents can communicate with each other with time delay. We propose and study a gradient descent-based algorithm using two-point gradient estimators with diminishing smoothing parameters and diminishing step-size and we establish the convergence rate to a first-order stationary point for general nonconvex problems. A byproduct of our proposed method with diminishing step size and smoothing parameters, as opposed to the fixed-parameter scheme, is that our proposed algorithm does not require any information regarding the local cost functions. This makes the solution appealing in practice as it allows for optimizing an unknown (black-box) global function without prior knowledge of its smoothness parameters. At the same time, the performance will adaptively match the problem instance parameters.

Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography

We propose a distributed Quantum State Tomography (QST) protocol, named Local Stochastic Factored Gradient Descent (Local SFGD), to learn the low-rank factor of a density matrix over a set of local machines. QST is the canonical procedure to characterize the state of a quantum system, which we formulate as a stochastic non-convex smooth optimization problem. Physically, the estimation of a low-rank density matrix helps characterizing the amount of noise introduced by quantum computation. Theoretically, we prove the local convergence of Local SFGD for a general class of restricted strongly convex/smooth loss functions. Local SFGD converges locally to a small neighborhood of the global optimum at a linear rate with a constant step size, while it locally converges exactly at a sub-linear rate with diminishing step sizes. With a proper initialization, local convergence results imply global convergence. We validate our theoretical findings with numerical simulations of QST on the Greenberger-Horne-Zeilinger (GHZ) state.

Distributed Convex Optimization With State-Dependent (Social) Interactions and Time-Varying Topologies

In this paper, an unconstrained collaborative optimization of a sum of convex functions is considered where agents make decisions using local information from their neighbors. The communication between nodes are described by a time-varying sequence of possibly state-dependent weighted networks. A new framework for modeling multi-agent optimization problems over networks with state-dependent interactions and time-varying topologies is proposed. A gradient-based discrete-time algorithm using diminishing step size is proposed for converging to the optimal solution under suitable assumptions. The algorithm is totally asynchronous without requiring B-connectivity assumption for convergence. The algorithm still works even if the weighted matrix of the graph is periodic and irreducible in synchronous protocol. Finally, a case study on a network of robots in an automated warehouse is provided in order to demonstrate the results.

Distributed Optimization Under Unbalanced Digraphs With Node Errors: Robustness of Surplus-Based Dual Averaging Algorithm

In this article, the robustness of distributed constrained optimization algorithms for weight-unbalanced directed multiagent networks is studied. Specifically, it is assumed that each agent is subject to additive random node errors , which are caused by the imperfect communication in networks. Under this framework, it is shown that a typical algorithm, called surplus-based dual averaging (SDA), can successfully achieve the convergence to the optimal value of the considered optimization problem, which exhibits the robustness to the node errors. Technically, in the proof of this result, an important augmentation of the node errors in the fusion and surplus variables is first introduced, and then by means of the random process theory and the attenuation effect of diminishing step size on node errors, the robustness of SDA is proved. In addition, inspired by the SDA, a robust algorithm is developed to solve the distributed online optimization problem with node errors. Finally, simulations are given to verify the theoretical results.

Network flows that solve least squares for linear equations

This paper presents a first-order distributed continuous-time algorithm for computing the least-squares solution to a linear equation over networks. Given the uniqueness of the solution, with nonintegrable and diminishing step size, convergence results are provided for fixed graphs. The exact rate of convergence is also established for various types of step size choices falling into that category. For the case where non-unique solutions exist, convergence to one such solution is proved for constantly connected switching graphs with square integrable step size. Validation of the results and illustration of the impact of step size on the convergence speed are made using a few numerical examples.

Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization

This paper addresses consensus optimization problems in a multi-agent network, where all agents collaboratively find a minimizer for the sum of their private functions. We develop a new decentralized algorithm in which each agent communicates only with its neighbors. State-of-the-art decentralized algorithms use communications between either all pairs of adjacent agents or a random subset of them at each iteration. Another class of algorithms uses a random walk incremental strategy, which sequentially activates a succession of nodes; these incremental algorithms require diminishing step sizes to converge to the solution, so their convergence is relatively slow. In this work, we propose a random walk algorithm that uses a fixed step size and converges faster than the existing random walk incremental algorithms. Our algorithm is also communication efficient. Each iteration uses only one link to communicate the latest information for an agent to another. Since this communication rule mimics a man walking around the network, we call our new algorithm Walkman. We establish convergence for convex and nonconvex objectives. For decentralized least squares, we derive a linear rate of convergence and obtain a better communication complexity than those of other decentralized algorithms. Numerical experiments verify our analysis results.

Nonconvex Robust Low-Rank Matrix Recovery

In this paper we study the problem of recovering a low-rank matrix from a number of random linear measurements that are corrupted by outliers taking arbitrary values. We consider a nonsmooth nonconvex formulation of the problem, in which we explicitly enforce the low-rank property of the solution by using a factored representation of the matrix variable and employ an $\ell_1$-loss function to robustify the solution against outliers. We show that even when a constant fraction (which can be up to almost half) of the measurements are arbitrarily corrupted, as long as certain measurement operators arising from the measurement model satisfy the so-called $\ell_1/\ell_2$-restricted isometry property, the ground-truth matrix can be exactly recovered from any global minimum of the resulting optimization problem. Furthermore, we show that the objective function of the optimization problem is sharp and weakly convex. Consequently, a subgradient Method (SubGM) with geometrically diminishing step sizes will converge linearly to the ground-truth matrix when suitably initialized. We demonstrate the efficacy of the SubGM for the nonconvex robust low-rank matrix recovery problem with various numerical experiments.

Distributed Multiagent Convex Optimization Over Random Digraphs

This paper considers an unconstrained collaborative optimization of a sum of convex functions, where agents make decisions using local information in the presence of random interconnection topologies. We recast the problem as minimization of the sum of convex functions over a constraint set defined as the set of fixed-value points of a random operator derived from weighted matrices of random graphs. We show that the derived random operator has nonexpansivity property; therefore, this formulation does not need the distribution of random communication topologies. Hence, it includes random networks with/without asynchronous protocols. As an extension of the problem, we define a novel optimization problem, namely minimization of a convex function over the fixed-value point set of a nonexpansive random operator. We propose a discrete-time algorithm using diminishing step size for converging almost surely and in mean square to the global solution of the optimization problem under suitable assumptions. Consequently, as a special case, it reduces to a totally asynchronous algorithm for the distributed optimization problem. We show that fixed-value point is a bridge from deterministic analysis to random analysis of the algorithm. Finally, a numerical example illustrates the convergence of the proposed algorithm.

On Global Linear Convergence in Stochastic Nonconvex Optimization for Semidefinite Programming

Nonconvex reformulations via low-rank factorization for stochastic convex semidefinite optimization problem have attracted arising attention due to their empirical efficiency and scalability. Compared with the original convex formulations, the nonconvex ones typically involve much fewer variables, allowing them to scale to scenarios with millions of variables. However, it opens a new challenge that under what conditions the nonconvex stochastic algorithms may find the population minimizer within the optimal statistical precision despite their empirical success in applications. In this paper, we provide an answer that the stochastic gradient descent (SGD) method can be adapted to solve the nonconvex reformulation of the original convex problem, with a global linear convergence when using a fixed step size, i.e., converging exponentially fast to the population minimizer within an optimal statistical precision in the restricted strongly convex case. If a diminishing step size is adopted, the bad effect caused by the variance of gradients on the optimization error can be eliminated but the rate is dropped to be sublinear. The core of our treatment relies on a novel second-order descent lemma , which is more general than the existing best result in the literature and improves the analysis on both online and batch algorithms. The developed theoretical results and effectiveness of the suggested SGD are also verified by a series of experiments.

On gradient projection methods for strongly pseudomonotone variational inequalities without Lipschitz continuity

In this paper, we propose two new self-adaptive algorithms for solving strongly pseudomonotone variational inequalities. Our algorithms use dynamic step-sizes, chosen based on information of the previous step and their strong convergence is proved without the Lipschitz continuity of the underlying mappings. In contrast to the existing self-adaptive algorithms in Bello Cruz et al. (Comput Optim Appl 46:247–263, 2010), Santos et al. (Comput Appl Math 30:91–107, 2011), which use fast diminishing step-sizes, the new algorithms use arbitrarily slowly diminishing step sizes. This feature helps to speed up our algorithms. Moreover, we provide the worst case complexity bound of the proposed algorithms, which have not been done in the previous works. Some preliminary numerical experiences and comparisons are also reported.

Distributed Randomized Gradient-Free Optimization Protocol of Multiagent Systems Over Weight-Unbalanced Digraphs.

In this paper, a distributed randomized gradient-free optimization protocol of multiagent systems over weight-unbalanced digraphs described by row-stochastic matrices is proposed to solve a distributed constrained convex optimization problem. Each agent possesses its local nonsmooth, but Lipschitz continuous, objective function and assigns the weight to information gathered from in-neighbor agents to update its decision state estimation, which is applicable and straightforward to implement. In addition, our algorithm relaxes the requirements of diminishing step sizes to only a nonsummable condition under convex bounded constraint sets. The boundedness and ultimate limit, instead of the supermartingale convergence theorem, are utilized to analyze the consistency and convergence and demonstrate convergence rates with different step sizes. Finally, the validity of the proposed algorithm is verified through numerical examples.

Distributed Energy Resource Coordination Over Time-Varying Directed Communication Networks

In this paper, we consider the optimal coordination problem for distributed energy resources (DERs), including distributed generators and energy storages. We first propose an algorithm based on the push-sum and gradient method to solve the optimal DER coordination problem in a distributed manner. In the proposed algorithm, each DER only maintains a set of variables and updates them through information exchange with a few neighboring DERs over a time-varying directed communication network. We show that the proposed distributed algorithm with appropriately chosen diminishing step sizes solves the optimal DER coordination problem if the time-varying directed communication network is uniformly jointly strongly connected. Moreover, in order to improve the convergence speed and to reduce the communication burden, we propose an accelerated distributed algorithm with a fixed step size. We show that the new proposed algorithm exponentially solves the optimal DER coordination problem if the cost functions satisfy an additional assumption and the selected step size is less than a certain critical value. Both proposed distributed algorithms are validated and evaluated using the IEEE 39-bus system.