Saddle Point Algorithm Research Articles

Decentralized multi-task stochastic optimization with compressed communications

We consider a multi-agent network where each node has a stochastic (local) cost function that depends on the decision variable of that node and a random variable, and further, the decision variables of neighboring nodes are pairwise constrained. There is an aggregated objective function for the network, composed additively of the expected values of the local cost functions at the nodes, and the overall goal of the network is to obtain the minimizing solution to this aggregate objective function subject to all the pairwise constraints. This is to be achieved at the level of the nodes using decentralized information and local computation, with exchanges of only compressed information allowed by neighboring nodes. The paper develops algorithms and obtains performance bounds for two different models of local information availability at the nodes: (i) sample feedback, where each node has direct access to samples of the local random variable to evaluate its local cost, and (ii) bandit feedback, where samples of the random variables are not available, but only the values of the local cost functions at two random points close to the decision are available to each node. For both models, with compressed communication between neighbors, we have developed decentralized saddle-point algorithms that deliver performances no different (in order sense) from those without communication compression; specifically, we show that deviation from the global minimum value and violations of the constraints are upper-bounded by O(T−12) and O(T−14), respectively, where T is the number of iterations. Numerical examples provided in the paper corroborate these bounds.

Weak constraint 4D-Var data assimilation in the Regional Ocean Modeling System (ROMS) using a saddle-point algorithm: Application to the California Current Circulation

The saddle-point formulation of weak constraint 4-dimensional variational (4D-Var) data assimilation has been developed for the Regional Ocean Modeling System (ROMS), and is applied here to the California Current System (CCS). Unlike the conventional primal and dual forcing formulation of weak constraint 4D-Var, the saddle-point formulation can be efficiently parallelized in time, leading to a substantial decrease in wall-clock run time. The performance of the ROMS saddle-point 4D-Var algorithm is assessed here and compared to that of the dual forcing formulation which is the current standard in ROMS. While the rate of convergence of the saddle-point formulation is slower than the dual forcing formulation, the increase in computational speed due to time-parallelization can more than compensate for the additional inner-loop iterations required by the saddle-point algorithm in the CCS configuration considered here. Further gains in performance can be achieved by running the 4D-Var inner-loop iterations at reduced resolution and/or reduced arithmetic precision. The results presented here indicate that in high performance computing environments with large numbers of available compute cores, the saddle-point formulation of 4D-Var could significantly reduce the run-time compared to the dual forcing formulation for large data assimilation problems.

Privacy masking distributed saddle-point algorithm for dynamic economic dispatch

Privacy masking distributed saddle-point algorithm for dynamic economic dispatch

Stochastic Compositional Gradient Descent Under Compositional Constraints

This work studies constrained stochastic optimization problems where the objective and constraint functions are convex and expressed as compositions of stochastic functions. The problem arises in the context of fair classification, fair regression, and the design of queuing systems. Of particular interest is the large-scale setting where an oracle provides the stochastic gradients of the constituent functions, and the goal is to solve the problem with a minimal number of calls to the oracle. Owing to the compositional form, the stochastic gradients provided by the oracle do not yield unbiased estimates of the objective or constraint gradients. Instead, we construct approximate gradients by tracking the inner function evaluations, resulting in a quasi-gradient saddle point algorithm. We prove that the proposed algorithm is guaranteed to find the optimal and feasible solution almost surely. We further establish that the proposed algorithm requires <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/\epsilon ^{4})$</tex-math></inline-formula> data samples in order to obtain an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -approximate optimal point while also ensuring zero constraint violation. The result matches the sample complexity of the stochastic compositional gradient descent method for unconstrained problems and improves upon the best-known sample complexity results for the constrained settings. The efficacy of the proposed algorithm is tested on both fair classification and fair regression problems. The numerical results show that the proposed algorithm outperforms the state-of-the-art algorithms in terms of the convergence rate.

Consensus-based Optimization in Multiplex Networks⋆

Multilayer networks are systems formed by several interacting networks. This framework generalizes the single-layer network scheme, developing a scenario where most real and engineering systems can be explored in a deeper and more accurately manner. Despite the wide set of results in distributed optimization in single-layer networks, there is a lack of developments in multilayer systems. In this paper, using the role of the supra-Laplacian matrix of a multiplex network and its diffusion dynamics, we develop a distributed primal-dual saddle-point algorithm, and a gradient descent algorithm for distributed optimization problems. Taking advantage of the relation between the consensus and diffusion dynamics via the graph Laplacian, we establish a set of saddle-point equations driven by the multiplex supra-Laplacian matrix, obtained by imposing an analogous to the traditional Laplacian restriction in our multiplex distributed optimization problem. This extension is performed to the distributed gradient descent algorithm for multiplex networks. Two theorems are proposed to study the convergence analysis for both methods, where it is demonstrated that both flows converge to the unique optimizer. We present numerical examples showing the validation of our results.

Distributed Constrained Online Convex Optimization Over Multiple Access Fading Channels

In this paper, we study distributed constrained online convex optimization for a wireless system consisting of a parameter server and multiple agents. Each agent has a local constraint function and a time-varying local loss function, and needs to choose sequential actions based on causal information. The goal of the overall system is to minimize the accumulated total loss of all agents over a time horizon subject to total constraints of the agents. To this end, the agents communicate with the server over multiple access noisy fading channels, where the information is exchanged imperfectly. We first consider the full information scenario, where the local loss function of each agent is fully revealed to the corresponding agent in each time slot. We propose a modified saddle-point algorithm, where each agent sends an analog signal pertaining to the current value of the local constraint function and the server receives a superposition of these signals distorted by the noisy fading channels. We analyze the performance of the proposed algorithm, and establish <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(\sqrt{T})$</tex-math></inline-formula> regret bound and <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(T^\frac{3}{4})$</tex-math></inline-formula> constraint violation bound for the algorithm, where <inline-formula><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the time horizon. Further, we extend the algorithm and performance analyses to the scenario of bandit feedback, where only the values of the local loss functions at two random points are disclosed to the agents in every time slot. In such a case, performance bounds similar to the full information scenario are established. Finally, numerical examples are presented to corroborate the efficacy of the proposed algorithms.

Constrained Online Convex Optimization With Feedback Delays

In this article, we study constrained online convex optimization (OCO) in the presence of feedback delays, where a decision maker chooses sequential actions without knowing the loss functions and constraint functions <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> . The loss/constraint functions vary with time and their feedback information is revealed to the decision maker with delays, which arise in many applications. We first consider the scenario of delayed function feedback, in which the complete information of the loss/constraint functions is revealed to the decision maker with delays. We develop a modified online saddle point algorithm suitable for constrained OCO with feedback delays. Sublinear regret and sublinear constraint violation bounds are established for the algorithm in terms of the delays. In practice, the complete information (functional forms) of the loss/constraint functions may not be revealed to the decision maker. Thus, we further examine the scenario of delayed bandit feedback, where only the values of the loss/constraint functions at two random points close to the chosen action are revealed to the decision maker with delays. A delayed version of the bandit online saddle point algorithm is proposed by utilizing stochastic gradient estimates of the loss/constraint functions based on delayed bandit feedback. We also establish sublinear regret and sublinear constraint violation bounds for this bandit optimization algorithm in terms of the delays. Finally, numerical results for online quadratically constrained quadratic programs are presented to corroborate the efficacy of the proposed algorithms.

Distributed Resource Allocation via Accelerated Saddle Point Dynamics

In this paper, accelerated saddle point dynamics is proposed for distributed resource allocation over a multi-agent network, which enables a hyper-exponential convergence rate. Specifically, an inertial fast-slow dynamical system with vanishing damping is introduced, based on which the distributed saddle point algorithm is designed. The dual variables are updated in two time scales, i.e., the fast manifold and the slow manifold. In the fast manifold, the consensus of the Lagrangian multipliers and the tracking of the constraints are pursued by the consensus protocol. In the slow manifold, the updating of the Lagrangian multipliers is accelerated by inertial terms. Hyper-exponential stability is defined to characterize a faster convergence of our proposed algorithm in comparison with conventional primal-dual algorithms for distributed resource allocation. The simulation of the application in the energy dispatch problem verifies the result, which demonstrates the fast convergence of the proposed saddle point dynamics.

Адаптивный метод операторной экстраполяции для вариационных неравенств в банаховых пространствах

Многие актуальные задачи исследования операций и математической физики могут быть записаны в форме вариационных неравенств. Разработка и исследование алгоритмов решения вариационных неравенств является направлением прикладного активно развивающегося нелинейного анализа. Отметим, что часто негладкие задачи оптимизации могут эффективно решаться, если переформулировать их в виде седловых задач и применить алгоритмы решения вариационных неравенств. В последнее время наметился прогресс в изучении алгоритмов для задач в банаховых пространствах. Это обусловлено широким привлечением результатов и конструкций геометрии банаховых пространств. В работе предложен и исследован новый алгоритм для разрешения вариационных неравенств в банаховом пространстве. Предлагаемый алгоритм является адаптивным вариантом «forward-reflected-backward algorithm», где используется правило обновления величины шага, не требующее знания лепшицевой константы оператора. Кроме того, вместо метрической проекции на допустимое множество используется обобщенная проекция Альбера. Преимуществом использования алгоритма является лишь одно вычисление на итерационном шаге проекции на допустимое множество. Для вариационных неравенств с монотонными, лепшицевыми операторами, действующими в 2-равномерно выпуклом и равномерно гладком банаховом пространстве, доказана теорема о слабой сходимости метода.

Global optimization-based dimer method for finding saddle points

<p style='text-indent:20px;'>Searching saddle points on the potential energy surface is a challenging problem in the rare event. When there exist multiple saddle points, sampling different initial guesses are needed in local search methods in order to find distinct saddle points. In this paper, we present a novel global optimization-based dimer method (GOD) to efficiently search saddle points by coupling ant colony optimization (ACO) algorithm with optimization-based shrinking dimer (OSD) method. In particular, we apply OSD method as a local search algorithm for saddle points and construct a pheromone function in ACO to update the global population. By applying a two-dimensional example and a benchmark problem of seven-atom island on the (111) surface of an FCC crystal, we demonstrate that GOD shows a significant improvement in computational efficiency compared with OSD method. Our algorithm is the first try to apply the global optimization technique in searching saddle points and it offers a new framework to open up possibilities of adopting other global optimization methods.

Input–Output Performance of Linear–Quadratic Saddle-Point Algorithms With Application to Distributed Resource Allocation Problems

Saddle-point or primal-dual methods have recently attracted renewed interest as a systematic technique to design distributed algorithms which solve convex optimization problems. When implemented online for streaming data or as dynamic feedback controllers, these algorithms become subject to disturbances and noise; convergence rates provide incomplete performance information, and quantifying input-output performance becomes more important. We analyze the input-output performance of the continuous-time saddle-point method applied to linearly constrained quadratic programs, providing explicit expressions for the saddle-point H2 norm under a relevant input-output configuration. We then proceed to derive analogous results for regularized and augmented versions of the saddle-point algorithm. We observe some rather peculiar effects -- a modest amount of regularization significantly improves the transient performance, while augmentation does not necessarily offer improvement. We then propose a distributed dual version of the algorithm which overcomes some of the performance limitations imposed by augmentation. Finally, we apply our results to a resource allocation problem to compare the input-output performance of various centralized and distributed saddle-point implementations and show that distributed algorithms may perform as well as their centralized counterparts.

Decentralized Multi-Agent Stochastic Optimization With Pairwise Constraints and Quantized Communications

Decentralized optimization methods often entail information exchange between neighbors. In many circumstances, due to the limited communication bandwidth, the exchanged information has to be quantized. In this paper, we investigate the impact of quantization on the performance of decentralized stochastic optimization. We consider a multi-agent network, in which each node is associated with a stochastic cost and each pair of neighbors is associated with a constraint. The goal of the network is to minimize the aggregate expected cost subject to all the constraints. We first develop a decentralized stochastic saddle point algorithm with quantized communications for the scenario of sample feedback, in which a sample of the random variable in the stochastic cost function is revealed to each node at each time. We establish performance bounds for the expected cost suboptimality and expected constraint violations of the algorithm in terms of the quantization intervals. We show that asymptotic optimality can be achieved if the quantization intervals approach zero. On the other hand, if the quantization intervals are fixed and nonzero, then non-diminishing performance gaps exist, which indicate a tradeoff between optimization performance and communication overhead. We further extend the algorithm and analysis to the scenario of bandit feedback, where samples of the random variables are not available and only the values of the cost function at two random points close to the current decision are disclosed. We show that the performance of the algorithm does not degrade in order sense compared to its counterpart with sample feedback. Finally, two numerical examples, namely decentralized quadratically constrained quadratic program and decentralized logistic regression, are presented to corroborate the efficacy of the proposed algorithms.

Atomistic approach to simulate kink migration and kink-pair formation in silicon: The kinetic activation-relaxation technique

The energy conversion efficiency of solar cells based on multicrystalline silicon is greatly deteriorated by dislocations. However, an in-depth understanding on the dislocation motion dynamics down to atomic scale is still lacking. In this paper, we propose a novel atomistic approach to simulate the kink migration and kink-pair formation which govern dislocation motion in silicon, namely the kinetic activation-relax technique (k-ART). With this method, long timescale events can be simulated and complex energy landscapes can be explored. Four mechanisms for kink migration are observed, with total activation energy of 0.16, 0.25, 0.32, and 0.25 eV. New nontrivial kink structures that participate in kink migration are identified due to the open-ended search algorithm for saddle points in k-ART. In addition, a new pathway for kink-pair formation, with a minimum activation energy of 1.11 eV is discovered. The effect of shear stress on kink migration is also investigated. It shows that shear stress shifts the energy barriers of available events to lower energies, resulting in a change of the preferred kink-migration mechanism and a reduction of kink-pair formation energy.

Asynchronous Online Learning in Multi-Agent Systems With Proximity Constraints

We consider the problem of distributed learning from sequential data via online convex optimization. A multi-agent system is considered where each agent has a private objective but is willing to cooperate in order to minimize the network cost, which is the sum of local cost functions. Different from the classical distributed settings, where the agents coordinate through the use of consensus constraints, we allow the neighboring agent actions to be related via a non-linear proximity function. A decentralized saddle point algorithm is proposed that is capable of handling gradient delays arising from computational issues. The proposed online asynchronous algorithm is analyzed under adversarial settings by developing bounds on the regret of O(√T), which measures the cumulative loss incurred by the online algorithm against a clairvoyant, and network discrepancy of O(T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3/4</sup> ), which measures the cumulative constraint violation or agent disagreement. By allowing the agents to utilize stale gradient information, the proposed algorithm embraces the nuances of distributed learning and serves to be the first distributed online algorithm that can handle adversarial delays. A modified saddle point algorithm is also proposed that explicitly forces the agents to agree as per the constraint function resulting in zero network discrepancy while incurring a slightly higher regret. To showcase the efficacy of the proposed asynchronous algorithm, a spatially correlated random field estimation problem is formulated and solved. Additionally, an application of vision-based target localization with moving cameras demonstrates the benefits of this approach in practice.

Asynchronous Saddle Point Algorithm for Stochastic Optimization in Heterogeneous Networks

We consider expected risk minimization in multi-agent systems comprised of distinct subsets of agents operating without a common time scale. Each individual in the network is charged with minimizing the global objective function, which is an average of sum of the statistical average loss function of each agent in the network. Since agents are not assumed to observe data from identical distributions, the hypothesis that all agents seek a common action is violated, and thus the hypothesis upon that consensus constraints are formulated is violated. Thus, we consider nonlinear network proximity constraints, which incentivize nearby nodes to make decisions that are close to one another but do not necessarily coincide. Moreover, agents are not assumed to receive their sequentially arriving observations on a common time index, and thus seek to learn in an asynchronous manner. An asynchronous stochastic variant of the Arrow–Hurwicz saddle point method is proposed to solve this problem that operates by alternating primal stochastic descent steps and Lagrange multiplier updates that penalize the discrepancies between agents. This tool leads to an implementation that allows for each agent to operate asynchronously with local information only and message passing with neighbors. Our main result establishes that the proposed method yields convergence in expectation both in terms of the primal sub-optimality and constraint violation to radii of sizes ${\mathcal O}(\sqrt{T})$ and ${\mathcal O}(T^{3/4})$ , respectively. Empirical evaluation on an asynchronously operating wireless network that manages user channel interference through an adaptive communications pricing mechanism demonstrates that our theoretical results translates well to practice.

Distributed Constrained Online Learning

In this paper, we consider groups of agents in a network that select actions in order to satisfy a set of constraints that vary arbitrarily over time and minimize a time-varying function of which they have only local observations. The selection of actions, also called a strategy, is causal and decentralized, i.e., the dynamical system that determines the actions of a given agent depends only on the constraints at the current time and on its own actions and those of its neighbors. To determine such a strategy, we propose a decentralized saddle point algorithm and show that the corresponding global fit and regret are bounded by functions of the order of $\sqrt{T}$. Specifically, we define the global fit of a strategy as a vector that integrates over time the global constraint violations as seen by a given node. The fit is a performance loss associated with online operation as opposed to offline clairvoyant operation which can always select an action if one exists, that satisfies the constraints at all times. If this fit grows sublinearly with the time horizon it suggests that the strategy approaches the feasible set of actions. Likewise, we define the regret of a strategy as the difference between its accumulated cost and that of the best fixed action that one could select knowing beforehand the time evolution of the objective function. Numerical examples support the theoretical conclusions.

D4L: Decentralized Dynamic Discriminative Dictionary Learning

We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn, from sequential observations, statistical model parameters jointly with data-driven signal representations. We formulate this problem as a distributed stochastic program with a nonconvex objective that quantifies the merit of the choice of model parameters and dictionary. We consider the use of a block variant of the Arrow–Hurwicz saddle point algorithm to solve this problem, which exploits factorization properties of the Lagrangian to yield a protocol in that only requires exchange of model information among neighboring nodes. We show that decisions made with this saddle point algorithm asymptotically achieve a first-order stationarity condition on average. The learning rate depends on the signal source, network structure, and discriminative task. We illustrate the algorithm performance for solving a large-scale image classification task on a network of interconnected servers and observe that practical performance is comparable to a centralized approach. We further apply this method to the problem of a robotic team seeking to autonomously navigate in an unknown environment by predicting unexpected maneuvers, demonstrating the proposed algorithm's utility in a field setting.

Distributed Optimization over Directed Graphs with the help of Lie Brackets

This work addresses the problem of solving distributed optimization problems over directed graphs. Using techniques from geometric control theory, a novel methodology is proposed to design continuous-time distributed algorithms that approximate the solutions to the saddle-point algorithms over directed graphs. Graph-theoretic conditions are established under which the proposed methodology provides a distributed solution.

Proximity Without Consensus in Online Multiagent Optimization

We consider stochastic optimization problems in multi-agent settings, where a network of agents aims to learn parameters which are optimal in terms of a global objective, while giving preference to locally observed streaming information. To do so, we depart from the canonical decentralized optimization framework where agreement constraints are enforced, and instead formulate a problem where each agent minimizes a global objective while enforcing network proximity constraints. This formulation includes online consensus optimization as a special case, but allows for the more general hypothesis that there is data heterogeneity across the network. To solve this problem, we propose using a stochastic saddle point algorithm inspired by Arrow and Hurwicz. This method yields a decentralized algorithm for processing observations sequentially received at each node of the network. Using Lagrange multipliers to penalize the discrepancy between them, only neighboring nodes exchange model information. We establish that under a constant step-size regime the time-average suboptimality and constraint violation are contained in a neighborhood whose radius vanishes with increasing number of iterations. As a consequence, we prove that the time-average primal vectors converge to the optimal objective while satisfying the network proximity constraints. We apply this method to the problem of sequentially estimating a correlated random field in a sensor network, as well as an online source localization problem, both of which demonstrate the empirical validity of the aforementioned convergence results.

A Saddle Point Algorithm for Networked Online Convex Optimization

An algorithm to learn optimal actions in convex distributed online problems is developed. Learning is online because cost functions are revealed sequentially and distributed because they are revealed to agents of a network that can exchange information with neighboring nodes only. Learning is measured in terms of the global network regret, which is defined here as the accumulated loss of causal prediction with respect to a centralized clairvoyant agent to which the information of all times and agents is revealed at the initial time. A variant of the Arrow–Hurwicz saddle point algorithm is proposed to control the growth of global network regret. This algorithm uses Lagrange multipliers to penalize the discrepancies between agents and leads to an implementation that relies on local operations and exchange of variables between neighbors. We show that decisions made with this saddle point algorithm lead to regret whose order is not larger than $O(\sqrt{T})$ , where $T$ is the total operating time. Numerical behavior is illustrated for the particular case of distributed recursive least squares. An application to computer network security in which service providers cooperate to detect the signature of malicious users is developed to illustrate the practical value of the proposed algorithm.