Online Convex Optimization Problems Research Articles

Distributed online constrained convex optimization with event-triggered communication

This paper focuses on the distributed online convex optimization problem with time-varying inequality constraints over a network of agents, where each agent collaborates with its neighboring agents to minimize the cumulative network-wide loss over time. To reduce communication overhead between the agents, we propose a distributed event-triggered online primal–dual algorithm over a time-varying directed graph. With several classes of appropriately chose decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. Finally, a numerical simulation example is provided to verify the theoretical results.

Distributed online primal-dual subgradient method on unbalanced directed networks

In this paper, we investigate a distributed online optimization method on multiagent communication networks. We consider a distributed prime-dual subgradient algorithm for an online convex optimization problem with time-varying coupled constraints. Each agent updates the estimations for the primal and dual optimizers by a consensus-based online algorithm. In the proposed algorithm, the gradient direction is scaled by estimating the left eigenvector of a weight matrix associated with the directed communication network. The scaling procedure enables agents in the network to estimate the time-varying optimal solutions by counterbalancing the unbalanced communication flows. The performance of the proposed algorithm is examined by a dynamic regret and a fit, which evaluate the cumulative error against the time-varying optimal cost function and the constraint function, respectively. We provide a sufficient condition under which both the dynamic regret and the fit are sublinear. A numerical example of an online economic dispatch problem confirms the validity of the proposed method.

Distributed online optimization for heterogeneous linear multi-agent systems with coupled constraints

This paper studies a class of distributed online convex optimization problems for heterogeneous linear multi-agent systems. Agents in a network, knowing only their own outputs, need to minimize the time-varying costs through neighboring interaction subject to time-varying coupled inequality constraints. Based on the saddle-point technique, we design a continuous-time distributed controller which is shown to achieve constant regret bound and sublinear fit bound, matching those of the standard centralized online method. We further extend the control law to the event-triggered communication mechanism and show that the constant regret bound and sublinear fit bound are still achieved while reducing the communication frequency. Additionally, we study the situation of communication noise, i.e., the agent’s measurement of the relative states of its neighbors is disturbed by a noise. It is shown that, if the noise is not excessive, the regret and fit bounds are unaffected, which indicates the controller’s noise-tolerance capability to some extent. Finally, a numerical simulation is provided to support the theoretical conclusions.

Distributed online convex optimization with multiple coupled constraints: A double accelerated push–pull algorithm

This paper is concerned with the distributed online convex optimization problems with multiple coupled constraints over unbalanced digraphs, which are widely applied in diverse fields such as machine learning, smart grids, and resource allocation problems, etc. In such practical scenarios, multiple coupled constraints, where each coupled constraint might only contain a portion of all nodes, are more general and complex than global coupled constraints. Additionally, due to certain constraint conditions such as energy and communication limitations, each node can only exchange information with other nodes in an identical constraint. Besides, the network connectivity of practical constrained problems might not be only confined to one type of weight matrices. By adopting an augmented Lagrangian method, the multiple coupled constrained optimization problem is converted into a saddle-point problem. To tackle this problem, a novel double accelerated push–pull (DAPP) algorithm is proposed which concurrently employs row-stochastic (RS) and column-stochastic (CS) weight matrices. Specifically, each node pushes gradient information weighted by CS matrices to out-neighbors, while pulls dual variables fused by RS matrices from in-neighbors. Combined with CS matrices, the gradient tracking scheme is employed to promote the efficiency of tracking the averaged gradients. Furthermore, aiming at improving the convergence rate, the heavy-ball method is used to update both primal and dual variables. Then, rigorous theoretical results indicate that DAPP obtains the sublinear bounds of regret and constraint violation. Finally, the plug-in electric vehicles (PEVs) charging problem is utilized to demonstrate the validity of the proposed algorithm.

Online convex optimization with switching cost and delayed gradients

We consider the online convex optimization (OCO) problem with quadratic and linear switching cost in the limited information setting, where an online algorithm can choose its action using only gradient information about the previous objective function. For L-smooth and μ-strongly convex objective functions, we propose an online multiple gradient descent (OMGD) algorithm and show that its competitive ratio for the OCO problem with quadratic switching cost is at most 4(L+5)+16(L+5)μ. The competitive ratio upper bound for OMGD is also shown to be order-wise tight in terms of L,μ. In addition, we show that the competitive ratio of any online algorithm is max{Ω(L),Ω(Lμ)} in the limited information setting when the switching cost is quadratic. We also show that the OMGD algorithm achieves the optimal (order-wise) dynamic regret in the limited information setting. For the linear switching cost, the competitive ratio upper bound of the OMGD algorithm is shown to depend on both the path length and the squared path length of the problem instance, in addition to L,μ, and is shown to be order-wise, the best competitive ratio any online algorithm can achieve. Consequently, we conclude that the optimal competitive ratio for the quadratic and linear switching costs are fundamentally different in the limited information setting.

Composite Zero-Effort-Miss Guidance with Predictor–Corrector Steps for Ballistic Interception

In this paper, a midcourse guidance law for exoatmospheric interception using a solid-propellant rocket is presented. Existing guidance laws based on the zero-effort-miss concept lack consideration for remaining thrust, potentially resulting in the excessive magnitude and rate of the thrust direction command. To address this issue, the zero-effort-miss concept is combined with an iterative predictor–corrector-based guidance law that is adapted for solid-propellant rockets. To apply the iterative predictor–corrector-based guidance law to the interception problem with a solid-propellant rocket, a composite zero-effort-miss concept is introduced, and the correction step is formulated as an online convex optimization problem. The composite zero-effort-miss consists of two zero-effort-misses. One miss is obtained from current state and the other miss is obtained from predicted state at burnout. Those render the proposed guidance law robust to thrust uncertainty allow for a noniterative approach and eliminate the singularity problem. Also, the proposed guidance law reduces maneuvering during the early part of the midcourse phase, leading to less velocity loss, and resolves intercept performance degradation due to the rate limit of thrust vectoring. Numerical simulations comparing the proposed method with the modified zero-effort-miss guidance demonstrate its effectiveness and robustness.

Decentralized online convex optimization with compressed communications

Due to the iterative information exchange between agents, decentralized multi-agent optimization algorithms often incur large communication overhead, which is not affordable in many practical systems with scarce resources. To address this communication bottleneck, we compress the exchanged information between neighboring agents to solve decentralized online convex optimization problem, where each agent has a time-varying local loss function and the goal is to minimize the accumulated global loss by choosing actions sequentially. We develop a decentralized online gradient descent algorithm based on compressed communications, where the compression operators can reduce the amount of data to be sent significantly. Error-compensation technique is utilized to mitigate the error accumulation caused by the communication compression. We further analyze the performance of the proposed algorithm. It is shown that the regret is bounded from above by O(T) (T is the time horizon), which is the same as (in order sense) the regret bound for vanilla decentralized online gradient descent with perfect communication. Thus, the proposed algorithm is capable of reducing the communication overhead remarkably without much degradation of the optimization performance. This is further corroborated by numerical experiments, where compressed communication reduces the number of transmitted bits by an order of magnitude without compromising the optimization performance.

Differentially private distributed online mirror descent algorithm

This paper examines a private distributed online convex optimization problem in which each agent strives to minimize the sum of objective functions while also tending to keep their individual objective functions confidential. We use differential privacy as the metric to safeguard each agent’s privacy and offer a distributed online mirror descent technique that is differentially private. We demonstrate that for strongly convex objective functions, our proposed algorithm satisfies the expected regret bound of OlnT while maintaining differential privacy, where T is the number of iterations. The established expected regret bound matches the optimal theoretical regret bound with respect to T. We further illuminate the trade-off between the extent of privacy-preserving and the expected regret bound through numerical simulations.

Regret and Cumulative Constraint Violation Analysis for Distributed Online Constrained Convex Optimization

This paper considers the distributed online convex optimization problem with time-varying constraints over a network of agents. This is a sequential decision making problem with two sequences of arbitrarily varying convex loss and constraint functions. At each round, each agent selects a decision from the decision set, and then only a portion of the loss function and a coordinate block of the constraint function at this round are privately revealed to this agent. The goal of the network is to minimize the network-wide loss accumulated over time. Two distributed online algorithms with full-information and bandit feedback are proposed. Both dynamic and static network regret bounds are analyzed for the proposed algorithms, and network cumulative constraint violation is used to measure constraint violation, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. In particular, we show that the proposed algorithms achieve $\mathcal{O}(T^{\max\{\kappa,1-\kappa\}})$ static network regret and $\mathcal{O}(T^{1-\kappa/2})$ network cumulative constraint violation, where $T$ is the time horizon and $\kappa\in(0,1)$ is a user-defined trade-off parameter. Moreover, if the loss functions are strongly convex, then the static network regret bound can be reduced to $\mathcal{O}(T^{\kappa})$. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Distributed online convex optimization with a bandit primal-dual mirror descent push-sum algorithm

The distributed online convex optimization problem with time-varying constraints for multi-agent networks is addressed in this article. The purpose is to optimize a sequence of time-varying global cost functions defined as the accumulated values of local cost functions, also attempt to meet the requirement of a sequence of time-varying coupled constraint functions which denote the sum of local constraint functions. Cost functions and constraint functions are unknown to agents beforehand. It is supposed that each agent in the network communicates with its neighbours through a uniformly strongly connected sequence of time-varying directed communication topologies. This paper proposes the bandit distributed primal-dual mirror descent push-sum (BDPDMDPS) algorithm constructed by bandit primal-dual, mirror descent and push-sum methods. Operational performance of the presented algorithm is measured by expected regret and expected constraint violation, both of which are proved to be sublinear with respect to the total iteration span T in this paper. Finally, a numerical simulation example is shown, which confirms the results for expected regret and expected constraint violation of BDPDMDPS algorithm.

Online Ensemble Approach for Probabilistic Wind Power Forecasting

Probabilistic wind power forecasting is an important input in the decision-making process in future electric power grids with large penetrations of renewable generation. Traditional probabilistic wind power forecasting models are trained offline and are then used to make predictions online. However, this strategy cannot make full use of the most recent information during the prediction process. In addition, ensemble learning is recognized as an effective approach for further improving forecasting performance by combining multiple forecasting models. This paper studies an online ensemble approach for probabilistic wind power forecasting by taking full advantage of the most recent information and leveraging the strengths of multiple forecasting models. The online ensemble approach is first formulated as an online convex optimization problem. On this basis, a quantile passive-aggressive regression model is proposed to solve the online convex optimization problem. Case studies and comparisons with other online learning methods are conducted on an open wind power data set from Belgium. Results show that the proposed method outperforms competing methods in terms of pinball loss and Winkler score with high reliability.

Simultaneously achieving sublinear regret and constraint violations for online convex optimization with time-varying constraints

In this paper, we develop a novel virtual-queue-based online algorithm for online convex optimization (OCO) problems with long-term and time-varying constraints and conduct a performance analysis with respect to the dynamic regret and constraint violations. We design a new update rule of dual variables and a new way of incorporating time-varying constraint functions into the dual variables. To the best of our knowledge, our algorithm is the first parameter-free algorithm to simultaneously achieve sublinear dynamic regret and constraint violations. Our proposed algorithm also outperforms the state-of-the-art results in many aspects, e.g., our algorithm does not require the Slater condition. Meanwhile, for a group of practical and widely-studied constrained OCO problems in which the variation of consecutive constraints is smooth enough across time, our algorithm achieves O(1) constraint violations. Furthermore, we extend our algorithm and analysis to the case when the time horizon T is unknown. Finally, numerical experiments are conducted to validate the theoretical guarantees of our algorithm, and some applications of our proposed framework will be outlined.

A Primal-Dual Online Algorithm for Online Matching Problem in Dynamic Environments

Recently, the online matching problem has attracted much attention due to its wide application on real-world decision-making scenarios. In stationary environments, by adopting the stochastic user arrival model, existing methods are proposed to learn dual optimal prices and are shown to achieve a fast regret bound. However, the stochastic model is no longer a proper assumption when the environment is changing, leading to an optimistic method that may suffer poor performance. In this paper, we study the online matching problem in dynamic environments in which the dual optimal prices are allowed to vary over time. We bound the dynamic regret of online matching problem by the sum of two quantities, including a regret of online max-min problem and a dynamic regret of online convex optimization (OCO) problem. Then we propose a novel online approach named Primal-Dual Online Algorithm (PDOA) to minimize both quantities. In particular, PDOA adopts the primal-dual framework by optimizing dual prices with the online gradient descent (OGD) algorithm to eliminate the online max-min problem's regret. Moreover, it maintains a set of OGD experts and combines them via an expert-tracking algorithm, which gives a sublinear dynamic regret bound for the OCO problem. We show that PDOA achieves an O(K sqrt{T(1+P_T)}) dynamic regret where K is the number of resources, T is the number of iterations and P_T is the path-length of any potential dual price sequence that reflects the dynamic environment. Finally, experiments on real applications exhibit the superiority of our approach.

Decentralized online convex optimization based on signs of relative states

In this paper, we study a class of decentralized online convex optimization problems with time-varying loss functions over multi-agent networks. We propose a decentralized online subgradient method by using only the signs of the relative states of neighbors, which considerably reduces the sensing and communication requirements for the agents. We show that, despite the loss of information, the proposed algorithm can still achieve O(T) regret bound (T is the time horizon), which matches that of the standard distributed online subgradient method with exact relative state information. We further investigate the scenario of using noisy signs, where the measurements of the directions of the relative states are perturbed by noise. We show that the regret bound is not affected as long as the noise is not too large, which manifests certain noise-tolerance property of the proposed algorithm. Additionally, we extend the algorithm to the case of bandit feedback, where only the values of the local loss functions at two points are revealed to agents at each time. We demonstrate that the regret bound is not influenced by bandit feedback in order sense. Finally, numerical experiments on decentralized online least squares and logistic regression are conducted to corroborate the efficacy of the proposed algorithms.

Competitive Online Convex Optimization With Switching Costs and Ramp Constraints

We investigate competitive online algorithms for online convex optimization (OCO) problems with linear in-stage costs, switching costs and ramp constraints. While OCO problems have been extensively studied in the literature, there are limited results on the corresponding online solutions that can attain small competitive ratios. We first develop a powerful computational framework that can compute an optimized competitive ratio based on the class of affine policies. Our computational framework can handle a fairly general class of costs and constraints. Compared with other competitive results in the literature, a key feature of our proposed approach is that it can handle scenarios where infeasibility may arise due to hard feasibility constraints. Second, we design a robustification procedure to produce an online algorithm that can attain good performance for both average-case and worst-case inputs. We conduct a case study on Network Functions Virtualization (NFV) orchestration and scaling to demonstrate the effectiveness of our proposed methods.

Distributed adaptive online learning for convex optimization with weight decay

AbstractThis paper investigates an adaptive gradient‐based online convex optimization problem over decentralized networks. The nodes of a network aim to track the minimizer of a global time‐varying convex function, and the communication pattern among nodes is captured as a connected undirected graph. To tackle such optimization problems in a collaborative and distributed manner, a weight decay distributed adaptive online gradient algorithm, called WDDAOG, is firstly proposed, which incorporates distributed optimization methods with adaptive strategies. Then, our theoretical analysis clearly illustrates the difference between weight decay and L2 regularization for distributed adaptive gradient algorithms. The dynamic regret bound for the proposed algorithm is further analyzed. It is shown that the dynamic regret bound for convex functions grows with order of , where T and n represent the time horizon and the number of nodes associated with the network, respectively. Numerical experiments demonstrate that WDDAOG works well in practice and compares favorably to existing distributed online optimization schemes.

Decentralized Online Convex Optimization With Event-Triggered Communications

Decentralized multi-agent optimization usually relies on information exchange between neighboring agents, which can incur unaffordable communication overhead in practice. To reduce the communication cost, we apply event-triggering technique to the decentralized multi-agent online convex optimization problem, where each agent is associated with a time-varying local loss function and the goal is to minimize the accumulated total loss (the sum of all local loss functions) by choosing appropriate actions sequentially. We first develop an event-triggered decentralized online subgradient descent algorithm for the full information case, where the local loss function is fully revealed to each agent at each time. We establish an upper bound for the regret of each agent in terms of the event-triggering thresholds. It is shown that the regret is sublinear provided that the event-triggering thresholds converge to zero as time goes to infinity. The algorithm and analysis are further extended to the scenario of bandit feedback, where only the values of the local loss function at two random points close to the current action are disclosed to each agent. We show that the two-point bandit feedback does not degrade the performance of the proposed algorithm in order sense and a regret bound similar to the full information case can be established. Finally, numerical results on the problem of decentralized online least squares are presented to validate the proposed algorithms.

Push-Sum Distributed Online Optimization With Bandit Feedback.

In this article, we concentrate on distributed online convex optimization problems over multiagent systems, where the communication between nodes is represented by a class of directed graphs that are time varying and uniformly strongly connected. This problem is in bandit feedback, in the sense that at each time only the cost function value at the committed point is revealed to each node. Then, nodes update their decisions by exchanging information with their neighbors only. To deal with Lipschitz continuous and strongly convex cost functions, a distributed online convex optimization algorithm that achieves sublinear individual regret for every node is developed. The algorithm is built on the algorithm called the push-sum scheme that releases the request of doubly stochastic weight matrices, and the one-point gradient estimator that requires the function value at only one point at every iteration, instead of the gradient information of loss function. The expected regret of our proposed algorithm scales as O (T2/3 ln2/3(T)) , and T is the number of iterations. To validate the performance of the algorithm developed in this article, we give a simulation of a common numerical example.

Efficient Projection-Free Online Methods with Stochastic Recursive Gradient

This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems. Existing projection-free methods either achieve suboptimal regret bounds or have high per-round computational costs. To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are proposed for solving stochastic and adversarial OCO problems, respectively. By employing a recursive gradient estimator, our methods achieve optimal regret bounds (up to a logarithmic factor) while possessing low per-round computational costs. Experimental results demonstrate the efficiency of the proposed methods compared to state-of-the-arts.

Distributed Online Optimization With Long-Term Constraints

In this article, we consider distributed online convex optimization problems, where the distributed system consists of various computing units connected through a time-varying communication graph. In each time step, each computing unit selects a constrained vector, experiences a loss equal to an arbitrary convex function evaluated at this vector, and may communicate to its neighbors in the graph. The objective is to minimize the system-wide loss accumulated over time. We propose a decentralized algorithm with regret and cumulative constraint violation in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{\max \lbrace c,1-c\rbrace })$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{1-c/2})$</tex-math></inline-formula> , respectively, for any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c\in (0,1)$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the time horizon. When the loss functions are strongly convex, we establish improved regret and constraint violation upper bounds in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(\log (T))$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(\sqrt{T\log (T)})$</tex-math></inline-formula> . These regret scalings match those obtained by state-of-the-art algorithms and fundamental limits in the corresponding centralized online optimization problem (for both convex and strongly convex loss functions). In the case of bandit feedback, the proposed algorithms achieve a regret and constraint violation in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{\max \lbrace c,1-c/3 \rbrace })$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{1-c/2})$</tex-math></inline-formula> for any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c\in (0,1)$</tex-math></inline-formula> . We numerically illustrate the performance of our algorithms for the particular case of distributed online regularized linear regression problems on synthetic and real data.