Multi-agent Optimization Problem Research Articles

Distributed optimization under edge agreements: A continuous-time algorithm

Generalized from the concept of consensus, this paper considers a group of edge agreements, i.e. constraints defined for neighboring agents, in which each pair of neighboring agents is required to satisfy one edge agreement constraint. Edge agreements are defined locally to allow more flexibility than a global consensus. This work formulates a multi-agent optimization problem under edge agreements and proposes a continuous-time distributed algorithm to solve it. Both analytical proof and numerical examples are provided to validate the effectiveness of the proposed algorithm.

Stable Convergence of a Primal-Dual Method for Multi-agent Optimization Problems

Stable Convergence of a Primal-Dual Method for Multi-agent Optimization Problems

Augmented Lagrangian Tracking for distributed optimization with equality and inequality coupling constraints

In this paper we propose a novel Augmented Lagrangian Tracking distributed optimization algorithm for solving multi-agent optimization problems where each agent has its own decision variables, cost function and constraint set, and the goal is to minimize the sum of the agents’ cost functions subject to local constraints plus some additional coupling constraint involving the decision variables of all the agents. In contrast to alternative approaches available in the literature, the proposed algorithm jointly features a constant penalty parameter, the ability to cope with unbounded local constraint sets, and the ability to handle both affine equality and nonlinear inequality coupling constraints, while requiring convexity only. The effectiveness of the approach is shown first on an artificial example with complexity features that make other state-of-the-art algorithms not applicable and then on a realistic example involving the optimization of the charging schedule of a fleet of electric vehicles.

Event-Triggered Distributed Stochastic Mirror Descent for Convex Optimization.

This article is concerned with the distributed convex constrained optimization over a time-varying multiagent network in the non-Euclidean sense, where the bandwidth limitation of the network is considered. To save the network resources so as to reduce the communication costs, we apply an event-triggered strategy (ETS) in the information interaction of all the agents over the network. Then, an event-triggered distributed stochastic mirror descent (ET-DSMD) algorithm, which utilizes the Bregman divergence as the distance-measuring function, is presented to investigate the multiagent optimization problem subject to a convex constraint set. Moreover, we also analyze the convergence of the developed ET-DSMD algorithm. An upper bound for the convergence result of each agent is established, which is dependent on the trigger threshold. It shows that a sublinear upper bound can be guaranteed if the trigger threshold converges to zero as time goes to infinity. Finally, a distributed logistic regression example is provided to prove the feasibility of the developed ET-DSMD algorithm.

Computation Resource Optimization for Large-Scale Intelligent Urban Rail Transit: A Mean-Field Game Approach

Offloading tasks in smart devices (SDs) to an edge intelligence-empowered service centre (EISC) is a promising solution to support burgeoning intelligent applications in large-scale intelligent urban rail transits (URTs). However, dynamic computation resource allocation is still a crucial challenge, facing that various large-scale SDs share the EISC computation resources and the reality that the allocated computation resource for an SD is coupled with the offloading rate of all SDs. This paper proposes a joint dynamic offloading rate control and computation resource optimization method for large-scale intelligent URTs. Firstly, we model the large-scale multi-agent computation resource competition problem by a multi-player differential game (MPDG) and prove that the Nash equilibrium (NE) based optimal solution exists for each SDs. Then, we transform the MPDG model into a mean-field game (MFG). By introducing the mean-field into the game, we can solve the multi-agent optimization problem with a single-agent optimization method. We illustrate the rationality of the MFG model and propose an iterative solution method based on the finite difference method to derive the solution. Finally, we propose a z-transforming-based control method to dynamically reschedule computation resources among intelligent applications to achieve a satisfactory quality of service (QoS). Extensive simulation results show that our proposed scheme can significantly improve the performance of intelligent URTs.

Intermediate service facility planning in a stochastic and competitive market: Incorporating agent-infrastructure interactions over networks

This paper presents a network-based multi-agent optimization model for the strategic planning of service facilities in a stochastic and competitive market. We focus on the type of service facilities that are of intermediate nature, i.e., users may need to deviate from the shortest path to receive/provide services in between the users’ planned origins and destinations. This problem has many applications in emerging transportation mobility, including dynamic ride-sharing hub design and competitive facility location and allocation problems for alternative fuel vehicle refueling stations. The main contribution of this paper is establishing a new multi-agent optimization framework considering decentralized decision makings of facility investors and users over a transportation network and providing rigorous analyses of its mathematical properties, such as uniqueness and existence of system equilibrium. In addition, we develop an exact convex reformulation of the original multi-agent optimization problems to overcome computational challenges brought by non-convexity. Extensive analysis on case studies showed how the proposed model can capture the complex interaction between different stakeholders in an uncertain environment. Additionally, our model allowed quantifying the value of stochastic modeling and information availability by exploring stochastic metrics, including value of stochastic solution (VSS) and expected value of perfect information (EVPI), in a multi-agent framework.

Evaluating cooperative-competitive dynamics with deep Q-learning

We model cooperative-competitive social group dynamics with multi-agent environments, specialized in cases with a large number of agents from only a few distinct types. The multi-agent optimization problems are addressed in turn with multi-agent reinforcement learning algorithms to obtain flexible and robust solutions. We analyze the effectiveness of centralized and decentralized algorithms using three variants of deep Q-networks on these cooperative-competitive environments: first, we use the decentralized training independent learning with deep Q-networks, secondly the centralized monotonic value factorizations for deep learning, and lastly the multi-agent variational exploration. We test the algorithms in simulated predator–prey multi-agent environments in two distinct environments: the adversary pursuit and simple tag. The experiments highlight the performance of the different deep Q-learning methods, and we conclude that decentralized training of deep Q-networks accumulates higher episode rewards during training and evaluation in comparison with the selected centralized learning approaches.

Verification of coset weighted potential game and its application to optimisation of multi-agent systems

In this paper, we propose an algorithm to verify whether a finite game is a coset weighted potential game (WPG) without pre-knowledge on its coset weights. This algorithm can also provide a recursive method to calculate the unknown coset weights. Then we give the concept of near coset WPGs based on evolutionary equivalence between two finite games, and its algebraic verification is obtained. Finally, the application of near coset WPGs to game-theoretical optimisation of multi-agent systems is discussed, which can improve the applicability of potential games in multi-agent optimisation problems.

Zeroth-order feedback optimization for cooperative multi-agent systems

We study a class of cooperative multi-agent optimization problems, where each agent is associated with a local action vector and a local cost, and the goal is to cooperatively find the joint action profile that minimizes the average of the local costs. We consider the setting where gradient information is not readily available, and the agents only observe their local costs incurred by their actions as a feedback to determine their new actions. We propose a zeroth-order feedback optimization scheme and provide explicit complexity bounds for the constrained convex setting with noiseless and noisy local cost observations. We also discuss briefly on the impacts of knowledge of local function dependence between agents. The algorithm’s performance is justified by a numerical example of distributed routing control.

Privacy Preservation of Optimization Algorithm Over Unbalanced Directed Graph

This paper proposes an algorithm with the property of privacy preservation to solve the multi-agent optimization problem, where the communication topology is a fixed and strongly connected directed graph with a row stochastic weight matrix. Here, we consider the external eavesdropper attack model and take the gradient information of the objective function as the agent's privacy information. In this algorithm, each agent adds an additional state variable that interacts with it with a time-varying weight. And this new variable performs gradient iterative calculation. The original state variable interacts with the new variable and the original neighbors. It is proved that the algorithm converges to the optimal solution of the problem and at the same time achieves the purpose of privacy preservation. In addition, the algorithm neither requires additional hidden signals, nor does it increase a large amount of calculation. Finally, a simulation example is given to verify the performance of the algorithm.

Distributed Augmented Lagrangian Method for Link-Based Resource Sharing Problems of Multiagent Systems

A multi-agent optimization problem motivated by the management of energy systems is discussed. The associated cost function is separable and convex although not necessarily strongly convex and there exist edge-based coupling equality constraints. In this regard, we propose a distributed algorithm based on solving the dual of the augmented problem. Furthermore, we consider that the communication network might be time-varying and the algorithm might be carried out asynchronously. The time-varying nature and the asynchronicity are modeled as random processes. Then, we show the convergence and the convergence rate of the proposed algorithm under the aforementioned conditions.

Dynamic Coordination-Based Reinforcement Learning for Driving Policy

With the development of communication technology and artificial intelligence technology, intelligent vehicle has become a very important part of Internet of Things technology. At present, the single-vehicle intelligence is gradually improved, and more and more unmanned vehicles appear on the road. In the future, these individual intelligence applications need to be transformed into collective intelligence to give full play to the greater advantages of unmanned driving. For example, individual intelligence is self-interest. If there is no collective cooperation, it may affect the whole traffic flow for its own speed. Although the vehicle ad hoc network technology provides a guarantee for the communication between vehicles and makes cooperation between vehicles possible, there are still challenges in how to adapt to coordination learning. Coordination reinforcement learning is one of the most promising methods to solve the multiagent coordination optimization problems. However, existing coordinative learning approaches that usually rely on static topologies cannot be easily adopted to solve the vehicle coordination problems in the dynamic environment. We propose a dynamic coordination reinforcement learning to help vehicles make their driving decisions. First, we apply driving safety field theory to construct the dynamic coordination graph (DCG), representing the dynamic coordination behaviors among vehicles. Second, we design reinforcement learning techniques on our DCG model to implement the joint optimal action reasoning for the multivehicle system and eventually derive the optimal driving policy for each vehicle. Finally, compared with other multiagent learning methods, our method has a significant improvement in security and speed, which is about 1% higher than other multiagent learning methods, but its training speed is also significantly improved about 8%.

Distributed Randomized Gradient-Free Mirror Descent Algorithm for Constrained Optimization

This article is concerned with the multiagent optimization problem. A distributed randomized gradient-free mirror descent (DRGFMD) method is developed by introducing a randomized gradient-free oracle in the mirror descent scheme where the non-Euclidean Bregman divergence is used. The classical gradient descent method is generalized without using subgradient information of objective functions. The proposed algorithms are the first distributed non-Euclidean zeroth-order methods, which achieve an approximate <inline-formula><tex-math notation="LaTeX">$O(\frac{1}{\sqrt{T}})$</tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX">$T$</tex-math></inline-formula>-rate of convergence, recovering the best known optimal rate of distributed nonsmooth constrained convex optimization. Moreover, a decentralized reciprocal weighted averaging (RWA) approximating sequence is first investigated, the convergence for RWA sequence is shown to hold over time-varying graph. Rates of convergence are comprehensively explored for the algorithm with RWA (DRGFMD-RWA). The technique on constructing the decentralized RWA sequence provides new insight in searching for minimizers in distributed algorithms.

The Distributed Dual Ascent Algorithm is Robust to Asynchrony

The distributed dual ascent is an established algorithm to solve strongly convex multi-agent optimization problems with separable cost functions, in the presence of coupling constraints. In this letter, we study its asynchronous counterpart. Specifically, we assume that each agent only relies on the outdated information received from some neighbors. Differently from the existing randomized and dual block-coordinate schemes, we show convergence under heterogeneous delays, communication and update frequencies. Consequently, our asynchronous dual ascent algorithm can be implemented without requiring any coordination between the agents.

On the probabilistic feasibility of solutions in multi-agent optimization problems under uncertainty

We investigate the probabilistic feasibility of randomized solutions to two distinct classes of uncertain multi-agent optimization programs. We first assume that only the constraints of the program are affected by uncertainty, while the cost function is arbitrary. Leveraging recent developments on a posteriori analysis within the scenario approach, we provide probabilistic guarantees for all feasible solutions of the program under study. This is particularly useful in cases where the numerical implementation of the solution-seeking algorithm prevents the exact quantification of the optimal solution. Furthermore, this result provides guarantees for the entire solution set of optimization programs with uncertain convex constraints and (possibly) non-convex cost function. We then focus on optimization programs with deterministic constraints, where the cost function depends on uncertainty and admits an aggregate representation of the agents’ decisions. By exploiting the structure of the program under study and leveraging the so called support rank notion, we provide agent-independent robustness certificates for the optimal solution, i.e., the constructed bound on the probability of constraint violation does not depend on the number of agents, but only on the dimension of each agent’s decision space. This substantially reduces the amount of samples required to achieve a certain level of probabilistic robustness for a larger number of agents. All robustness certificates provided in this paper are distribution-free and can be used alongside any optimization algorithm. Our theoretical results are accompanied by a numerical case study of a charging control problem for a fleet of electric vehicles.

Distributed discrete-time optimization of heterogeneous multi-agent networks with unbounded position constraints and nonconvex velocity constraints

The distributed discrete-time heterogeneous multi-agent optimization problem with unbounded position constraints and nonconvex velocity constraints was considered in this paper, where the models of agents are described by first- or second- order difference equations, the position constraints are assumed to be unbounded and convex and the velocity constraints are assumed to be bounded, closed and nonconvex. An improved distributed algorithm is adopted, under which, all agents could collaboratively minimize the entire objective function and the positions and velocities could always stay at the corresponding constraint sets. Then the convergence analysis is completed by the coordination transformation and the properties of projection operator and constraint operator. Finally, we give examples to illustrate the correctness of results.

Decentralized multi-agent optimization based on a penalty method

We propose a decentralized penalty method for general convex constrained multi-agent optimization problems. Each auxiliary penalized problem is solved approximately with a special parallel descent splitting method. The method can be implemented in a computational network where each agent sends information only to the nearest neighbours. Convergence of the method is established under rather weak assumptions. We also describe a specialization of the proposed approach to the feasibility problem.

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framework that unifies many existing state-of-the-art algorithms. We establish linear convergence of the proposed method to the exact minimizer in the presence of the nonsmooth term. Moreover, for the more general class of problems with agent specific nonsmooth terms, we show that linear convergence cannot be achieved (in the worst case) for the class of algorithms that uses the gradients and the proximal mappings of the smooth and nonsmooth parts, respectively. We further provide a numerical counterexample that shows how some state-of-the-art algorithms fail to converge linearly for strongly convex objectives and different local non smooth terms.

A Decentralized Multi-objective Optimization Algorithm

During the past few decades, multi-agent optimization problems have drawn increased attention from the research community. When multiple objective functions are present among agents, many works optimize the sum of these objective functions. However, this formulation implies a decision regarding the relative importance of each objective: optimizing the sum is a special case of a multi-objective problem in which all objectives are prioritized equally. To enable more general prioritizations, we present a distributed optimization algorithm that explores Pareto optimal solutions for non-homogeneously weighted sums of objective functions. This exploration is performed through a new rule based on agents’ priorities that generates edge weights in agents’ communication graph. These weights determine how agents update their decision variables with information received from other agents in the network. Agents initially disagree on the priorities of objective functions, though they are driven to agree upon them as they optimize. As a result, agents still reach a common solution. The network-level weight matrix is (non-doubly) stochastic, contrasting with many works on the subject in which the network-level weight matrix is doubly-stochastic. New theoretical analyses are therefore developed to ensure convergence of the proposed algorithm. This paper provides a gradient-based optimization algorithm, proof of convergence to solutions, and convergence rates of the proposed algorithm. It is shown that agents’ initial priorities influence the convergence rate of the proposed algorithm and that these initial choices affect its long-run behavior. Numerical results performed with different numbers of agents illustrate the performance and effectiveness of the proposed algorithm.

Distributed optimization without boundedness of gradients for second-order multi-agent systems over unbalanced network

This paper is mainly devoted to the distributed second-order multi-agent optimization problem with unbalanced and directed networks, where the gradient of the private objective functions might be unbounded. To deal with this problem, a new distributed algorithm is proposed based on the local neighbor information and the private objective functions. By a coordination transformation, the closed-loop system is divided into two simple first-order subsystems. Under the assumption of the strong connectivity of networks, it is proved that all agents can collaboratively converge to some optimal solution of the team objective function. At last, we give two numerical examples to show the effectiveness of our proposed algorithm.