Multi-agent Resource Allocation Problem Research Articles

Network slicing for vehicular communications: a multi-agent deep reinforcement learning approach

This paper studies the multi-agent resource allocation problem in vehicular networks using non-orthogonal multiple access (NOMA) and network slicing. Vehicles want to broadcast multiple packets with heterogeneous quality-of-service (QoS) requirements, such as safety-related packets (e.g., accident reports) that require very low latency communication, while raw sensor data sharing (e.g., high-definition map sharing) requires high-speed communication. To ensure heterogeneous service requirements for different packets, we propose a network slicing architecture. We focus on a non-cellular network scenario where vehicles communicate by the broadcast approach via the direct device-to-device interface (i.e., sidelink communication). In such a vehicular network, resource allocation among vehicles is very difficult, mainly due to (i) the rapid variation of wireless channels among highly mobile vehicles and (ii) the lack of a central coordination point. Thus, the possibility of acquiring instantaneous channel state information to perform centralized resource allocation is precluded. The resource allocation problem considered is therefore very complex. It includes not only the usual spectrum and power allocation, but also coverage selection (which target vehicles to broadcast to) and packet selection (which network slice to use). This problem must be solved jointly since selected packets can be overlaid using NOMA and therefore spectrum and power must be carefully allocated for better vehicle coverage. To do so, we first provide a mathematical programming formulation and a thorough NP-hardness analysis of the problem. Then, we model it as a multi-agent Markov decision process. Finally, to solve it efficiently, we use a deep reinforcement learning (DRL) approach and specifically propose a deep Q learning (DQL) algorithm. The proposed DQL algorithm is practical because it can be implemented in an online and distributed manner. It is based on a cooperative learning strategy in which all agents perceive a common reward and thus learn cooperatively and distributively to improve the resource allocation solution through offline training. We show that our approach is robust and efficient when faced with different variations of the network parameters and compared to centralized benchmarks.

Heterogeneous multi-agent resource allocation through multi-bidding with applications to precision agriculture

Abstract In this paper we consider the problem of allocating multiple resources to a number of clients by a group of heterogeneous agents over time such that the clients can produce products while maximizing a profit function. We propose an approximate optimization framework in which every client provides multiple bids from which the agents choose such that an allocation is feasible and that the profit function is maximized over time. The proposed framework exploits decomposition techniques that can be used for large-scale multi-agent resource allocation problems in which the cost objective is additive, the dynamics of product generation is non-linear and the agents have different capabilities. Interestingly, the decomposition can be solved in a distributed fashion, enabling application to large-scale problems. We apply this decomposition to the management of resources and agents in precision agriculture as an inspirational and important application domain of the obtained results. We show that our framework can be used in order to schedule the time, location and quantity of resources that every agent must provide whilst optimizing the profit of the entire farm over the growing season.

Multiagent resource allocation with sharable items

We study a particular multiagent resource allocation problem with indivisible, but sharable resources. In our model, the utility of an agent for using a bundle of resources is the difference between the value the agent would assign to that bundle in isolation and a congestion cost that depends on the number of agents she has to share each of the resources in her bundle with. The valuation function determining the value and the delay function determining the congestion cost can be agent-dependent. When the agents that share a resource also share control over that resource, then the current users of a resource will require some compensation when a new agent wants to join them using the resource. For this scenario of shared control, we study the existence of distributed negotiation protocols that lead to a social optimum. Depending on constraints on the valuation functions (mainly modularity), on the delay functions (such as convexity), and on the structural complexity of the deals between agents, we prove either the existence of a sequences of deals leading to a social optimum or the convergence of all sequences of deals to such an optimum. We also analyse the length of the path leading to such optimal allocations. For scenarios where the agents using a resource do not have shared control over that resource (i.e., where any agent can use any resource she wants), we study the existence of pure Nash equilibria, i.e., allocations in which no single agent has an incentive to add or drop any of the resources she is currently holding. We provide results for modular valuation functions, we analyse the length of the paths leading to a pure Nash equilibrium, and we relate our findings to results from the literature on congestion games.

Characterizing Contract-Based Multiagent Resource Allocation in Networks

We consider a multiagent resource allocation problem where individual users intend to route traffic by requesting the help of entities across a network, and a cost is incurred at each network node that depends on the amount of traffic to be routed. We propose to study contract-based network resource allocation. In our model, users and nodes in the network make contracts before nodes route traffic for the users. The problem is an interesting self-interested negotiation problem because it requires the complete assembly of a set of distinct resources, and there are multiple combinations of distinct resources that could satisfy the goal of negotiation. First, we characterize the network allocation problem and show that finding optimal allocations is NP-complete and is inapproximable. We take both Nash equilibrium and pairwise Nash equilibrium as the solution concepts to characterize the equilibrium allocations. We find that, for any resource allocation game, Nash equilibrium and pairwise Nash equilibrium always exist. In addition, socially optimal allocations are always supported by Nash equilibrium and pairwise Nash equilibrium. We introduce best-response dynamics in which each agent takes a myopic best-response strategy and interacts with each other to dynamically form contracts. We analyze the convergence of the dynamics in some special cases. We also experimentally study the convergence rate of the dynamics and how efficient the evolved allocation is as compared with the optimal allocation in a variety of environments.

Multi-agent resource allocation: an incomplete information perspective

Explores a multi-agent resource allocation problem in an environment of incomplete information. The paper posits the existence of a central policy maker who integrates information acquisition with decision making, in order to optimally allocate resources among multiple agents. The study focuses on three salient aspects: first, the policy maker uses a nonparametric revealed preference approach for the elicitation of each agent's preference profiles. This allows the policy maker to avoid distorting the allocation decision and emerging with suboptimal allocations. Second, the process of information gathering and decision making is a challenging task in view of the extremely large numbers of possible profiles for each agent, even for problems of modest size. To find an optimal allocation, the policy maker can use schemes that utilize partial preference information to handle the complexity of the problem. Third, the incorporation of costs into the information process and the minimal assumptions made about the problem environment allow the policy maker to realistically analyze the resource allocation problem. >