Self-interested Agents Research Articles

Space Debris Removal: Learning to Cooperate and the Price of Anarchy.

In this paper we study space debris removal from a game-theoretic perspective. In particular we focus on the question whether and how self-interested agents can cooperate in this dilemma, which resembles a tragedy of the commons scenario. We compare centralised and decentralised solutions and the corresponding price of anarchy, which measures the extent to which competition approximates cooperation. In addition we investigate whether agents can learn optimal strategies by reinforcement learning. To this end, we improve on an existing high fidelity orbital simulator, and use this simulator to obtain a computationally efficient surrogate model that can be used for our subsequent game-theoretic analysis. We study both single- and multi-agent approaches using stochastic (Markov) games and reinforcement learning. The main finding is that the cost of a decentralised, competitive solution can be significant, which should be taken into consideration when forming debris removal strategies.

Utility Distribution Strategy of the Task Agents in Coalition Skill Games

This paper focuses on the rational distribution of task utilities in coalition skill games, which is a restricted form of coalition game, where each service agent has a set of skills and each task agent needs a set of skills in order to be completed. These two types of agents are assumed to be self-interested. Given the task selection strategy of service agents, the utility distribution strategies of task agents play an important role in improving their individual revenues and system total revenue. The problem that needs to be resolved is how to design the task selection strategies of the service agents and the utility distribution strategies of the task agents to make the self-interested decisions improve the system whole performance. However, to the best of our knowledge, this problem has been the topic of very few studies and has not been properly addressed. To address this problem, a task allocation algorithm for self-interested agents in a coalition skill game is proposed, it distributes the utilities of tasks to the needed skills according to the powers of the service agents that possess the corresponding skills. The final simulation results verify the effectiveness of the algorithm.

Investigating the characteristics of one-sided matching mechanisms under various preferences and risk attitudes

One-sided matching mechanisms are fundamental for assigning a set of indivisible objects to a set of self-interested agents when monetary transfers are not allowed. Two widely-studied randomized mechanisms in multiagent settings are the Random Serial Dictatorship (RSD) and the Probabilistic Serial Rule (PS). Both mechanisms require only that agents specify ordinal preferences and have a number of desirable economic and computational properties. However, the induced outcomes of the mechanisms are often incomparable and thus there are challenges when it comes to deciding which mechanism to adopt in practice. In this paper, we first consider the space of general ordinal preferences and provide empirical results on the (in)comparability of RSD and PS. We analyze their respective economic properties under general and lexicographic preferences. We then instantiate utility functions with the goal of gaining insights on the manipulability, efficiency, and envyfreeness of the mechanisms under different risk-attitude models. Our results hold under various preference distribution models, which further confirm the broad use of RSD in most practical applications.

HogRider: Champion Agent of Microsoft Malmo Collaborative AI Challenge

It has been an open challenge for self-interested agents to make optimal sequential decisions in complex multiagent systems, where agents might achieve higher utility via collaboration. The Microsoft Malmo Collaborative AI Challenge (MCAC), which is designed to encourage research relating to various problems in Collaborative AI, takes the form of a Minecraft mini-game where players might work together to catch a pig or deviate from cooperation, for pursuing high scores to win the challenge. Various characteristics, such as complex interactions among agents, uncertainties, sequential decision making and limited learning trials all make it extremely challenging to find effective strategies. We present HogRider---the champion agent of MCAC in 2017 out of 81 teams from 26 countries. One key innovation of HogRider is a generalized agent type hypothesis framework to identify the behavior model of the other agents, which is demonstrated to be robust to observation uncertainty. On top of that, a second key innovation is a novel Q-learning approach to learn effective policies against each type of the collaborating agents. Various ideas are proposed to adapt traditional Q-learning to handle complexities in the challenge, including state-action abstraction to reduce problem scale, a warm start approach using human reasoning for addressing limited learning trials, and an active greedy strategy to balance exploitation-exploration. Challenge results show that HogRider outperforms all the other teams by a significant edge, in terms of both optimality and stability.

Cooperative Games With Bounded Dependency Degree

Cooperative games provide a framework to study cooperation among self-interested agents. They offer a number of solution concepts describing how the outcome of the cooperation should be shared among the players. Unfortunately, computational problems associated with many of these solution concepts tend to be intractable---NP-hard or worse. In this paper, we incorporate complexity measures recently proposed by Feige and Izsak (2013), called dependency degree and supermodular degree, into the complexity analysis of coopera- tive games. We show that many computational problems for cooperative games become tractable for games whose dependency degree or supermodular degree are bounded. In particular, we prove that simple games admit efficient algorithms for various solution concepts when the supermodular degree is small; further, we show that computing the Shapley value is always in FPT with respect to the dependency degree. Finally, we observe that, while determining the dependency among players is computationally hard, there are efficient algorithms for special classes of games.

Political Interconnection in the Operation of Digital Terrestrial Free-to- Air Television Broadcasting

Agent(s) role in the implementation of policies is frequently considered as the determining factor for the success of policy implementation. This is reflected quite clearly in the “principle- agent” theory that describes how self-interested agent influences the implementation process. However, is self-interested agent still relevant in explaining Indonesia’s broadcasting policy implementation? What if policy implementation involved many actors with their respective personal interests? How will agents position themselves amidst numerous personal interest- bearing actors? By using the political economy approach, this research aims to reveal the role of agents in the constellation of actors’ relation to Indonesia’s broadcasting policy implementation. The operation of digital terrestrial free-to-air television broadcasting case is used to provide a reflection of agents’ position and political behavior in responding to the interest among actors. This research was conducted using the qualitative approach by implementing the data collection technique through in-depth interviews and document analysis. The research result shows that broadcasting policy implementation is not merely influenced by a self-interested agent but is also influenced by political interconnection and multiple-principles’ political-economic interest.

Swarm intelligence in humans: A perspective of emergent evolution

The origin of intelligence has fascinated scientists for a long time. Over the past 100 years, many scholars have observed the connection between entropy and intelligence. In the present study, we investigated a potential origin of the swarm intelligence in humans. The present study shows that a competitive economy consisting of a large number of self-interested agents can be mapped to a Boltzmann-like system, where entropy and energy play roles of swarm intelligence and income, respectively. However, different from the physical entropy in the Boltzmann system, the entropy (or swarm intelligence) in the economic system is a self-referential variable, which may be a key characteristic for distinguishing between biological and physical systems. Furthermore, we employ the household income data from 66 countries and Hong Kong SAR to test the validity of the Boltzmann-like distribution. Remarkably, the empirical data are perfectly consistent with the theoretical results. This finding implies that the competitive behaviors among a colony of self-interested agents will spontaneously prompt the colony to evolve to a state of higher technological level, although each agent has no willingness to evolve.

Inducing Exploration in Service Platforms

Crowd-sourced content in the form of online product reviews or recommendations is an integral feature of most Internet-based service platforms and marketplaces, including Yelp, TripAdvisor, Netflix, and Amazon. Customers may find such information useful when deciding between potential alternatives; at the same time, the process of generating such content is mainly driven by the customers' decisions themselves. In other words, the service platform or marketplace the set of available options through its customers' decisions, while they exploit the information they obtain from the platform about past experiences to determine whether and what to purchase. Unlike the extensive work on the trade-off between exploration and exploitation in the context of multi-armed bandits, the canonical framework we discuss in this chapter involves a principal that explores a set of options through the actions of self-interested agents. In this framework, the incentives of the principal and the agents towards exploration are misaligned, but the former can potentially incentivize the actions of the latter by appropriately designing a payment scheme or an information provision policy.

Swarm Intelligence in Humans: A Perspective of Emergent Evolution

The origin of intelligence has fascinated scientists for a long time. Over the past 100 years, many scholars started to observe the connection between entropy and intelligence. Recently, as an important advance, the physical relationship between the individual intelligence (cognition) and the entropy maximization has been established. Here we make an effort to investigate a possible origin of the swarm intelligence in humans. Our study shows that a competitive economy consisting of a large number of self-interested agents can be mapped to a many-particle Boltzmann system, where entropy and energy will play the roles of swarm intelligence and income, respectively. Furthermore, we employ the household income data from 60 countries to test the validity of Boltzmann distribution and chemical potential. Remarkably, the empirical data perfectly agrees with the theoretical results. Our finding implies that the competitive behaviors among a colony of self-interested agents will spontaneously prompt the colony to evolve to a state of higher technology progress, though each agent has no willingness to do this.

Mathematical Modeling and Experimental Evaluation of Optimal Agendas and Procedures for N-Issue Negotiation

This paper addresses the multi-issue negotiation between two self-interested agents in context of online auction, an auction where items are sold over internet. The outcome of such negotiations depends on the agenda and the negotiation procedure. In this paper we study the impact of varying agenda and procedure on the negotiation outcome in general and for an auction system in particular. This is done under incomplete information settings with the equilibrium strategies defined for issue-by-issue and package deal negotiation procedures. On the basis of these strategies, the optimal agenda-procedure combination for each agent is determined. An experimental evaluation of the same has also been done with multiple agents for an auction system, with satisfactory results.

Delegated Testing of Design Alternatives: The Role of Incentives and Testing Strategy

Testing a large variety of different product concepts is an integral part of nearly all new product development initiatives—especially in the concept selection phase, where firms seek to identify the most promising concept for further development. Test results are usually collected by agents, who must be incentivized not only to exert effort in testing the concepts but also to report their findings truthfully. We ask: How should a firm structure its concept testing processes when testing efforts must be delegated to self-interested agents? To answer this question, we devise a principal–(multi-)agent model that allows us to analyze different testing processes and to compare their relative benefits. We find that several factors determine how the firm should construct its testing processes: the efficiency of concept testing, the extent of heterogeneity in the quality of test outcomes, and the severity of information asymmetry between agents and the firm. Finally, we determine how the firm should adjust agents’ incentives to reflect its chosen testing process as well as which product concepts should (or should not) be tested.

A better-response strategy for self-interested planning agents

When self-interested agents plan individually, interactions that prevent them from executing their actions as planned may arise. In these coordination problems, game-theoretic planning can be used to enhance the agents’ strategic behavior considering the interactions as part of the agents’ utility. In this work, we define a general-sum game in which interactions such as conflicts and congestions are reflected in the agents’ utility. We propose a better-response planning strategy that guarantees convergence to an equilibrium joint plan by imposing a tax to agents involved in conflicts. We apply our approach to a real-world problem in which agents are Electric Autonomous Vehicles (EAVs). The EAVs intend to find a joint plan that ensures their individual goals are achievable in a transportation scenario where congestion and conflicting situations may arise. Although the task is computationally hard, as we theoretically prove, the experimental results show that our approach outperforms similar approaches in both performance and solution quality.

Cooperation in Social Dilemmas Through Position Uncertainty

We present a natural environment that sustains full cooperation in one-shot social dilemmas among a finite number of self-interested agents. Players sequentially decide whether to contribute to a public good. They do not know their position in the sequence, but observe the actions of some predecessors. Since agents realize that their own action may be observed, they have an incentive to contribute in order to induce potential successors to also do so. Full contribution can then emerge in equilibrium. The same environment leads to full cooperation in the prisoners’ dilemma.

Social Cooperation in Autonomous Agents to Avoid the Tragedy of the Commons

In this paper, we address the “Tragedy of the Commons” (TOC) problem for shared-resource systems by considering different types of behaviors of agents. On one extreme are self-interested agents while on the other one, agents are concerned about the welfare of the society. Algorithms to capture the different behaviors of the agents with and without interaction among the agents are proposed. An extensive experimental analysis for the different cases has been carried out as well as comparisons of our algorithms with an existing approach. Our study shows that if the agents are willing to sacrifice for some period of time, the sustainability of the society increases considerably.

Generic negotiation mechanisms with side payments – Design, analysis and application for decentralized resource-constrained multi-project scheduling problems

Negotiation mechanisms have been proven useful for decentralized resource-constrained multi-project scheduling with self-interested parties (agents) that work together in completing a project. The current paper is concerned with such kinds of multi-agent problems, where each involved party pursues the individual goal of maximizing the discounted cash flow that is connected to its own project activities. Since the parties assess schedules and proposed changes to solutions in terms of money, it is appropriate to utilize money transfers (side payments). We design and analyze different automated negotiation procedures with side payments for resolving conflicts of interest between two agents and experimentally compare results with negotiation mechanisms without the use of money transfers. In principle, we consider two generic negotiation approaches: firstly, an agent may offer tentative payments in order to incentivize the other agent to accept changes to the project schedule within iterative improvement-based negotiation processes; secondly, agents offer or claim money for selecting the final solution from an approximate set of Pareto-optimal solutions that have been generated by a preceding search process. Taking account of self-interested agents and private preference information we discuss suitable elements of the mechanism with regard to strategic agent behavior. We provide insights regarding the effectiveness of different behavioral choices and analyze reasonable strategies of the agents from a prescriptive perspective. Overall, the proposed solution mechanism improves existing results from the literature for the considered kind of problem. Since the approach is built on general concepts, it may be also useful for other applications.

Facility location with double-peaked preferences

We study the problem of locating a single facility on a real line based on the reports of self-interested agents, when agents have double-peaked preferences, with the peaks being on opposite sides of their locations. We observe that double-peaked preferences capture real-life scenarios and thus complement the well-studied notion of single-peaked preferences. As a motivating example, assume that the government plans to build a primary school along a street; an agent with single-peaked preferences would prefer having the school built exactly next to her house. However, while that would make it very easy for her children to go to school, it would also introduce several problems, such as noise or parking congestion in the morning. A 5-min walking distance would be sufficiently far for such problems to no longer be much of a factor and at the same time sufficiently close for the school to be easily accessible by the children on foot. There are two positions (symmetrically) in each direction and those would be the agent’s two peaks of her double-peaked preference. Motivated by natural scenarios like the one described above, we mainly focus on the case where peaks are equidistant from the agents’ locations and discuss how our results extend to more general settings. We show that most of the results for single-peaked preferences do not directly apply to this setting, which makes the problem more challenging. As our main contribution, we present a simple truthful-in-expectation mechanism that achieves an approximation ratio of 1+b/c for both the social and the maximum cost, where b is the distance of the agent from the peak and c is the minimum cost of an agent. For the latter case, we provide a 3 / 2 lower bound on the approximation ratio of any truthful-in-expectation mechanism. We also study deterministic mechanisms under some natural conditions, proving lower bounds and approximation guarantees. We prove that among a large class of reasonable strategyproof mechanisms, there is no deterministic mechanism that outperforms our truthful-in-expectation mechanism. In order to obtain this result, we first characterize mechanisms for two agents that satisfy two simple properties; we use the same characterization to prove that no mechanism in this class can be group-strategyproof.

Optimal delegation with self-interested agents and information acquisition

A team composed of a principal and an agent needs to choose a project to run, while they may have different preferences regarding the project. The agent has various types of bias in terms of project selection, and he can make an effort in acquiring the information regarding the promise of projects. The principal can either keep the decision-making authority of choosing which project to run, or delegate it to the agent. We find that the optimal effort level under the principal's authority is at its highest when the agent is the most biased, while under the agent's authority it is also at its highest when the agent has an intermediate bias. Therefore, the principal should keep the authority and communicate with the agent when the agent is relatively biased, and delegate the authority when the agent has an intermediate bias. However, from the team's point of view where both players’ payoffs are taken into account, the principal may delegate too much authority to a relatively unbiased agent, and too little authority to a relatively biased agent.

A Rawlsian algorithm for autonomous vehicles

Autonomous vehicles must be programmed with procedures for dealing with trolley-style dilemmas where actions result in harm to either pedestrians or passengers. This paper outlines a Rawlsian algorithm as an alternative to the Utilitarian solution. The algorithm will gather the vehicle's estimation of probability of survival for each person in each action, then calculate which action a self-interested person would agree to if he or she were in an original bargaining position of fairness. I will employ Rawls' assumption that the Maximin procedure is what self-interested agents would use from an original position, and then show how the Maximin procedure can be operationalized to produce unique outputs over probabilities of survival.

A distributed combinatorial optimisation heuristic for the scheduling of energy resources represented by self-interested agents

The aggregation of controllable distributed energy resources (DER) to virtual power plants (VPPs) forms a possible integration path for DER in future energy systems. The authors present a fully distributed scheduling heuristic for VPPs. The approach is realised by representing each participant of a VPP by a self-interested agent. Both the global, operator-driven scheduling objective of a VPP as well as arbitrary individual local objectives of the agents are integrated efficiently in a fully distributed coordination paradigm. Convergence and termination of the heuristic are proven in the presence of unreliable environments, e.g., with communication delays.

A distributed combinatorial optimisation heuristic for the scheduling of energy resources represented by self-interested agents

A distributed combinatorial optimisation heuristic for the scheduling of energy resources represented by self-interested agents