Bilateral Multi-issue Negotiation Research Articles

Effective Acceptance Strategy Using Deep Reinforcement Learning in Bilateral Multi-issue Negotiation

Effective Acceptance Strategy Using Deep Reinforcement Learning in Bilateral Multi-issue Negotiation

Reward-Based Negotiating Agent Strategies

This study proposed a novel reward-based negotiating agent strategy using an issue-based represented deep policy network. We compared the negotiation strategies with reinforcement learning (RL) by the tournaments toward heuristics-based champion agents in multi-issue negotiation. A bilateral multi-issue negotiation in which the two agents exchange offers in turn was considered. Existing RL architectures for a negotiation strategy incorporate rich utility function that provides concrete information even though the rewards of RL are considered as generalized signals in practice. Additionally, in existing reinforcement learning architectures for negotiation strategies, both the issue-based representations of the negotiation problems and the policy network to improve the scalability of negotiation domains are yet to be considered. This study proposed a novel reward-based negotiation strategy through deep RL by considering an issue-based represented deep policy network for multi-issue negotiation. Comparative studies analyzed the significant properties of negotiation strategies with RL. The results revealed that the policy-based learning agents with issue-based representations achieved comparable or higher utility than the state-of-the-art baselines with RL and heuristics, especially in the large-sized domains. Additionally, negotiation strategies with RL based on the policy network can achieve agreements by effectively using each step.

VeNAS: Versatile Negotiating Agent Strategy via Deep Reinforcement Learning (Student Abstract)

Existing research in the field of automated negotiation considers a negotiation architecture in which some of the negotiation components are designed separately by reinforcement learning (RL), but comprehensive negotiation strategy design has not been achieved. In this study, we formulated an RL model based on a Markov decision process (MDP) for bilateral multi-issue negotiations. We propose a versatile negotiating agent that can effectively learn various negotiation strategies and domains through comprehensive strategies using deep RL. We show that the proposed method can achieve the same or better utility than existing negotiation agents.

Bilateral multi-issue negotiation of execution contexts by proactive document agents

A proactive document can react to its actual environment by autonomously selecting and performing actions integrated into its body and interact with its user. When migrating over a network of execution devices it may encounter diverse execution contexts, each one set up according to temporal characteristics of a receiving device and preferences of its owner. A concept to augment proactive documents with negotiation capability is proposed - to make them responsive to such dynamically changing contexts, and implemented in a system, where they can migrate as attachments to e-mail messages, owing to a dedicated e-mail client capable of handling them. Negotiation is based on a simple game-theoretic mechanism to minimise computation load on execution devices. Four negotiation algorithms are proposed and two of them evaluated in more detail in a series of experiments, when respectively, negotiating parties do not or do have knowledge on past encounters and negotiated contracts.

Bilateral multi-issue negotiation of execution contexts by proactive document agents

A proactive document can react to its actual environment by autonomously selecting and performing actions integrated into its body and interact with its user. When migrating over a network of execution devices it may encounter diverse execution contexts, each one set up according to temporal characteristics of a receiving device and preferences of its owner. A concept to augment proactive documents with negotiation capability is proposed – to make them responsive to such dynamically changing contexts, and implemented in a system, where they can migrate as attachments to e-mail messages, owing to a dedicated e-mail client capable of handling them. Negotiation is based on a simple game-theoretic mechanism to minimise computation load on execution devices. Four negotiation algorithms are proposed and two of them evaluated in more detail in a series of experiments, when respectively, negotiating parties do not or do have knowledge on past encounters and negotiated contracts.

Reinforcement learning‐based negotiation for spectrum micro‐trading framework

Spectrum trading has been permitted in most of the major wireless markets to facilitate better utilisation of spectrum. The authors have considered a spectrum trading framework in which a wireless service provider (WSP) leases its available channel(s) to another WSP for use at the designated base station (BS) of the latter for short duration. In their model, the agents of WSPs carry out the negotiation on the specifications of the channel usage such as transmission power, antenna height, spectrum band of the channel and price of the channel. In this work, they have modelled the negotiation as a multi-issue bilateral negotiation problem. Initially, they have solved the problem with the help of the Bayesian learning-based negotiation (BLBN) method. Furthermore, they have devised the novel reinforcement learning-based technique namely, reinforcement learning-based negotiation (RLBN) considering the adaptability of the BS to the new channel configuration. Surplus utility and convergence time of the negotiation process are considered as performance indices for the above techniques. The simulation results show that the RLBN outperforms BLBN and static negotiation technique as far as the objective of surplus utility is concerned.

An iterative and fair protocol for bilateral multi-issue negotiation with complex utility space

Throughout the study of bilateral negotiation, researchers have strived to reach an agreement in a simplified scenario: with definite utility functions (linear or polynomial) or over independent issues, which in realistic is not that simple. To remove this limitation, in this paper, we illustrate a complex nonlinear utility space: bilateral negotiations over many issues with interdependent valuations. An iterative and fair protocol is proposed to explore it, seeking for a ‘win-win’ and ‘envy-free’ contract to both agents. It works by introducing a utility secret transportation mechanism, combined with the heuristic algorithm NSGA-II, to find the Pareto efficient outcomes without the exposure of negotiators’ utility preferences. Experimental results analyze the benefit of employing Nash Product in the proposed protocol, and demonstrate its strengths in optimality rate and negotiating efficiency.

POPPONENT: Highly accurate, individually and socially efficient opponent preference model in bilateral multi issue negotiations

In automated bilateral multi issue negotiations, two intelligent automated agents negotiate on behalf of their owners regarding many issues in order to reach an agreement. Modeling the opponent can excessively boost the performance of the agents and increase the quality of the negotiation outcome. State of the art models accomplish this by considering some assumptions about the opponent which restricts the applicability of the models in real scenarios. In this study, a less restricted technique where perceptron units (POPPONENT) are applied in modeling the preferences of the opponent is proposed. This model adopts the Multi Bipartite version of the Standard Gradient Descent search algorithm (MBGD) to find the best hypothesis, which is the best preference profile. In order to evaluate the accuracy and performance of this proposed opponent model, it is compared with the state of the art models available in the Genius repository. This results in the devised setting which approves the higher accuracy of POPPONENT compared to the most accurate state of the art model. Evaluating the model in the real world negotiation scenarios in the Genius framework also confirms its high accuracy in relation to the state of the art models in estimating the utility of offers. The findings here indicate that this proposed model is individually and socially efficient. This proposed MBGD method could also be adopted in other practical areas of Artificial Intelligence.

A warranty based bilateral multi-issue negotiation approach

Purpose – The purpose of this paper is to propose a warranty-based bilateral automated multi-issue negotiation approach. Design/methodology/approach – A methodology for bilateral automated negotiation process is developed considering the targets such as warranty attractiveness, warranty cost, mean time between failures, spare parts cost to the end user over the useful life of the life. The negotiation methodology is explained using different cases of negotiation. The optimization for each negotiation step is carried out using genetic algorithm with elitism strategy. Findings – The result after optimization indicates that the desired target values are achieved and manufacturer obtained desired profit margin. Practical implications – Application of automated negotiation model is illustrated using a real life case of an automobile engine manufacturer. The proposed approach helps the manufacturer of any product to develop a methodology for carrying out the negotiation process. The approach also results into taking warranty-related decisions at the design stage. Originality/value – This paper contributes in proposing a generalized methodology for warranty-based negotiation in which the negotiation is carried out between the manufacturer and the customer.

Bayesian-based preference prediction in bilateral multi-issue negotiation between intelligent agents

Agent negotiation is a form of decision making where two or more agents jointly search for a mutually agreed solution to a certain problem. In multi-issue negotiation, with information available about the agents’ preferences, a negotiation may result in a mutually beneficial agreement. In a competitive negotiation environment, however, self-interested agents may not be willing to reveal their preferences, and this can increase the difficulty of negotiating a mutually beneficial agreement. In order to solve this problem, this paper proposes a Bayesian-based approach which can help an agent to predict its opponent’s preference in bilateral multi-issue negotiation. The proposed approach employs Bayesian theory to analyse the opponent’s historical offers and to approximately predict the opponent’s preference over negotiation issues. A counter-offer proposition algorithm is also integrated into the prediction approach to help agents to propose mutually beneficial offers based on the prediction results. Experimental results indicate good performance of the proposed approach in terms of utility gain and negotiation efficiency.

An approach to complex agent-based negotiations via effectively modeling unknown opponents

Negotiation among computational autonomous agents has gained rapidly growing interest in previous years, mainly due to its broad application potential in many areas such as e-commerce and e-business. This work deals with automated bilateral multi-issue negotiation in complex environments. Although tremendous progress has been made, available algorithms and techniques typically are limited in their applicability for more complex situations, in that most of them are based on simplifying assumptions about the negotiation complexity such as simple or partially known opponent behaviors and availability of negotiation history. We propose a negotiation approach called OMAC★ that aims at tackling these problems. OMAC★ enables an agent to efficiently model opponents in real-time through discrete wavelet transformation and non-linear regression with Gaussian processes. Based on the approximated model the decision-making component of OMAC★ adaptively adjusts its utility expectations and negotiation moves. Extensive experimental results are provided that demonstrate the negotiation qualities of OMAC★, both from the standard mean-score performance perspective and the perspective of empirical game theory. The results show that OMAC★ outperforms the top agents from the 2012, 2011 and 2010 International Automated Negotiating Agents Competition (ANAC) in a broad range of negotiation scenarios.

Tri-Training based Bilateral Multi-Issue Negotiation Framework

Negotiation has been extensively discussed in electronic commerce for decades. Recent growing interest in importing machine learning algorithm in electronic commerce has given increased importance to automated negotiation. A Tri-Training based algorithm was proposed to learn opponent's negotiation preference. The process of negotiation was viewed as a proposal's sequence which can be mapped into bidding trajectory feature space to form sample set. Due to fierce competition, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, Tri-Training, as a semi- supervised method, was imported into negotiation framework to increase the number of samples and improve perdition accuracy of opponent's negotiation preference learning. Based on negotiation preference of both side, an optimization algorithm is conducted to compute win-win counter proposal. The experimental results show that the proposed method can decrease the number of negotiation steps and increase the overall utility of negotiation.

Evaluating practical negotiating agents: Results and analysis of the 2011 international competition

This paper presents an in-depth analysis and the key insights gained from the Second International Automated Negotiating Agents Competition (ANAC 2011). ANAC is an international competition that challenges researchers to develop successful automated negotiation agents for scenarios where there is no information about the strategies and preferences of the opponents. The key objectives of this competition are to advance the state-of-the-art in the area of practical bilateral multi-issue negotiations, and to encourage the design of agents that are able to operate effectively across a variety of scenarios. Eighteen teams from seven different institutes competed. This paper describes these agents, the setup of the tournament, including the negotiation scenarios used, and the results of both the qualifying and final rounds of the tournament. We then go on to analyse the different strategies and techniques employed by the participants using two methods: (i) we classify the agents with respect to their concession behaviour against a set of standard benchmark strategies and (ii) we employ empirical game theory (EGT) to investigate the robustness of the strategies. Our analysis of the competition results allows us to highlight several interesting insights for the broader automated negotiation community. In particular, we show that the most adaptive negotiation strategies, while robust across different opponents, are not necessarily the ones that win the competition. Furthermore, our EGT analysis highlights the importance of considering metrics, in addition to utility maximisation (such as the size of the basin of attraction), in determining what makes a successful and robust negotiation agent for practical settings.

A Novel E-Commerce Negotiation Optimization Model Based on Improved Genetic Algorithm

Electronic commerce has rapidly become a major player in the business market .This paper proposes a new electronic commerce negotiation optimization model based on improved genetic algorithm which depends on not only price, but also other factors of commodity. The proposed model illustrates the relationship between the business components required to support the e-commerce processes with the value creation factor and the controlling complexity. The experiment results show that the proposed algorithm can gain the optimal negotiation result more efficiently than other three kinds of negotiation algorithms in competitive bilateral multi-issue negotiation.

Comparison of decision rules to mediate negotiations with incomplete information

Decisions in society resulting from negotiation among two parties are often inefficient (in terms of Pareto efficiency) as the parties fail to identify an agreement that would make both better off. A third party mediating the negotiation can increase the chances of reaching a Pareto efficient alternative. However, in most practical situations the mediator does not have a precise model of the parties’ preferences, and therefore cannot identify which alternatives are efficient. This paper assesses the performance of decision rules that a mediator can apply in multi-issue bilateral negotiation analysis given ordinal information about the importance of the issues and the value of the potential alternatives in each issue. We assume the preferences of the parties can be modelled by an additive multiattribute value function, but without assuming the precise parameters of this model are known. We study three mediation criteria: maximising the sum of the values, maximising the product of the excesses, or maximising the minimal proportion of potential. Monte-Carlo simulation is used to assess how good the alternative chosen by each decision rule is, and to provide guidelines about the use of these rules in a context of selecting a subset of promising alternatives.

Quickly locating efficient, equitable deals in automated negotiations under two-sided information uncertainty

This paper develops an automated negotiation procedure inclusive of mechanism design and agent design for bilateral multi-issue negotiations under two-sided information uncertainty. The proposed negotiation mechanism comprises a protocol called MUP (Monotonic Utility-granting Protocol) and a matching strategy called WYDIWYG (What You Display Influences What You Get). The proposed preference elicitation procedure makes the agents faithful surrogates of the user they represent while the proposed Frontier Tracking Proposal Construction Algorithm (FTPCA) makes them learn the opponent's flexibility in negotiation and respond appropriately. The mechanism design and the agent design together help in locating efficient and equitable deals quickly. The efficiency, stability, simplicity, distribution symmetry and incentive compatibility of the proposed procedure are demonstrated through negotiation simulation experiments.

AutoMed: an automated mediator for multi-issue bilateral negotiations

In this paper, we present AutoMed, an automated mediator for multi-issue bilateral negotiation under time constraints. AutoMed elicits the negotiators preferences and analyzes them. It monitors the negotiations and proposes possible solutions for resolving the conflict. We conducted experiments in a simulated environment. The results show that negotiations mediated by AutoMed are concluded significantly faster than non-mediated ones and without any of the negotiators opting out. Furthermore, the subjects in the mediated negotiations are more satisfied with the resolutions than the subjects in the non-mediated negotiations.

Improving trade-offs in automated bilateral negotiations for expressive and inexpressive scenarios

In multi-issue bilateral negotiations, it is possible to make issue trade-offs to allow agents to search for win-win solutions. Different techniques may be used to perform these trade-offs. In particular, similarity criteria have been successfully used in this context. However, one drawback of similarity-based trade-offs is that this approach behaves differently depending on the knowledge each agent has about its counterpart, and depending on the order in which the different issues are considered. To address this problem, in this paper we propose three new mechanisms to improve the search for joint gains. Two of them are applicable in expressive scenarios, where agents may be willing to share preference information, while the third one is intended for inexpressive scenarios. The experimental evaluation shows how our proposals improve the efficiency and optimality of the negotiation process over previous approaches.

A Bilateral Multi-Issue Negotiation Based on Hybrid Genetic Algorithm

A Bilateral Multi-Issue Negotiation Based on Hybrid Genetic Algorithm

LEARNING NEGOTIATION POLICIES USING ENSEMBLE-BASED DRIFT DETECTION TECHNIQUES

In this work we compare drift detection techniques and we show how they can improve the performance of trade agents in multi-issue bilateral dynamic negotiations. In a dynamic negotiation the utility values and functions of trade agents can change on the fly. Intelligent trade agents must identify and take such drift in the competitors into account changing also the offer policies to improve the global utility throughout the negotiation. However, traditional learning mechanisms disregard possible changes in a competitor's offer/counter-offer policy. In that case, the agent performance may decrease drastically. In our approach, a trade agent has a staff of weighted learner agents used to predict interesting offers. The staff uses the Dynamic Weighted Majority (DWM) algorithm to adapt itself creating, deleting and adapting staff members. The results obtained with the IB3 (Instance-based) learners and IB3-DWM learners show that ensemble methods like DWM are suitable for correctly identifying changes in agent negotiations.