Collaborative Reinforcement Learning of Energy Contracts Negotiation Strategies

Abstract

This paper presents the application of collaborative reinforcement learning models to enable the distributed learning of energy contracts negotiation strategies. The learning model combines the learning process on the best negotiation strategies to apply against each opponent, in each context, from multiple learning sources. The diverse learning sources are the learning processes of several agents, which learn the same problem under different perspectives. By combining the different independent learning processes, it is possible to gather the diverse knowledge and reach a final decision on the most suitable negotiation strategy to be applied. The reinforcement learning process is based on the application of the Q-Learning algorithm; and the continuous combination of the different learning results applies and compares several collaborative learning algorithms, namely BEST-Q, Average (AVE)-Q; Particle Swarm Optimization (PSO)-Q, and Weighted Strategy Sharing (WSS)-Q. Results show that the collaborative learning process enables players’ to correctly identify the negotiation strategy to apply in each moment, context and against each opponent.