Pricing Cloud Resource Based on Reinforcement Learning in the Competing Environment

Abstract

Multiple cloud providers compete against each other in order to attract cloud users and make profits in the cloud market. In doing so, each provider needs to charge fees to users in a proper way. In this paper, we will analyze how a cloud provider sets price effectively when competing against other cloud providers. Specifically, we model this problem as a Markov game, and then use minimax-Q and Q learning algorithms to design the pricing policies respectively. Based on this, we run extensive experiments to analyze the effectiveness of minimax-Q and Q learning based pricing policies. We find that although minimax-Q is more suitable in analyzing the competing game with multiple self-interested cloud providers, Q learning based pricing policy performs better in terms of making profits. We also find that minimax-Q learning based pricing policy performs better in terms of keeping cloud users. Our experimental results can provide useful insights on designing practical pricing policies in different situations.