A multi-agent reinforcement learning approach to dynamic service composition

Abstract

As a promising implementation of software systems, service composition has attracted significant attention and research, which constructs complex and value-added applications by composing existing single services to reduce the deployment time and cost. However, as the services on the Internet and the external environment are frequently changeable, these demand the service composition must be adaptive and dynamic to address these changes automatically. Therefore, this paper describes a multi-agent reinforcement learning model for the dynamic optimization of web service composition. In this model, agent can utilize reinforcement learning algorithms to interact with environment in real time to compute optimal composition strategy dynamically, and multi-agents mechanism can keep higher effectiveness in contrast to single-agent reinforcement learning. We propose a distributed Q-learning algorithm, which decompose the task into many sub-tasks and make every agent focus on own sub-task, to accelerate the convergence rate. In addition, we also introduce experience sharing strategy to improve the efficiency. As a result, these methods allow composite service to dynamically adjust itself to fit a varying environment, where the properties of the component services continue changing. Finally, a series of comparable experiments with traditional Q-learning algorithm demonstrate that our algorithms have certain validity, higher efficiency and obvious advantages.