Self-Adaptation Through Reinforcement Learning Using a Feature Model

Abstract

Typically, self-adaptation is achieved by implementing the MAPE-K Control Loop. The researchers agree that multiple control loops should be assigned if the system is complex and large-scale. The hierarchical control has proven to be a good compromise to achieve SAS goals, as there is always some degree of decentralization but it also retains a degree of centralization. The decentralized entities must be coordinated to ensure the consistency of adaptation processes. The high cost of data transfer between coordinating entities may be an obstacle to achieving system scalability. To resolve this problem, coordination should only define between entities that require communication between them. However, most of the current SAS relies on static MAPE-K. In this article, authors present a new method that allows changing the structure and behavior of loops. Authors base on exploration strategies for online reinforcement learning, using the feature model to define the adaptation space.