Framework for Building Self-Adaptive Component Applications Based on Reinforcement Learning

Abstract

Component-based applications entail a composition of heterogeneous components often running in different contexts. The complexity and dynamic nature of their contexts result in an increasing maintenance efforts. Autonomic computing came to provide systems with an autonomic behavior based on predefined policies. However, in addition to being knowledge-intensive, the constructed policies may easily become obsolete due to context changes. Decision policies should be dynamically learned to self-adapt to context dynamics. However, currently built autonomic systems are tailored to specific management needs, neither reusable for other management concerns nor endowed with learning abilities. In this paper, we introduce a generic framework that facilitates building self-adaptive component-based applications. Unlike the existing initiatives, our framework provides means to transform an existing application by equipping it with a self-adaptive behavior to dynamically learn an optimal policy at runtime. To validate our approach, we have developed a realistic application and used the framework to render it self-adaptive. The experimental results have shown a negligible overhead and a dynamic adjustment of the transformed application to its changing context. They have also shown less frequent time spent in SLA (Service Level Agreement) violations during the learning phase and a better performing application after convergence.