Abstract
Evaluating the performance of self-adaptive systems is challenging due to their interactions with often highly dynamic environments. In the specific case of self-healing systems, the performance evaluations of self-healing approaches and their parameter tuning rely on the considered characteristics of failure occurrences and the resulting interactions with the self-healing actions. In this paper, we first study the state-of-the-art for evaluating the performances of self-healing systems by means of a systematic literature review. We provide a classification of different input types for such systems and analyse the limitations of each input type. A main finding is that the employed inputs are often not sophisticated regarding the considered characteristics for failure occurrences. To further study the impact of the identified limitations, we present experiments demonstrating that wrong assumptions regarding the characteristics of the failure occurrences can result in large performance prediction errors, disadvantageous design-time decisions concerning the selection of alternative self-healing approaches, and disadvantageous deployment-time decisions concerning parameter tuning. Furthermore, the experiments indicate that employing multiple alternative input characteristics can help with reducing the risk of premature disadvantageous design-time decisions.
Highlights
A self-adaptive system (SAS) is capable of modifying itself at runtime in response to the changes of the environment and the system itself
This is inline with the related finding of [6], which suggests that simulation-based experiments are the most dominant approach employed for the evaluation of SAS in general
We demonstrate the impact of the identified limitation concerning the input for self-healing system (SHS) under evaluation through validating four hypotheses proposed in this paper
Summary
A self-adaptive system (SAS) is capable of modifying itself at runtime in response to the changes of the environment and the system itself. Violations of certain functional and nonfunctional goals trigger the self-adaptation [1]. Equipping the software system (adaptable software) with an external adaptation engine, such as a MAPE-K feedback loop, enables the realization of self-adaptive capabilities. The evaluation of self-adaptive systems is not trivial. On one hand, these systems often have a complex structure due to an additional control layer. The system is subject to changes in unforeseen ways as a result of the adaptation [2]. These systems are designed to be operated in highly dynamic environments, and require continuous monitoring of their behavior and execution environment [3]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
