Assessing the benefits of search-based approaches when designing self-adaptive systems: a controlled experiment

Sandro S Andrade,Raimundo J De A Macêdo

doi:10.1186/s40411-015-0016-z

Abstract

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive systems. In particular, designing modern self-adaptive systems requires intricate decision-making over a remarkably complex problem space and a vast array of solution mechanisms. Nowadays, a large number of approaches tackle the issue of endowing software systems with self-adaptive behavior from different perspectives and under diverse assumptions, making it harder for architects to make judicious decisions about design alternatives and quality attributes trade-offs. It has currently been claimed that search-based software design approaches may improve the quality of resulting artifacts and the productivity of design processes, as a consequence of promoting a more comprehensive and systematic representation of design knowledge and preventing design bias and false intuition. To the best of our knowledge, no empirical studies have been performed to provide sound evidence of such claim in the self-adaptive systems domain. This paper reports the results of a quasi-experiment performed with 24 students of a graduate program in Distributed and Ubiquitous Computing. The experiment evaluated the design of self-adaptive systems using a search-based approach proposed by us, in contrast to the use of a non-automated approach based on architectural styles catalogs. The goal was to investigate to which extent the adoption of search-based design approaches impacts on the effectiveness and complexity of resulting architectures. In addition, we also analyzed the approach’s potential for leveraging the acquisition of distilled design knowledge. Our findings show that search-based approaches can improve the effectiveness of resulting self-adaptive systems architectures and reduce their design complexity. We found no evidence regarding the approach’s potential for leveraging the acquisition of distilled design knowledge by novice software architects. This study contributes to reveal empirical evidence on the benefits of search-based approaches when designing self-adaptive systems architectures. The results presented herein increase our belief that the systematic representation of distilled design knowledge and the adoption of search-based design approaches indeed lead to improved architectures.

Highlights

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive systems
4 Methods The goal of the experiment we report was to analyze the design of SA systems, for the purpose of evaluating the search-based design approach we propose and a design process based on architecture styles catalogs, with respect to the effectiveness and complexity of resulting architectures, as well as the method’s potential for leveraging the acquisition of distilled design knowledge by novice SA systems architects, from the viewpoint of researchers, and in the context of graduate students endowing systems with self-adaptation capabilities
Such assumptions are: i) data is taken from an interval or ratio scale; ii) observations are independent; iii) measured values are normally distributed in the populations; and iv) population variances are equal between groups

Summary

Introduction

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive systems. Energy-aware mobile systems, self-tuning databases, and reconfigurable network services are some of the application domains in which self-adaptive mechanisms play a paramount role (Patikirikorala et al 2012). Such scenarios are usually characterized by incomplete knowledge about user requirements, workloads, and available resources. The MAPE-K approach (Kephart and Chess 2003) is a widely accepted reference architecture for adaptation loops It defines the basic components for the loop’s tasks of Monitoring, Analyzing, Planning, and Executing; performed with the support of a Knowledge Base

Methods

Results

Discussion

Conclusion