Session details: Special Issue on Challenges in Software Performance

Abstract

Software systems have become increasingly complex. Such complexity makes it extremely challenging to deliver systems that meet their performance requirements. In recent years, there have been several high profile instances of projects that failed due to performance-related issues, e.g., the rollout of the healthcare.gov Web site. This motivates the need for new approaches for software developers to address performance problems. This special issue presents 6 papers that address this theme. A brief summary of these papers follows. Gesvindr and Buhnova study the challenges of building Platform as a Service (PaaS) cloud applications that meet their performance requirements. Using two case studies involving the Microsoft Azure cloud platform, the authors show that the use of a cloud platform in itself does not guarantee good performance and scalability. The authors then identify several best practices for ensuring the performance and scalability of PaaS applications. The work of Heinrich considers the problem of managing\ the performance of software systems composed of thirdparty cloud services. The lack of control over such thirdparty services can cause deviations from performance objectives that can be hard to foresee and manage. The author outlines a model-based framework called iObserve that can help a system adapt and evolve so as to address such problems. Li, O'Brien, and Kihl address the lack of standard methodologies for experimental performance evaluation of software systems. The paper outlines a method for defining domainspecific methodologies for performance evaluation in a way that is consistent across domains. The authors demonstrate the utility of their approach using a cloud services case study. The paper by Grottke et al. considers the problem of handling cyber attacks on networked computer systems andconnected industrial control systems supporting critical infrastructuresuch as energy, water, and gas. Using analytic models, the authors study the performance of various attack mitigation approaches. In particular, the model shows how parameters such as malware scanning rates, network topology, and types of infections impact the effective processing capacity of a system. Rehmann et al. describe the performance monitoring in SAP HANA's continuous integration process. They describe how performance regression tests have been integrated in the delivery process and what services are available to detect and report performance anomalies. Being an extended version of a previous workshop paper, this paper now also presents the reporting of performance test results and anomaly detection. Finally, Nambiar et al. examine issues and experience with software performance, and consider the shortcomings and potential benefits of modeling from the perspective of a potential industrial user. In particular, it considers why modeling is little used, and what conditions it would have to satisfy to fulfill its potential. We sincerely thank all authors for their work and contribution to this special issue. Additionally, we thank our reviewers Davide Arcelli, Danilo Ardagna, Alberto Avritzer, Steffen Becker, Ian Gorton, Andre van Hoorn, Raffaela Mirandola, Juan F. Perez, and Murray Woodside for their timely and constructive reviews as well as editor-in-chief Nidhi Hedge for her support.