Test infrastructure and environment

Abstract

This issue contains two papers. These papers focus on test infrastructure and testing environment, respectively. Falling into the category of Software Note, the first paper, “JUGE: An infrastructure for benchmarking Java unit test generators” by Xavier Devroey, Alessio Gambi, Juan Pablo Galeotti, René Just, Fitsum Kifetew, Annibale Panichella and Sebastiano Panichella, presents JUnit Generation Benchmarking Infrastructure (JUGE), which supports test generators (such as search-based, random-based and symbolic execution) by automating the production of unit tests for validation, regression testing, fault localization and so on. JUGE aims to reduce the overall benchmarking effort, ease the comparison of multiple generators and enhance the knowledge transfer between academia and industry. JUGE was used and evolved during multiple editions of a unit testing tool competition since 2013. The authors discuss the expected impact of JUGE in improving the knowledge transfer on tools and approaches for test generation between academia and industry (recommended by Marc Roper). Falling into the typical category of Research Article, the second paper, “Test environments for large-scale software systems—An industrial study of intrinsic and extrinsic success factors” by Torvald Mårtensson, Göran Ancher and Daniel Ståhl, presents a study that investigates the characteristics of a test environment to support an organisation's testing objectives. The study includes a series of interviews with 30 individuals, a series of focus groups with 31 individuals and a cross-company workshop with 30 participants from five large-scale companies in different industry segments. The study findings reveal a list of success factors, including (1) characteristics and capabilities existing within a test environment (intrinsic success factors) and (2) properties not inherent to the test environment, but still vital for a successfully implemented test environment (extrinsic success factors). All the five companies included in the study well acknowledge the reported list of intrinsic and extrinsic success factors (recommended by Per Runeson).