Patterns of Code-to-Test Co-evolution for Automated Test Suite Maintenance

Abstract

Software systems are characterized by continual change which often occurs concurrently across various artifact types. While prior work has focused on the evolution of individual artifacts, this paper studies the patterns of co-evolution between source and test code. In this research, with a reference to the literature, as well as our manual analysis of several open-source software systems we first, patternize and document common patterns of co-evolution between source code and test suites. Leveraging the proposed patterns, we further infer the necessary remedies in the test suite in response to source code changes. Our approach enables to add missing test cases to the current version of a system (augmentation), but additionally allows to reuse and evolve the existing test suite for a modified version of the system (evolution). Furthermore, identifying patterns of concurrent evolution provides opportunities for a bi-directional change detection and remediation for both artifacts, source code and test cases, and additionally automates the process of maintaining code-to-test trace links. The evaluation of the patterns and remedies in five large open-source applications indicated the patterns contained up to 42% of the source code changes and the remediation recovered up to 100% of the impacted test cases in certain cases.