Ginger2: an environment for computer-aided empirical software engineering

Abstract

Empirical software engineering can be viewed as a series of actions to obtain knowledge and a better understanding about some aspects of software development, given a set of problem statements in the form of issues, questions or hypotheses. Experience has made us aware of the criticality of integrating the various types of data that are collected and analyzed as well as the criticality of integrating the various types of activities that take place, such as experiment design and the experiment itself. This has led us to develop a Computer-Aided Empirical Software Engineering (CAESE) framework to support the empirical software engineering lifecycle. The paper first presents the CAESE framework that consists of three elements: (1) a process model for the lifecycle of empirical software engineering studies, including needs analysis, experiment design, actual experimentation, and analyzing and packaging results; (2) a model that helps empirical software engineers decide how to look at the world to be studied in a coherent manner; (3) an architecture, based on which CAESE environments can be built, consisting of tool sets for each phase of the process model, a process management mechanism, and the two types of integration mechanism that are vital for handling multiple types of data: data integration and control integration. Next, the paper describes the Ginger2 environment as an instantiation of our framework. It concludes with reports on case studies using Ginger2, which dealt with a variety of empirical data types including mouse and keystrokes, eye traces, 3D movement, skin resistance level, and videotaped data.