Impacts of electronic process guides by types of user: An experimental study

Abstract

The design and utilization of Electronic Process Guides (EPGs) have been studied in Software Engineering (SwE) since the 1990s. However, the empirical findings from surveys, case studies, and experiments on the beneficial effects of their utilization are still lacking. Thus, we suggest that further research on the utilization of EPGs is required. In this study, we are interested in gaining insights on the effects of using EPGs on objective metrics (learning score, time effort) and subjective metrics (perceived usefulness, ease of use, and value), by comparing three EPG designs (a simple PDF-based EPG, a normal HTML-based EPG, and a sophisticated Java-based EPG) with different blocks of experimental subjects (practitioners, academicians, novices, and experts). To this end, we have conducted a controlled experiment with a sample of international participants in the domain of IT Service Management. We found that the utilization of EPGs improves the objective metrics while no improvements were perceived on the subjective ones, and that the sophisticated EPG design is more appropriate for the academic and expert types of users than for the practitioner and novice types. Thus, our main recommendation for the design and utilization of EPGs is to consider the type of end-user.