An Externally Replicated Experiment for Evaluating the Learning Effectiveness of Using Simulations in Software Project Management Education

Abstract

The increasing demand for software project managers in industry requires strategies for the development of the management-related knowledge and skills of the current and future software workforce. Although several approaches help teach the required skills in a university setting, few empirical studies are currently available to characterize and compare their effects. This paper presents results of an externally replicated controlled experiment that evaluates the learning effectiveness of using a process simulation model for educating computer science students in software project management. While the experimental group applies a system dynamics (SD) simulation model, the control group uses the well-known COCOMO model as a predictive tool for project planning. The results of the empirical study indicate that students using the simulation model gain a better understanding about typical behavior patterns of software development projects. The combination of the results from the initial experiment and the replication corroborates this finding. Additional analysis shows that the observed effect can mainly be attributed to the use of the simulation model in combination with a web-based role-play scenario. This finding is strongly supported by information gathered from the debriefing questionnaires of subjects in the experimental group. They consistently rated the simulation-based role-play scenario as a very useful approach for learning about issues in software project management.