Issues in using students in empirical studies in software engineering education

Abstract

Several empirical studies have been carried out with college students as subjects in the last few years. Researchers often use these studies to pilot experiments before they are carried out in industrial environments. Reports on these studies usually focus on the results obtained and issues such as their external validity. However, the effects and value of empirical studies with students may go beyond the contribution to scientific literature. For instance, the pedagogical challenges and value of these studies is hardly ever stressed. We identify four primary actors that are involved in these empirical studies, i.e., researchers, students, instructors, and industry. We discuss the costs and benefits of empirical studies with students for these actors, which are different because of the actors' different goals, expectations, and constraints, which must be recognized to fully exploit empirical studies with students. We also provide some advice on how to carry out empirical studies with students based on our experiences.