TEAMSCOPE: measuring software engineering processes with teamwork telemetry

Abstract

Project-based learning is an important teaching method in software engineering education. However, it is unclear how student projects can be evaluated objectively and systematically in classrooms. Measurements used in industry, such as quality of the codebase, are not the only expected outcomes in classrooms; informative assessments in project-based learning require more details about how students behave as individuals and as a team. In this paper, we establish the importance of measuring processes in project-based software engineering courses and present metrics mined from software development tools for monitoring and observing processes to facilitate teaching. A case study at a US university confirms that 1) teams with better conformance to software development processes achieve better outcomes, and 2) our approach can be used to design metrics that serve as early detectors of violations to software development processes. Our results suggest that instructors for software engineering courses can use our approach to design process metrics for systematic, targeted, and automatic evaluation of team projects. Furthermore, metrics designed using our approach can be used as building blocks for automated systems, and thus increase the scalability of project-based software engineering courses.