Toward objective and quantitative assessment and prediction of teamwork effectiveness in software engineering courses

Abstract

Forward Today’s software engineering projects require teamwork which students practice in upper division software engineering courses. However, do they really ‘learn’ teamwork practices? This month’s column addresses this question. While reading this article please think about how the concepts presented might be effectively applied in a corporate setting. Mark & Peter Introduction One of the critical challenges in effective software engineering (SE) education is the lack of objective assessment methods of how well student teams learn the critically needed teamwork practices, defined as the ability: (i) to learn and effectively apply SE processes in a teamwork setting, and (ii) to work as a team to develop satisfactory software (SW) products. In addition, there are no effective methods for predicting learning effectiveness in order to enable early intervention in the classroom. This is further complicated with the emergence of global SW teams. Current approaches to assess achievement of SE teamwork skills rely on qualitative and subjective data taken as surveys at the end of the class with only rudimentary data analysis. In this article we present initial progress in our research to address the assessment and prediction of student learning of teamwork effectiveness in SE education. Our novel approach is based on: a) extracting only objective and quantitative student team activity data during their team class project; b) pairing this data with related independent observations and grading of student team effectiveness in SE process and SE product components in order to create “training database”; and c) applying a machine learning (ML) approach, namely random forest classification (RF), to the above training database in order to create ML models, ranked factors and rules that can both explain and assess, as well as provide prediction of the student teamwork effectiveness. Student team activity data are collected from ongoing and synchronously offered SE classes at San Francisco State University (SFSU), Florida Atlantic University (FAU) and Fulda University, Germany (Fulda), for approximately 80 students each year, working in about 15 teams, where student teams are both local and global (with students from multiple schools). In this article we summarize our approach and present preliminary data analysis results which served to test the concept, data gathering and ML tools we intend to use. We believe that success in this project will transform teaching (e.g. assessment) of critically important SE teamwork and will be of benefit to managing SE projects in industry.