To Dev or to Doc?: Predicting College IT Students’ Prominent Functions in Software Teams Using LMS Activities and Academic Profiles

Abstract

A software team comprises software practitioners with diverse backgrounds and responsibilities, such as programmers, reviewers, testers, and documentation experts. Whether developing the architecture, implementing new features, creating test cases, or providing documentation for users and the development team, each of these jobs is essential to the accomplishment of software tasks. Current methods for determining a student’s software development skill include sending questionnaires and monitoring students while they work. Not only are these techniques restricted in coverage, but they also rely on intervention strategies, which may result in social desirability bias and student exhaustion. In this research, we offer a multivariate time-series classification strategy for automatically identifying students’ expertise in software development based on information passively accessible via LMS logs and course grades. Several machine learning and deep learning models, including XGBoost, Random Forest, SVM, Stochastic Gradient Descent, Multi-layer Perceptron, Gaussian Naive Baye, Complement Naive Bayes, Long Short-Term Memory (LSTM), and XceptionTime, are examined for their ability to model students’ LMS activities and academic performance at various degrees of granularity, namely semester and daily levels. A case study of 33 IT-majoring college students is utilized to validate the effectiveness of the proposed strategy. The experimental findings demonstrate that our best models yield F1 values of 79.52% and 75.68% for the developer and documenter identification tasks, utilizing Multilayer Perceptron with daily features and LSTM with semester features, respectively. We are the first to attempt to determine the roles of students in software development using passively accessible data. The findings not only shed light on the ability to create personalized education tailored to each student’s needs but also pave the way for numerous intelligent education technology applications that aim to automatically evaluate certain student characteristics in order to optimize student learning outcomes.