Using early assessment performance as early warning signs to identify at-risk students in programming courses

Abstract

This full paper presents results of a model developed using early assessment tasks as predictors to identify at-risk students. To date several studies have been conducted to identify and retain at-risk students in computer science courses. However, both researchers and teachers have long sought to understand early warning signs for identifying at-risk students. While coursework-based predictive models have been developed, they need further investigation, due to inconsistencies in a range of identified factors and techniques employed. This paper presents a classification tree analysis (manually created) and a Random forest classification-based predictive model that uses two variables to predict student performance in introductory programming. Visualisation of the decision tree results is employed as early warning signs for instructors to assist students who identified as at-risk. Data for the formative assessment tasks in the first two weeks of the semester was used for model development, validation and testing. The overall prediction accuracy of the model was 60%. The results of this study showed that it is possible to predict 77% of students that need support, as early as Week 3, based on student performance in continuous formative assessment tasks in a 12-week introductory programming course. Moreover, our classification tree analysis revealed that students who secured less than or equal to 25% in formative assessment tasks in the first two weeks are unlikely to attend or indeed fail the final exam. Additionally, the results provide useful insights for early interventions, to prevent attrition and failure and to increase student retention and student success.