A-Learn EvId: A Method for Identifying Evidence of Computer Programming Skills Through Automatic Source Code Assessment

Abstract

Contextualized in the teaching of computer programming in Computing courses, this research investigates aspects and strategies for automatic source code assessment. Continuous on-time assessment of source codes produced by students is a challenging task for teachers. The literature presents different methods for automatic assessment of source code, mostly focusing on technical aspects, such as functional correctness assessment and error detection. This paper presents the A-Learn EvId method, having as the main characteristic its focus on the assessment of high-level skills instead of technical aspects. Automatically assessing high-level skills gives insights into the thought process students used to elaborate their responses, contributing to quality and timely feedback generation. The method is characterized by three fundamental steps: (1) inserting students' source code as input data; (2) identifying evidence of skills through automatic strategies; and (3) representing identified skills through a student model. The following contributions are highlighted: updating the state of the art on the topic; a set of 37 skills identifiable through 9 automatic source code assessment strategies; construction of datasets totaling 8651 source codes.