Virtual Teaching Assistant for Grading Programming Assignments: Non-dichotomous Pattern based Program Output Matching and Partial Grading Approach

Abstract

This study proposes an automated programming assessment system with a virtual teaching assistant (VTA) grading mechanism to automatically assess and grade functional correctness of students' programming assignments by using a non-dichotomous pattern-based program output matching and partial grading approach. The VTA grading mechanism matches student program output with location-free or location-specific patterns to release the strict specification of program output and offer partial marks on programs with partial functional correctness to let students know they are on the right track. The VTA grading results were applied as immediate formative assessments to help students refine their programs. The results of an evaluation showed that the VTA grading had high accuracy and helped students refine their programs, particularly for complex programs. In addition, most students expressed a positive attitude toward the VTA grading mechanism.