Using software tools to automate the assessment of student programs

Abstract

Attempting to automate the process of assessing and grading student programs is not a frivolous exercise concerned merely with relieving tutors of this burden: the highly desirable goals of objectivity and accuracy of assessment form more serious considerations, while the advent of Computer Aided Instruction (CAI) systems for teaching introductory programming makes it imperative that such automation is attained. This paper examines the current status of approaches to automatic program assessment, and draws the lamentable conclusion that they are not yet sufficiently advanced to deal with the variety of program solutions that students tend to submit. It goes on to establish the pattern-matching and recognition capabilities that would be required of a system with the desired flexibility, and contends that the software technology for the provision of these facilities is already to hand in the form of compiler-writing tools. Several examples of typical student programming problems are considered, and the application of the Unix tools Lex and Yacc to the automatic assessment of these programs is described. Experience with this approach indicates that a comparatively small amount of effort is all that is necessary to generate automatic graders that are versatile and powerful enough to cope with a wide range of programming exercises.