Synthesis of nested loop exercises for practice in introductory programming

Abstract

Novice programmers struggle to comprehend specific programming constructs such as arrays, recursion, and loops. One way to address this challenge is to provide practice problems for students in these topics that are considered difficult to comprehend — such as nested loops. It is proven that practice aids program comprehension, hence, since it is time consuming to manually create many practice problems; synthesising these problems is an Expert Artificial Intelligence Task that is worth investigating. In this paper we present the synthesis of nested loop exercises in Python for practice using a context-free grammar. We defined the grammar rules for modeling program templates. Algorithms were designed and implemented for the generation of structurally different nested loop exercises based on the defined production rules of the context-free grammar. Each program generated consists of two or more looping statements, with a nesting, and the context-free grammar rules also allow for the spawning of infinitely many sub-loops for every loop. We checked for the correctness of the synthesised nested loop programs with an experimental procedure of iteratively supplying these programs to the Python interpreter. The first 120,000 iterations returned no syntactic bugs, hence, showing that these programs do compile and have outputs. Furthermore, an evaluation of the programs was carried out in a survey with 210 participants (novice and expert programmers). The results of the evaluation show that over 82% of the participants agreed that the synthesised exercises were correctly generated, solvable and can be used as practice exercises for novice programmers in introductory programming modules. 120,000 iterations of synthesised loop programs can be found at: https://tinyurl.com/nestedloops2021.