Test data generation for programming exercises with symbolic execution in Java PathFinder

Abstract

Automatic assessment of programming exercises is typically based on testing approach. Most automatic assessment frameworks execute tests and evaluate test results automatically, but the test data generation is not automated. No matter that such test data generation techniques and tools are available.We have researched how the Java PathFinder software model checker can be adopted to the specific needs of test data generation in automatic assessment. Practical problems considered are: How to derive test data directly from students' programs (i.e. without annotation) and how to visualize and how to abstract test data automatically for students? Interesting outcomes of our research are that with minor refinements generalized symbolic execution with lazy initialization (a test data generation algorithm implemented in PathFinder) can be used to construct test data directly from students' programs without annotation, and that intermediate results of the same algorithm can be used to provide novel visualizations of the test data.