Design of an Online Programming Platform and a Study on Learners’ Testing Ability

Abstract

Online judge (OJ) systems are essential in programming education as they efficiently evaluate learners’ programming skills and reduce instructor workload. However, these systems often overlook the importance of software-testing concepts. To address this gap, we developed a system called Pytutor that integrates software-testing concepts to assess learners’ programming abilities and proficiency by exploring students’ test cases and learning behaviors. Drawing on software engineering theory and practical techniques, test capabilities are evaluated by analyzing the code coverage and mutation testing of Defining Test Cases. Since our experiment is conducted in an online environment, we can collect students’ learning behaviors and further analyze the relationship between software engineering abilities and learning behaviors. We also analyzed the differences in programming and testing abilities between computer science majors and non-computer-science majors. Our findings suggest that better testing abilities may contribute to the improvement in programming abilities, whereas in the current Taiwanese education context, computer science majors do not necessarily have better testing abilities. This result provides suggestions for us to strengthen software-testing education no matter which type of students it is targeted at.