Aspects on finding the optimal practical programming exercise for MOOCs

Abstract

Massive Open Online Courses (MOOCs) focus on manifold subjects, ranging from social sciences over languages to technical skills, and use different means to train the respective skills. MOOCs that are teaching programming skills aim to incorporate practical exercises into the course corpus to give students the hands-on experience necessary for understanding and mastering programming. These exercises, apart from technical challenges, come with a series of questions to be addressed, for example: which fraction of the participants' time should they take (compared to video lectures and other course activities), which difficulty should be aimed for, how much guidance should be offered and how much repetition should be incorporated? The perceived difficulty of a task depends on previous knowledge, supplied hints, the required time for solving and the number of failed attempts the participant made. Furthermore, the detail and accuracy of the problem description, the restrictiveness of the applied test cases and the preparation provided specifically for a given exercise also influence the perceived difficulty of a task. In this paper, we explore the data of three programming courses to find criteria for optimal practical programming exercises. Based on over 3 million executions and scoring runs of participants' task submissions, we aim to deduct exercise difficulty, student patterns in approaching the tasks and potential flaws in task descriptions and preparatory videos. We compare our findings to in class trainings and traditional, mostly video and quiz based MOOCs. Finally, we propose approaches and methods to improve programming courses for participants as well as instructors.