Identifying Difficult exercises in an eTextbook Using Item Response Theory and Logged Data Analysis

Abstract

the growing dependence on eTextbooks and Massive Open Online Courses (MOOCs) has led to an increase in the amount of students’ learning data. By carefully analyzing this data, educators can identify difficult exercises, and evaluate the quality of the exercises when teaching a particular topic. In this study, an analysis of log data from the semester usage of the OpenDSA eTextbook was offered to identify the most difficult data structure course exercises and to evaluate the quality of the course exercises. Our study is based on analyzing students’ responses to the course exercises. We applied Item Response Theory (IRT) analysis and a Latent Trait Mode (LTM) to identify the most difficult exercises. To evaluate the quality of the course exercises we applied the IRT theory. Our findings showed that the exercises that related to algorithm analysis topics represented the most difficult exercises, and there existing six exercises were classified as poor exercises which could be improved or need some attention.