‘Must See’ Videos? Why Educators Need Better Video Analytics to Measure Learning

Abstract

The pervasiveness of flipped classrooms and blended learning has tasked educators with creating content targeting fundamental concepts for asynchronous learning. The visual nature of anatomy subject matter makes video technology particularly well‐suited to convey important relationships. Education technology companies purport that video analytics may provide educators insights into their learners and the effectiveness of their videos. This study examines this claim using analytics recorded by a learning video platform during a 7‐week human anatomy course at the Mayo Clinic Alix School of Medicine (Arizona campus). First‐year medical students (n=48) were provided with daily faculty‐guided videos (n=21) of prosected cadavers [mean length ± SD (mm:ss) = 07:32 ± 3:28]. The videos highlighted structures covered during laboratory sessions. Each video was released less than 36 hours after the associated laboratory session. Video analytic data was recorded, including the number of times a video was viewed and played, average view time and the length of time a student watched a video before closing the page or pausing the video and never restarting it (drop‐off rate). Multiple linear regression analysis was conducted using JMP software to examine the relationship between these video analytic variables and laboratory practical examination performance. None of the video analytic variables, when calculated by individual video or individual student, were significant predictors of practical exam performance [mean exam performance = 88.83%; individual videos, R2 = 0.024, p = 0.78; individual student, R2 = 0.113, p = 0.63]. The results of this study are consistent with previous analyses of the educational effectiveness of videos in anatomy courses, in spite of the company’s claim that “videos can improve learning results”. While videos may make education more accessible and engaging, the impact on student learning is difficult to determine using the basic metrics supplied by the video learning platform. Until more sophisticated video analytics become available, such data will remain of limited use to educational researchers.