Automatic Online Lecture Highlighting Based on Multimedia Analysis

Abstract

Textbook highlighting is widely considered to be beneficial for students. In this paper, we propose a comprehensive solution to highlight the online lecture videos in both sentence- and segment-level, just as is done with paper books. The solution is based on automatic analysis of multimedia lecture materials, such as speeches, transcripts, and slides, in order to facilitate the online learners in this era of e-learning - especially with MOOCs. Sentence-level lecture highlighting basically uses acoustic features from the audio and the output is implemented in subtitle files of corresponding MOOC videos. In comparison with ground truth created by experts, the precision is over 60 percent, which is better than baseline works and also welcomed by user feedbacks. On the other hand, segment-level lecture highlighting works with statistical analysis, mainly by exploring the speech transcripts, the lecture slides and their connections. With the ground truth created by massive users, an evaluation process shows that general accuracy can reach 70 percent, which is fairly promising. Finally, we also attempt to find potential correlation between these two types of lecture highlights.