Automatic Lecture Annotation

Abstract

This Research to Practice Full Paper presents an application (AutoNotes) that improves the students’ learning outcomes by using natural language processing techniques (NLP) such as speech recognition and keywords extraction. AutoNotes is an android application that converts the speech of the instructor to text in real time during the lecture and then it synchronizes this text to the original lecture slides to produce new slides annotated with the explanation of the instructor. These new lecture slides can be displayed in a classroom. Moreover, it can be accessed by the students on their mobile phones. AutoNotes also provides a feature for the students who like to write their notes with their own way. A focus group was conducted to gather target audience opinions. The results have shown that students liked the idea of AutoNotes. They saw it beneficial for them especially if the original lecture slides are not detailed. They also like the idea of putting the notes with its corresponding slide. To evaluate AutoNotes, a between group design setup of two identical lectures with different groups of participants was conducted. In the first lecture, the lecturer used the original lecture slides. However, in the second one the annotated lecture slides was used. The participants were given several tests to compare their overall learning experiences. The results of the tests reveal that the participants that were exposed to the annotated lecture slides had a higher engagement level and learning gain than the other group exposed to the original lecture slides. Additionally, they had a workload level less than the other group. For the system usability test, the average usability scale that AutoNotes got is 78 which can be considered above average. Finally, the results of the accuracy testing reveal that the accuracy of the speech recognition was 83.2%. The precision and recall metrics were used to evaluate the synchronization of the speech with the slides. The average precision score was 75.6%. However, the average recall score was 51.1%.