Quality of the captions produced by students of an accessibility MOOC using a semi-automatic tool

Abstract

Some people have problems accessing multimedia services available on the web. Contributions to accessibility made by end users who lack institutional support in the design, production or deployment of media accommodations should be considered. The aim of our paper is to assess the quality of the captions produced using YouTube by non-professional subtitlers who only received basic training. We also identify potential improvements either in the subtitling tool or in the training resources, which could enhance the quality of the captions. We conducted a study in which 53 participants of a MOOC on digital accessibility used the automatic speech recognition (ASR) feature of YouTube to produce the captions for a video provided by the teaching staff. We assessed the quality of the captions produced by the students and then compared it with the quality of the captions produced by: (a) a human expert and (b) the ASR-based subtitling by YouTube. Students’ errors occurred mainly in the number of characters per line, the speed of the captions, failing to use a new line per participant, and not including sound effects. The course should warn students to use a new line per participant, teach them how to subtitle sound effects, specify the maximum number of characters per line of text, and inform that in some countries such as Spain, captions can be edited but in other countries this may not be possible. Our recommendations for the YouTube editor include improving both the user interface and the ASR, with a view to enhancing ongoing and future research.