Support for reduced presentation durations in subjective video quality assessment

Abstract

Video content distributors, codec developers and researchers in related fields often rely on subjective assessments to ensure that their video processing procedures result in satisfactory quality. The current 10s recommendation for the length of test sequences in subjective video quality assessment, however, has recently been questioned. Not only do sequences of this length depart from modern cinematic shooting styles, the use of shorter sequences would also enable substantial efficiency improvements to the data collection process. Our previous work, using a double-stimulus methodology, indicated that shortening test sequences had a limited impact upon rating behaviour. Here, using a larger database and additional opinion score measures, we also explore the same effect within the popular single-stimulus approach. Two groups of viewers assessed reference and distorted videos ranging in length from 1.5s to 10s. Analyses confirmed our previous findings using the DSCQS paradigm, and were replicated when using a similar single-stimulus paradigm: while viewers' DMOS for 1.5s videos was significantly lower than for 10s, no significant variation was found between the groups of 10s, 7s and 5s videos. Together with our previous research, these data lead us to recommend the use of 5s, temporally-consistent video clips in quality assessment studies that employ either DSCQS or its single-stimulus variant. The extension of our recommendation to further methodologies is also discussed.