Abstract
The assessment of multimedia accessibility is a relevant, complex and time-consuming task, which takes more than simply checking whether the video has audiodescription and captions or not. In our study, we face this challenge through the: 1) involvement of a cohort of novice evaluators, who previously took part in a MOOC on the accessibility of digital content and 2) the division of the accessibility assessment into the application of a set of criteria. Two groups of novice accessibility testers were asked to evaluate the accessibility of two similar videos, one video per group. While both videos were equivalent in terms of their pedagogical content, only one of them had non-severe accessibility barriers for people with low vision and for blind people. Each participant was asked to rate qualitatively a set of statements extracted from the WCAG 2.1 success criteria, one generic statement about the video accessibility, and a set of statements on the quality perception and the aspects of personal preference. The largest differences in ratings occurred for statements whose success criteria had been improved. It was also the case for one success criterion that is understandable but hardly applicable by novice evaluators, according to the literature. However, the difference was statistically significant only for the success criterion with more salient differences between both videos. As a main conclusion, a group of novice evaluators can identify accessibility problems in videos when using specific accessibility statements.
Highlights
Accessibility is recognised as a general principle by the United Nations Convention on the Rights of Persons with Disabilities [1]
Yesilada et al [5] concluded that expert evaluators spent significantly less time than non-experts, and that only expert evaluators were able to rate the accessibility of pages differently against different types of disability
They recognize that their research is not conclusive, but that it suggests that some parts of W3C Accessibility Guidelines (WCAG) 2.0 deserved special attention
Summary
Accessibility is recognised as a general principle by the United Nations Convention on the Rights of Persons with Disabilities [1]. Brajnik et al [4] concluded that an 80% agreement between experienced evaluators almost never occurred and that the average agreement was at the 70-75% mark, while the error rate was around 29%. They found that “trained—but novice—evaluators” performing the same audits exhibited the same agreement to that of the more experienced ones, but a reduction on validity of 6-13%, and that expertise improved by 19% the ability to avoid false positives. Alonso et al [6] found out that only 8 out of the 25 level-A SC of WCAG 2.0 were reliably testable by beginners. They recognize that their research is not conclusive, but that it suggests that some parts of WCAG 2.0 deserved special attention
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
