Automatic Identification of Widgets and their Subcomponents Based on a Classification Pipeline for DOM Mutation Records

Abstract

Since Web 2.0, web developers have increased the use of more sophisticated interaction mechanisms and visual effects, called widgets, to design the Rich Internet Applications' (RIA) user interface. However, many of the widgets currently available on websites do not implement accessibility design solutions standardized in the Accessible Rich Internet Applications (ARIA) specification and hence they are not accessible to disabled people. This paper come up with an approach for automatically classifying dropdown menu widgets and their subcomponents using a machine learning pipeline which analyses mutations that occur in RIAs' Document Object Model (DOM) structure, triggered by users interactions or visual effects. Classifying widgets and their subcomponents is an essential step for automatic evaluation of ARIA conformance and HTML code adaptation to mitigate accessibility issues. Thus, we also aim to take steps toward easing the software engineering process of RIAs in conformance with ARIA specifications. To validate, was conducted a case study with real websites to evaluate the proposed machine learning pipeline. The results provide evidences that our approach is capable of classifying dropdown menu widgets and their subcomponents with 0.91 F-measure in average.