Comparing Ontology Authoring Workflows with Protege: In the Laboratory, in the Tutorial and in the ‘Wild’

Abstract

The development of ontology engineering tools has traditionally lacked a user-centred perspective, instead being guided by the need to address particular gaps indicated by anecdotal evidence. This has typically resulted in prototypes that do not obtain traction beyond a narrow scope. Understanding the authoring patterns of ontology engineers is crucial to informing the development of ontology engineering tools that cater for the activity workflows of the users and, consequently, boosting the adoption of these tools. We report evidence about how Protege is used across three different authoring settings, addressing the threats to validity of relying on a single user study. These settings address the continuum of expertise (from intermediate to expert users), the type of tasks (whether they are free-form or prescriptive) and the effect of the location (laboratory, tutorial or on their own) and how the studies are administered (whether or not there is a close supervision). While there are activity workflows that are particular to settings, the results indicate a number of core workflows that are common to all of them. We discuss actionable recommendations for ontology engineering tools in light of these results.