Evaluating a Taxonomy of Textual Uncertainty for Collaborative Visualisation in the Digital Humanities

Alejandro Benito-Santos,Roberto Therón,Aleyda Rocha,Jennifer Edmond,Eveline Wandl-Vogt,Michelle Doran

doi:10.3390/info12110436

Abstract

The capture, modelling and visualisation of uncertainty has become a hot topic in many areas of science, such as the digital humanities (DH). Fuelled by critical voices among the DH community, DH scholars are becoming more aware of the intrinsic advantages that incorporating the notion of uncertainty into their workflows may bring. Additionally, the increasing availability of ubiquitous, web-based technologies has given rise to many collaborative tools that aim to support DH scholars in performing remote work alongside distant peers from other parts of the world. In this context, this paper describes two user studies seeking to evaluate a taxonomy of textual uncertainty aimed at enabling remote collaborations on digital humanities (DH) research objects in a digital medium. Our study focuses on the task of free annotation of uncertainty in texts in two different scenarios, seeking to establish the requirements of the underlying data and uncertainty models that would be needed to implement a hypothetical collaborative annotation system (CAS) that uses information visualisation and visual analytics techniques to leverage the cognitive effort implied by these tasks. To identify user needs and other requirements, we held two user-driven design experiences with DH experts and lay users, focusing on the annotation of uncertainty in historical recipes and literary texts. The lessons learned from these experiments are gathered in a series of insights and observations on how these different user groups collaborated to adapt an uncertainty taxonomy to solve the proposed exercises. Furthermore, we extract a series of recommendations and future lines of work that we share with the community in an attempt to establish a common agenda of DH research that focuses on collaboration around the idea of uncertainty.

Highlights

The adequate capture and communication of uncertainty in computational processes is a long-standing challenge of computer science and related fields, such as information visualisation (IV) or the digital humanities (DH), while the introduction of uncertainty in the data analysis pipeline is known to greatly increase its complexity, it has many proven benefits in enabling users to make better-informed decisions based on data
“capta” makes reference to information that is consumed and interpreted by a human actor, constituting a perspective. This differentiation is especially important in the context of DH research, in which typically, there is a lack of ground truth personal claims can be compared to and, rather, the confrontation of multiple points of view or perspectives [2] is sought after, setting up an interesting scenario for collaboration centred around the uncertainty that is perceived by each participant
We present two user-driven experiences that sought to evaluate this taxonomy in the context of computer-supported collaborative annotation in the DH domain, in an attempt to understand how DH scholars and users can employ it to support collective reasoning in real-world analysis scenarios by setting the underlying uncertainty and data models that would govern the implementation of a visual collaborative annotation system (CAS)

Summary

Introduction

The adequate capture and communication of uncertainty in computational processes is a long-standing challenge of computer science and related fields, such as information visualisation (IV) or the digital humanities (DH), while the introduction of uncertainty in the data analysis pipeline is known to greatly increase its complexity, it has many proven benefits in enabling users to make better-informed decisions based on data For this reason, there is the question of how much uncertainty needs to be incorporated into such systems to make them useful while keeping them manageable and accessible at the same time, which typically depends on the task, users’ level of expertise, or degree of familiarity with the data, among other traits. This differentiation is especially important in the context of DH research, in which typically, there is a lack of ground truth personal claims can be compared to and, rather, the confrontation of multiple points of view or perspectives [2] is sought after, setting up an interesting scenario for collaboration centred around the uncertainty that is perceived by each participant

Objectives

Methods

Results

Discussion

Conclusion