Constructing and Evaluating Visualisation Task Classifications: Process and Considerations

N Kerracher,J Kennedy

doi:10.1111/cgf.13167

Abstract

AbstractCategorising tasks is a common pursuit in the visualisation research community, with a wide variety of taxonomies, typologies, design spaces, and frameworks having been developed over the last three decades. While these classifications are universally purported to be useful in both the design and evaluation processes and in guiding future research, remarkably little attention has been paid to how these frameworks have—and can be—constructed and evaluated. In this paper we review the task classification literature and report on current practices in construction and evaluation. We consider the stages of task classification construction and identify the associated threats to validity arising at each stage and in response to the different methods employed. We provide guidance on suitable validation approaches in order to mitigate these threats. We also consider the appropriateness of evaluation strategies according to the different aspects of the classification which they evaluate. In so doing, we seek to provide guidance for developers of classifications in determining appropriate construction and evaluation strategies when developing a classification, and also for those selecting between competing classifications for use in the design and evaluation processes.

Highlights

Categorising tasks is an increasingly common pursuit in visualisation research, with a variety of taxonomies, typologies, design spaces, frameworks, and models having been developed over the last three decades
While evaluation practices have become a topic of increasing interest in the visualisation community e.g. [LBI∗11, IIC∗13], very little has been written regarding the evaluation of formal models such as task classifications
While we have identified the threats to validity and potential validation approaches at each stage of classification construction (Section 5), these were not widely cited in the literature we reviewed

Summary

Introduction

Categorising tasks is an increasingly common pursuit in visualisation research, with a variety of taxonomies, typologies, design spaces, frameworks, and models having been developed over the last three decades. These classifications have been used to pre-empt and make sense of both peoples’ aims and intentions, and the ever increasing body of work relating to visualisation tools, techniques, and systems. The usefulness of these classifications in the design and evaluation processes is widely accepted. Very little attention has so far been given to the construction and evaluation practices involved in developing task classifications.

Methods

Results

Conclusion