Evaluating visualizations: using a taxonomic guide

Abstract

Although visualizations are components of many information interfaces, testing of these visual elements is rarely undertaken except as a part of overall usability testing. For this reason, it is unclear what role, if any, visualizations actually perform. Our method involves the creation of simple visual prototypes and task sets based on a visual taxonomy which allows testing of the visualization in isolation from the rest of the system. By defining tests using a visual taxonomy rather than customary tasks from the application domain, our method circumvents the problems of restricting evaluation of newer more capable systems to only those tasks which might be accomplished with older, less capable ones. This paper will discuss methods for exhaustively testing the capabilities of a visualization by mapping from a domain-independent taxonomy of visual tasks to a specific domain, i.e. information retrieval. Experimental results are presented illustrating this approach to determining the role visualizations may play in supporting users in information-seeking environments. Our methods could easily be extended to other domains including data visualization.