Effectiveness of collaborative map-based decision support tools: Results of an experiment

Abstract

This article reports on the results of an empirical analysis of the effectiveness of a set of collaborative spatial decision support tools developed to support a land use allocation problem in a peat-meadow polder in the Netherlands. The tools feature spatial multicriteria analysis as the means to make spatially explicit trade-offs between stakeholder objectives in three different ways: as colors on multiple printed maps, qualitatively on a single digital map and quantitatively on a single digital map. An interactive mapping device is used as the interface between spatial information and participants. A series of controlled experiments was conducted with 30 participants, who were asked to use the tools and perform specific individual and group allocation tasks. Data on the responses of the participants were collected through questionnaires, observer notes, video film and multicriteria scores. The analysis focused on three aspects of effectiveness: usefulness of the tools; clarity of tool information; and impact on decisions. The findings of this analysis are discussed within the context of the information offered by the tools, individual and collaborative work of participants and the spatial and numerical quality of the resulting land use plans. From the experiments, it is clear that the cognitive effort related to the volume and format of information is a critical issue in spatial decision support. This holds true for both the level of detail per item of information and the amount of information to be supplied to the participants. Although the quantitative tool provided the most information, the participants did not consider this tool the most useful and it did not produce the best results.