Empirical and Heuristic-Based Evaluation of Collaborative Modeling Systems: An Evaluation Framework

Abstract

Within the field of groupware systems, collaborative modeling systems play an important role. They are useful and promising tools for many fields of application. In this paper we begin with the evaluation of the usefulness of one of these tools, SPACE-DESIGN, by applying it to a specific domain and by comparing it to a single-user tool which is shared by means of a shared window system. We have performed a study in two phases in order to evaluate usability aspects and the awareness support of both software tools. The first phase is an empirical study with users who have worked with both software tools, whilst the second is a heuristic study with some heuristic models that are well known in the context of the research work. Taking into account this study, we propose a complete evaluation framework, to be used as a theoretical lab which could be used to evaluate interactive and usability aspects of any collaborative modeling tool. This is based on a comparison strategy and proposes a specific sequence of phases: selection of collaborative modeling tools, empirical evaluation, heuristic evaluation, analysis of results, and obtaining of conclusions.