Evaluating user interaction with a web-based group decision support system: A comparison between two clustering methods

Abstract

Task-Technology Fit theory and the Technology Acceptance Model identify system utilization as an important indicator for the performance of complex software systems. Yet, empirical evaluations of user interaction with group decision support systems are scarce and often methodologically underdeveloped. For this study we employed an exploratory evaluation of user interaction in the context of web-based group decision support systems. Specifically, we used information-rich server logs captured through a web-based platform for participatory transportation planning to identify groups of users with similar use patterns. The groups were derived through multiple sequence alignment and hierarchical cluster analysis based on varying user activity measures. Subsequently, we assessed the reliability of the classifications obtained from the two clustering methods. Our results indicate limited reliability of classifications of activity sequences through multiple sequence alignment analysis and robust groupings from hierarchical cluster analysis for user activity initiations and durations. The presented work contributes a novel methodological framework for the evaluation of complex software systems that extends beyond the common approach of soliciting user satisfaction.