An Ethnographic Evaluation of a Computer-Supported Product Development System

Abstract

This paper addresses the evaluation of a computer-supported collaborative work (CSCW) system, designed using a context-based participatory approach, to support product development. The evaluation focused on the effectiveness and acceptance of the CSCW system in its intended use environment. Users were asked to use the system for a period of 60 business days to perform actual product development tasks. The frequency of use of the system and several subjective measures were recorded at fixed intervals during the evaluation. The highest usage was during the fourth (final) period of the evaluation and the lowest usage was during the second. Users tended to agree that the development capability and the potential benefits to the organization of the CSCW system exceeded those of the existing system. The results support the use of a context-based participatory approach to developing features of a CSCW system that meet not only functional needs, but also socio-technical ones. Through the evaluation, it was possible to determine tasks that were appropriate and others that were inappropriate for future integration into the CSCW system. The evaluation methodology also identified issues that hindered the adoption of the CSCW system. Strategies for adoption that deal with these issues are proposed.