Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials

Abstract

It is widely discussed whether the usefulness of mobile systems and services should be evaluated in the field or in the lab. The aim of this paper is to make a case for conducting longitudinal field trials when evaluating the usefulness of mobile services, and to partly base such evaluations on quantitative usage data automatically captured on the device. A framework for automatic capture and analysis of such data is introduced, and it is shown how it has been used to evaluate a mobile service for diabetes management. Results and experiences based on a three month field trial with a single diabetic user is presented and it is demonstrated how these results can be used to make inferences about the use of the service with regard to functionality and usage patterns.