A Comparison of Field-Based and Lab-Based Experiments to Evaluate User Experience of Personalised Mobile Devices

Xu Sun,Andrew May

doi:10.1155/2013/619767

Abstract

There is a growing debate in the literature regarding the tradeoffs between lab and field evaluation of mobile devices. This paper presents a comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices at large sports events. A lab experiment is recommended when the testing focus is on the user interface and application-oriented usability related issues. However, the results suggest that a field experiment is more suitable for investigating a wider range of factors affecting the overall acceptability of the designed mobile service. Such factors include the system function and effects of actual usage contexts aspects. Where open and relaxed communication is important (e.g., where participant groups are naturally reticent to communicate), this is more readily promoted by the use of a field study.

Highlights

Usability analysis of systems involving stationary computers has grown to be an established discipline within humancomputer interaction
Participants were able to identify some context related problems during the lab experiment
There has been much debate on whether mobile applications should be evaluated in the field or in a traditional lab environment, issues including users’ behaviour [11]; identification of usability problems [3]; the experiment settings [5, 8, 32]; the communication with participants [33]

Summary

Introduction

Usability analysis of systems involving stationary computers has grown to be an established discipline within humancomputer interaction. Researchers and practitioners have been encouraged to investigate further the criteria, methods, and data collection techniques for usability evaluation of mobile applications [2]. This paper focuses on content personalization [16], that is, the tailoring of information within a particular node within the human-device navigation space. This form of personalization is based on the key assumption that the optimal content for an individual is dependent on contextual factors relating to the individual, the situation they are in, and the activities they are undertaking—these factors can be used as triggers for the adaptation of content for the individual [17] in order to enhance their user experience

Methods

Results

Conclusion