Software Usability Testing: Do User Self-Consciousness and the Laboratory Environment Make any Difference?

Abstract

The purpose of the study was to evaluate the effect of laboratory environment, user self-consciousness, and user experience on the user's subjective evaluation of software usability. The study employed a 2 × 2 × 2 factorial between-subjects design with 2 levels of Laboratory Environment (Cameras and Mirror vs. No Cameras and Mirrors), 2 levels of User Self-Consciousness (Low vs. High), and 2 levels of User Experience (Novice, Experienced). The users were asked to learn, then use, and finally subjectively evaluate a restricted subset of common word processing features over three hours of participation. Day 1 was a training day and Day 2 was a test day. Results indicated that high self-conscious and novice users make more word processing errors. However, they were no more likely to make those errors in the presence of cameras and a mirror. More importantly, the evidence for any effect of the independent variables on subjective evaluation was sparse–limited to the interaction of self-consciousness and laboratory environment on just three of twelve factors. Moreover, the pattern of these interactions indicated that self-consciousness and the laboratory environment did not influence subjective evaluation in any predictable manner. Despite some methodological shortcomings, the conclusion was drawn that these variables do not have a major impact on subjective evaluation of software usability.