Advanced Usability Evaluation Methods

Abstract

Abstract : The Behavioral Sciences and Leadership Department at the United States Air Force Academy (USAFA) developed a low-cost human-computer interaction (HCI) laboratory in 2004. Since that time, the lab has grown into a teaching laboratory introducing HCI concepts to cadets in the Behavioral Sciences-Human Factors option as well as cadets in the Systems Engineering-Human Systems concentration. The HCI lab exposed cadets to contemporary methods and tools used in usability evaluation. The purpose of this final report is to document two studies recently conducted in the HCI laboratory. The first study examined the use of eye tracking as an advanced technique in determining the attentional focus of an evaluator watching a recorded usability highlight video. Current usability evaluation recording technology provides the usability practitioner with the capability to record audio, video of the user, and desktop screen activity in a picture-in-picture (PIP) format, allowing the evaluator to observe the interface screen and the human user simultaneously. The research question in the first study focused on how best to present the PIP video that is often displayed along with the desktop screen capture. In the second study, instructors in the Behavioral Sciences and Leadership Department were interested in examining the changes in a student's technique of identifying usability problems while using the HCI laboratory. Practitioners in the usability field have noted that experience contributes to the quality of usability problem reports, especially when that experience includes exposure to a framework for doing usability evaluation. Thirteen students in an undergraduate HCI course participated in this study during the Fall 2006 semester.