Evaluating visual attention for multi-screen television: measures, toolkit, and experimental findings

Abstract

New and emerging multi-screen television scenarios and applications need new evaluation procedures, methodologies, and tools to support multi-screen data analysis. In this work, we introduce a set of 12 measures to characterize viewers’ visual attention patterns for multi-screen TV. Out of these, eight measures are computed directly from eye-tracking data, while the other four are evaluated using questionnaires. We applied our measures during a controlled experiment involving nine distinct screen layouts with two, three, and four TV screens, for which we report new findings about viewers’ distributions of visual attention. For example, we found that viewers need an average discovery time up to 4.5 s to visually fixate four screens, and their subjective perceptions of what they watched and for how long they watched each screen are substantially accurate, i.e., we report Pearson’s correlation coefficients up to r = .892 with ground truth measured with eye-tracking equipment. We also analyze and discuss the evolution of our participants’ distributions of visual attention over time from the perspective of our new set of measures. For example, we found that people perform significantly more transitions between screens during the first seconds of watching television, after which their level of visual attention converges to a stable value. We complement the findings revealed by our objective eye gaze measurements with subjective data about participants’ perceived cognitive work load and comfortability while watching more than one TV screen, and we measure viewers’ capacities to understand and recall content delivered simultaneously on multiple screens. To foster new studies and explorations of viewers’ visual attention patterns during multi-screen television watching, we release in the community an update for an existing software toolkit (i.e., VATic-TV, the Visual Attention Toolkit for TV, now at version v2) that automatically computes our measures from data delivered by standard eye-tracking equipment. We hope that our new set of measures and the companion software will benefit the community as a first step toward understanding visual attention for emerging multi-screen TV applications and, consequently, will help researchers and practitioners to design new TV applications that will better exploit viewers’ visual attention patterns toward new, richer television experiences.