User's Gaze Behavior Research Articles

Instant Reality: Gaze-Contingent Perceptual Optimization for 3D Virtual Reality Streaming.

Media streaming, with an edge-cloud setting, has been adopted for a variety of applications such as entertainment, visualization, and design. Unlike video/audio streaming where the content is usually consumed passively, virtual reality applications require 3D assets stored on the edge to facilitate frequent edge-side interactions such as object manipulation and viewpoint movement. Compared to audio and video streaming, 3D asset streaming often requires larger data sizes and yet lower latency to ensure sufficient rendering quality, resolution, and latency for perceptual comfort. Thus, streaming 3D assets faces remarkably additional than streaming audios/videos, and existing solutions often suffer from long loading time or limited quality. To address this challenge, we propose a perceptually-optimized progressive 3D streaming method for spatial quality and temporal consistency in immersive interactions. On the cloud-side, our main idea is to estimate perceptual importance in 2D image space based on user gaze behaviors, including where they are looking and how their eyes move. The estimated importance is then mapped to 3D object space for scheduling the streaming priorities for edge-side rendering. Since this computational pipeline could be heavy, we also develop a simple neural network to accelerate the cloud-side scheduling process. We evaluate our method via subjective studies and objective analysis under varying network conditions (from 3G to 5G) and edge devices (HMD and traditional displays), and demonstrate better visual quality and temporal consistency than alternative solutions.

Impact of bookshelf locations using eye-tracking methodology

PurposeThis study aimed to understand better the user gaze behavior on bookshelves using eye-tracking technology.Design/methodology/approachAn eye-tracking experiment in a public library with 11 participants was performed. The impact of vertical shelf location of books on the number of times the books are looked at, the impact of horizontal location and the relationship between user behavior and location impact were examined by the findings.FindingsThe results showed that the vertical location of books has a significant impact on the number of times the books are looked at. More than 80% of the time spent looking at bookshelves was spent on books on the top to fourth rows. It was also revealed that the horizontal location of books has a little impact. Books located on the left side of shelves will be looked at significantly more often than those on the right side. No significant relationships between type of user behaviors and location impact were observed.Originality/valueThe study explored the impact of the vertical location of books on time spent looking at bookshelves using eye-tracking methodology. Few published studies do such experiments to address user gaze behavior on bookshelves. The study explored that the vertical location of books has a great impact, and horizontal location has a little impact on user gaze behavior.

Prediction of individual learning curves across information visualizations

Confident usage of information visualizations is thought to be influenced by cognitive aspects as well as amount of exposure and training. To support the development of individual competency in visualization processing, it is important to ascertain if we can track users' progress or difficulties they might have while working with a given visualization. In this paper, we extend previous work on predicting in real time a user's learning curve--a mathematical model that can represent a user's skill acquisition ability--when working with a visualization. First, we investigate whether results we previously obtained in predicting users' learning curves during visualization processing generalize to a different visualization. Second, we study to what extent we can make predictions on a user's learning curve without information on the visualization being used. Our models leverage various data sources, including a user's gaze behavior, pupil dilation, and cognitive abilities. We show that these models outperform a baseline that leverages knowledge on user task performance so far. Our best performing model achieves good accuracies in predicting users' learning curves even after observing users' performance on a few tasks only. These results represent an important step toward understanding how to support users in learning a new visualization.

Gaze awareness in conversational agents

In face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation, and they change their conversational strategy if the listener is not fully engaged. With the goal of building a conversational agent that can adaptively control conversations, in this study we analyze listener gaze behaviors and develop a method for estimating whether a listener is engaged in the conversation on the basis of these behaviors. First, we conduct a Wizard-of-Oz study to collect information on a user's gaze behaviors. We then investigate how conversational disengagement, as annotated by human judges, correlates with gaze transition, mutual gaze (eye contact) occurrence, gaze duration, and eye movement distance. On the basis of the results of these analyses, we identify useful information for estimating a user's disengagement and establish an engagement estimation method using a decision tree technique. The results of these analyses show that a model using the features of gaze transition, mutual gaze occurrence, gaze duration, and eye movement distance provides the best performance and can estimate the user's conversational engagement accurately. The estimation model is then implemented as a real-time disengagement judgment mechanism and incorporated into a multimodal dialog manager in an animated conversational agent. This agent is designed to estimate the user's conversational engagement and generate probing questions when the user is distracted from the conversation. Finally, we evaluate the engagement-sensitive agent and find that asking probing questions at the proper times has the expected effects on the user's verbal/nonverbal behaviors during communication with the agent. We also find that our agent system improves the user's impression of the agent in terms of its engagement awareness, behavior appropriateness, conversation smoothness, favorability, and intelligence.

Analysis of Human Information Acquisition Behavior for Natural Gaze-Based Wheelchair Control

Especially for severely disabled people, a powered wheelchair is an important means to participate in societal life and live as far as possible independently. To achieve this goal for users, who cannot operate their wheelchair with the traditional joystick or specialty controls, methods have been developed to enable steering the wheelchair on the basis of the user's gaze behavior. While existing approaches require the user to adapt his/her gaze behavior to match the characteristics of the human-technology interaction and/or only provide reasoning about the desired motion direction of the user, the conducted study gives crucial input about the relationship between the gaze behavior of wheelchair users and the - from the user - desired goal position as well as his/her anticipated mission. Implications for a natural gaze-based assistance system for electrically powered wheelchairs are drawn, which allows reasoning on the user's behavioral goal position and his/her current mission.