Gaze Behavior Research Articles

Can You Tell Real from Fake Face Images? Perception of Computer-Generated Faces by Humans

With recent advances in machine learning and big data, it is now possible to create synthetic images that look real. Face generation is often of particular interest, as faces can be used for various purposes. However, improper use of such content can lead to the dissemination of false information, such as fake news, and thus pose a threat to society. This work studies whether people believe the truthfulness of faces using eye tracking and self-reports, including free-form textual explanations when participants encounter real and computer-generated faces. We used three different datasets for our evaluations, and our experimental results show that while people are relatively better at identifying the truthfulness of real faces and faces generated by earlier machine learning algorithms with different gazing behaviors in viewing and rating phases, they perform less accurately when deciding the truthfulness of synthetic face images that are generated by newer algorithms. Our findings provide important insights for society and policymakers.

Examining gaze behavior in undergraduate students and educators during the evaluation of tooth preparation: an eye-tracking study

BackgroundGaze behavior can serve as an objective tool in undergraduate pre-clinical dental education, helping to identify key areas of interest and common pitfalls in the routine evaluation of tooth preparations. Therefore, this study aimed to investigate the gaze behavior of undergraduate dental students and dental educators while evaluating a single crown tooth preparation.MethodsThirty-five participants volunteered to participate in the study and were divided into a novice group (dental students, n = 18) and an expert group (dental educators, n = 17). Each participant wore a binocular eye-tracking device, and the total duration of fixation was evaluated as a metric to study the gaze behavior. Sixty photographs of twenty different tooth preparations in three different views (buccal, lingual, and occlusal) were prepared and displayed during the experimental session. The participants were asked to rate the tooth preparations on a 100 mm visual analog rating scale and were also asked to determine whether each tooth preparation was ready to make an impression. Each view was divided into different areas of interest. Statistical analysis was performed with a three-way analysis of the variance model with repeated measures.ResultsBased on the participants’ mean rates, the “best” and the “worst” tooth preparations were selected for analysis. The results showed a significantly longer time to decision in the novices compared to the experts (P = 0.003) and a significantly longer time to decision for both the groups in the best tooth preparation compared to the worst tooth preparation (P = 0.002). Statistical analysis also showed a significantly longer total duration of fixations in the margin compared to all other conditions for both the buccal (P < 0.012) and lingual (P < 0.001) views.ConclusionsThe current study showed distinct differences in gaze behavior between the novices and the experts during the evaluation of single crown tooth preparation. Understanding differences in gaze behavior between undergraduate dental students and dental educators could help improve tooth preparation skills and provide constructive customized feedback.

The predictive value of gaze behavior and mouse-clicking in testing listening proficiency: A sensor technology study

This study employed eye-tracking and mouse click frequency analysis to investigate the predictive power of gaze behaviors, mouse-clicking, and their interactive effects with linguistic backgrounds on the IELTS (International English Language Testing System) listening test scores. A total of 77 test takers (45 with English as their first language (E-L1) and 32 with English as their second language (E-L2)) participated in this study. Their eye movements and mouse click frequencies were recorded as they took a computer-based IELTS listening test. The subsequent data analysis, utilizing linear mixed models, showed that gaze patterns, mouse actions, and language background significantly predicted listening test outcomes across four listening test sections and between E-L1 and E-L2 candidates, accounting for 33.2% of the variance observed in test scores. These results indicate the effect of potential sources of construct-irrelevant variance on test scores, which are not predicted in the available construct definitions of the test used in the study. Implications for the listening construct and test validity are discussed.

Role of gaze behaviors, body movements, and bicycle movements during cycling on a straight and narrow path

Cycling requires the integration of gaze behaviors, body movements, and bicycle movements. However, whether these movements contribute to skilled cycling performance, such as cycling on straight and narrow paths are uncertain. The present study aimed to differentiate optokinetic nystagmus (OKN) from vestibulo-ocular reflex (VOR) that characterize the relationship between eye and head movements during cycling on straight and narrow path, and to identify gaze behaviors, body movements, and bicycle movements that contribute to cycling performance. Nineteen participants with no prior competitive experience cycled three times on a 12-cm wide path. The participants were asked to avoid deviating from the path as much as possible. The measured variables were gaze behavior in a sagittal plane, body movement, and bicycle movement. As a result, OKN was observed among 16 of the 19 participants. The cross-correlation between the eye and head did not show negative value, indicating the absence of VOR. These results suggest that the participants moved their eyes while keeping their heads stable during cycling on a straight and narrow path. In the results of the multiple regression analysis, the variables with small standard deviations (SD) of the steering angle and upward eye position were related to a lower deviation from the path. These results suggest that a small SD of the steering angle and directed gaze in the forward direction may contribute to skilled cycling.

Who you lookin' at? Perception of gaze direction in group settings depends on naturalness of gaze behavior and clutter

Who you lookin' at? Perception of gaze direction in group settings depends on naturalness of gaze behavior and clutter

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes

Audiovisual speech perception of multilingual learners of Japanese

ABSTRACT Due to the complex nature of languages and the way they are used, it is often difficult to research multilinguals because of the considerable variation in the ways their spoken languages may impact their perceptions. For this reason, foreign language education research has largely been focused on English L2 speakers, overlooking how different languages might impact perception in multilingual speakers. Focusing on Japanese as a Second Language (JSL) learners, this research investigated how multilinguals responded to audiovisual cues in Japanese and English. Utilising web-based eye-tracking technology facilitated by WebGazer.js. Participants were asked to complete a novel phonetic distinction task while their gaze behaviour was measured. Contrary to initial theoretical expectations, the findings did not reveal significant mediation by gaze behaviour on the relationship between multilingualism and task performance. However, the study found that when looking at the duration of fixation on the mouth, there's a significant difference between native English speakers and native speakers of Asian languages when listening to English or Japanese. These findings indicate that further research that explores the nuances of gaze behaviour among multilingual learners of non-English languages (such as Japanese) is necessary.

Gaze behavior in open-angle glaucoma patients during visuo-cognitive-motor tasks: a cross-sectional study

This study investigated gaze behavior during visuo-cognitive-motor tasks with a change of movement direction in glaucoma patients and healthy controls. Nineteen glaucoma patients (10 females, 9 males) and 30 healthy sighted controls (17 females, 13 males) participated in this cross-sectional study. Participants performed two visuo-cognitive-motor tasks with a change of movement direction: (i) the “Speed-Court-Test” that involved stepping on different sensors in response to a visual sign displayed on either a large or small screen (165″ and 55″, respectively); (ii) the “Trail-Walking-Test” that required walking to 15 cones labeled with numbers (1–8) or letters (A-G) in an alternately ascending order. During these tasks, the time needed for completing each task was determined and the gaze behavior (e.g., saccade duration, fixation duration) was recorded via eye tracking. Data were analyzed with repeated measures analyses of covariance (ANCOVA; GROUP × SCREEN) and one-way ANCOVA. No differences between groups were found for the time needed to complete the tasks. However, during the “Trail-Walking-Test”, the fixation duration was longer for glaucoma patients than for controls (p = 0.016, ηp2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\upeta }_{\ ext{p}}^{2}$$\\end{document} = 0.131). Furthermore, during the “Speed-Court-Test”, there was a screen size effect. Irrespective of group, saccade amplitudes were lower (p < 0.001, ηp2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\upeta }_{\ ext{p}}^{2}$$\\end{document} = 0.242) and fixation durations were higher (p = 0.021, ηp2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\upeta }_{\ ext{p}}^{2}$$\\end{document} = 0.125) for the small screen. Fixation durations were longer in glaucoma patients during the cognitively demanding “Trail-Walking-Test”, which might indicate a strategy to compensate for their visual impairment.

Enhancing social robot's direct gaze expression through vestibulo-ocular movements

Non-verbal communication, especially eye contact, plays a crucial role in human interaction. Integrating eye contact capabilities into robotic behavior presents challenges that require comprehensive interdisciplinary collaboration from the fields of vision science and robotics. This study introduces an innovative methodology to improve the user's perception of the tabletop social robot Haru's direct gaze by explaining how eye and head movements can be managed through simulating visual perception processing. By considering the mechanisms of eye and head movements during visual perception processing, the robot's eye and head gaze behaviors are intended to replicate human gaze movements with Vestibulo-Ocular Movement (VOM) during face-to-face interaction, ultimately enhancing the user's perception of the robot's direct gaze.

Gaze modeling in multi-party dialogues and extraversion expression through gaze aversion control

Gaze control plays a crucial role in the context of conversational robots involving multiple participants. Particularly, gaze aversion is not only important for improving human likeness but is also strongly associated with the expression of a robot's personality and should be accounted for dialogue interactions. While previous studies on gaze models considering multi-party conversations have been conducted, those considering gaze aversion are limited. In this study, we developed a gaze motion generation model, focusing on three features: (1) the gaze target (towards dialogue partners or gaze aversion), (2) the gaze duration, and (3) the eyeball direction during gaze aversion. We extracted the gaze model parameters of two individuals with distinct extraversion from a multimodal three-party dialogue database and created a model that can modulate extraversion expression by setting a gaze aversion ratio parameter. We implemented this model on an android robot, which can reproduce human-like gaze behaviors. Video-based experiments have been conducted where participants watched videos of the robot talking with two humans, replicating the three-party dialogue scenarios used in this study, and evaluated the impressions on the robot's extraversion. The results demonstrated that by controlling gaze aversion parameters, the robot could exhibit various levels of extraversion.

A new graph-based metric for modeling aggregated gaze behavior differences during map reading

A new graph-based metric for modeling aggregated gaze behavior differences during map reading

Impact of Socio-Demographic Attributes and Mutual Gaze of Virtual Humans on Users' Visual Attention and Collision Avoidance in VR.

This study investigated the extent that the non-verbal behaviors of virtual humans (VHs) and their socio-demographic attributes altered users' collision avoidance behaviors in Virtual Reality (VR). Users interacted with VHs representing different levels of ethnicities and gender, exhibiting different conditions of physical movement, and gaze behaviors. The VHs were depicted in three major ethnic conditions namely Asian, Caucasian, and Black. The physical movement states of the VHs were either static in the path of the user or walking toward the user in the opposite direction. The non-verbal gaze behavior of the VHs was either direct gaze or averted gaze. We used an HTC Vive tracking system to track users' performing real walking while we collected objective measures (i,e., continuous gaze, fixation gaze, clearance distance, and travel length), and subjective variables (i.e., game experiences and social presence). The results showed that the ethnicity of the VHs significantly impacted the gaze behavior of the users, and the gender of the VHs affected the user avoidance movement and their reciprocal gaze behavior. Our results revealed that users' physical movement, gaze behaviors, and collision avoidance were moderated by the VHs' perceived ethnicity, gender, and gaze behaviors. Understanding the impact of the socio-demographics attributes of VHs and their gaze behavior on users' collision avoidance is critical for applications in which users are navigating through virtual traffic, crowd, and other inter-personal simulations.

Using eye tracking to assess learning of a multifunction prosthetic hand: an exploratory study from a rehabilitation perspective

BackgroundEye tracking technology not only reveals the acquisition of visual information at fixation but also has the potential to unveil underlying cognitive processes involved in learning to use a multifunction prosthetic hand. It also reveals gaze behaviours observed during standardized tasks and self-chosen tasks. The aim of the study was to explore the use of eye tracking to track learning progress of multifunction hands at two different time points in prosthetic rehabilitation.MethodsThree amputees received control training of a multifunction hand with new control strategy. Detailed description of control training was collected first. They wore Tobii Pro2 eye-tracking glasses and performed a set of standardized tasks (required to switch to different grips for each task) after one day of training and at one-year-follow-up (missing data for Subject 3 at the follow up due to socket problem). They also performed a self-chosen task (free to use any grip for any object) and were instructed to perform the task in a way how they would normally do at home. The gaze-overlaid videos were analysed using the Tobii Pro Lab and the following metrics were extracted: fixation duration, saccade amplitude, eye-hand latency, fixation count and time to first fixation.ResultsDuring control training, the subjects learned 3 to 4 grips. Some grips were easier, and others were more difficult because they forgot or were confused with the switching strategies. At the one-year-follow-up, a decrease in performance time, fixation duration, eye-hand latency, and fixation count was observed in Subject 1 and 2, indicating an improvement in the ability to control the multifunction hand and a reduction of cognitive load. An increase in saccade amplitude was observed in both subjects, suggesting a decrease in difficulty to control the prosthetic hand. During the standardized tasks, the first fixation of all three subjects were on the multifunction hand in all objects. During the self-chosen tasks, the first fixations were mostly on the objects first.ConclusionThe qualitative data from control training and the quantitative eye tracking data from clinical standardized tasks provided a rich exploration of cognitive processing in learning to control a multifunction hand. Many prosthesis users prefer multifunction hands and with this study we have demonstrated that a targeted prosthetic training protocol with reliable assessment methods will help to lay the foundation for measuring functional benefits of multifunction hands.

Examining the effect of locomotion techniques on virtual prototype assessment: Gaze analysis using a Head-Mounted Display

Improvements in the performance and graphical quality of Head-Mounted Displays (HMDs) have led to their increasing use in Virtual Reality (VR) for product presentation and virtual prototype (VP) evaluations. Various locomotion techniques in VR make it possible to move through a virtual scenario and approach the VP for evaluation purposes. The integration of eye-tracking devices into recent HMDs allows the trajectory and gaze behavior of observers to be reported during the evaluation, often more objectively than self-report questionnaires. However, very few studies have used physiological measures for the evaluation of products embedded in VR environments. Therefore, this paper offers a study in which 95 people evaluated three VPs of street furniture presented in their environment of use using Meta Quest Pro headset and explored through teleport and natural walking. The influence of the locomotion techniques on the ratings recorded using a semantic differential, sense of presence, cybersickness, and the role of eye-tracking in understanding gaze behavior while evaluating products' Areas of Interest (AOIs), are investigated. This study found no evidence that the way of approaching the product influences the evaluation of some of its features, overall product evaluation, confidence in responses, sense of presence, or cybersickness differently. On the other hand, this work evidences that the locomotion technique had an impact on how the user approached the products, which could significantly influence the viewing time of some AOIs. The study revealed that the most observed AOIs coincided with those parts closely related to important features, generally located at the top of the products, so paying special attention to these parts when designing and evaluating similar VPs is recommended.

Tests of abnormal gaze behavior increase the accuracy of mild cognitive impairment assessments

Tests to detect mild cognitive impairment (MCI) should have high sensitivity and accuracy. Previously, we reported on a cognitive composition test (CCT) that we developed to detect MCI. In this study, we compared gaze behavior parameters, namely, gaze fixation duration and gaze-shift frequency, during the CCT in participants with MCI and healthy controls (HC) to determine whether these parameters would accurately detect MCI in older adults. Participants performed CCT-A, -B, and -C tasks with varying difficulty levels while wearing eye-tracking devices. Performance time, gaze fixation duration, and gaze-shift frequency were analyzed. Receiver operating characteristic curve analysis was performed to assess the identification accuracy. The MCI group was significantly slower in completing the CCT-C task and had a higher gaze-shift frequency into both the sample object space and workspace than the HC group. Gaze fixation duration in the sample object space increased in the MCI group as the CCT became difficult. Our findings indicated that combining the CCT with performance time and gaze pattern improved the accuracy of distinguishing between individuals with and without MCI and that patients with MCI have abnormal gaze behavior during cognitive tasks. Therefore, evaluation of gaze parameters may improve the accuracy of identifying patients with MCI.

Human Eyes-Inspired Recurrent Neural Networks Are More Robust Against Adversarial Noises.

Humans actively observe the visual surroundings by focusing on salient objects and ignoring trivial details. However, computer vision models based on convolutional neural networks (CNN) often analyze visual input all at once through a single feedforward pass. In this study, we designed a dual-stream vision model inspired by the human brain. This model features retina-like input layers and includes two streams: one determining the next point of focus (the fixation), while the other interprets the visuals surrounding the fixation. Trained on image recognition, this model examines an image through a sequence of fixations, each time focusing on different parts, thereby progressively building a representation of the image. We evaluated this model against various benchmarks in terms of object recognition, gaze behavior, and adversarial robustness. Our findings suggest that the model can attend and gaze in ways similar to humans without being explicitly trained to mimic human attention and that the model can enhance robustness against adversarial attacks due to its retinal sampling and recurrent processing. In particular, the model can correct its perceptual errors by taking more glances, setting itself apart from all feedforward-only models. In conclusion, the interactions of retinal sampling, eye movement, and recurrent dynamics are important to human-like visual exploration and inference.

Predicting and Optimizing Restorativeness in Campus Pedestrian Spaces based on Vision Using Machine Learning and Deep Learning

Restoring campus pedestrian spaces is vital for enhancing college students’ mental well-being. This study objectively and thoroughly proposed a reference for the optimization of restorative campus pedestrian spaces that are conducive to the mental health of students. Eye-tracking technology was employed to examine gaze behaviors in these landscapes, while a Semantic Difference questionnaire identified key environmental factors influencing the restorative state. Additionally, this study validated the use of virtual reality (VR) technology for this research domain. Building height difference (HDB), tree height (HT), shrub area (AS), ground hue (HG), and ground texture (TG) correlated significantly with the restorative state (ΔS). VR simulations with various environmental parameters were utilized to elucidate the impact of these five factors on ΔS. Subsequently, machine learning models were developed and assessed using a genetic algorithm to refine the optimal restorative design range of campus pedestrian spaces. The results of this study are intended to help improve students’ attentional recovery and to provide methods and references for students to create more restorative campus environments designed to improve their mental health and academic performance.

Top-Down Priors Disambiguate Target and Distractor Features in Simulated Covert Visual Search.

Several models of visual search consider visual attention as part of a perceptual inference process, in which top-down priors disambiguate bottom-up sensory information. Many of these models have focused on gaze behavior, but there are relatively fewer models of covert spatial attention, in which attention is directed to a peripheral location in visual space without a shift in gaze direction. Here, we propose a biologically plausible model of covert attention during visual search that helps to bridge the gap between Bayesian modeling and neurophysiological modeling by using (1) top-down priors over target features that are acquired through Hebbian learning, and (2) spatial resampling of modeled cortical receptive fields to enhance local spatial resolution of image representations for downstream target classification. By training a simple generative model using a Hebbian update rule, top-down priors for target features naturally emerge without the need for hand-tuned or predetermined priors. Furthermore, the implementation of covert spatial attention in our model is based on a known neurobiological mechanism, providing a plausible process through which Bayesian priors could locally enhance the spatial resolution of image representations. We validate this model during simulated visual search for handwritten digits among nondigit distractors, demonstrating that top-down priors improve accuracy for estimation of target location and classification, relative to bottom-up signals alone. Our results support previous reports in the literature that demonstrated beneficial effects of top-down priors on visual search performance, while extending this literature to incorporate known neural mechanisms of covert spatial attention.

Sight-singing in a group context: An eye-tracking study with experienced choral singers

Studies on music reading have provided insights into the cognitive processes in sight singing. However, there has been limited research on sight singing in a group setting. This study aims to assess how choral singers approach music reading as they perform previously unfamiliar choral scores together. We addressed (1) how singers’ gaze direction is distributed over staff systems, and (2) how their gaze behaviour is influenced by note density and repeated practice. Four quartets, a total of sixteen experienced singers, performed eight Baroque music excerpts three times, while the singers’ eye movements were recorded. Eye-movement measures were analysed in conjunction with the singers’ views regarding their music reading, obtained through questionnaires and group discussions. Results reveal that besides reading their own voice lines, singers typically inspected the neighbouring lines, seeking visual cues to coordinate the performance. The results of a generalised linear mixed model analysis underscore the substantial influence of note density on fixation durations on one’s own lines, but not on other voices’ lines. Practice, on the other hand, exhibited effects only on average fixation duration for one’s own lines, with no significant impact observed on other lines. The study provides evidence of coordination between an actual sight-reading process and a parallel information-gathering process that helps singers relate their parts to the overall musical structure.

Eye-Tracking Based Visual Search Training in Social Anxiety: Effects on Attentional Bias, Attentional Control, Gaze Behavior, and Anxious Responses to a Speech Task

Eye-Tracking Based Visual Search Training in Social Anxiety: Effects on Attentional Bias, Attentional Control, Gaze Behavior, and Anxious Responses to a Speech Task