Visual Gaze Research Articles

Neural network based reinforcement learning for audio–visual gaze control in human–robot interaction

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human–robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust in terms of parameter configuration, i.e. the selection of the parameter values employed by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of a socially acceptable gaze behavior.

Subtle behavioural responses during negative emotion reactivity and down-regulation in bipolar disorder: A facial expression and eye-tracking study

Abnormal processing of emotional information and regulation are core trait-related features of bipolar disorder (BD) but evidence from behavioural studies is conflicting. This study aimed to investigate trait-related abnormalities in emotional reactivity and regulation in BD using novel sensitive behavioural measures including facial expressions and eye movements. Fifteen patients with BD in full or partial remission and 16 healthy controls (HCs) were given a computerised task in which they were instructed to ‘react to’ unpleasant and neutral pictures or ‘dampen’ their emotional response to aversive pictures. Participants rated their emotional response after each picture block, and eye-movements and facial expressions were recorded. Patients generally gazed less at unpleasant and neutral pictures during emotion processing and regulation. During emotional reactivity, patients exhibited stronger facial expressions to neutral pictures, a lack of facial expressions to unpleasant pictures, and more surprised facial expressions to both neutral and unpleasant pictures compared to HCs. The results point to subtle abnormalities in visual gaze patterns and facial displays of emotion between patients with BD during emotion processing and regulation. Aberrant eye-movements and facial displays could provide a more sensitive measure of emotional reactivity in BD than traditional behavioural measures, which could aid future diagnostic accuracy.

Visual Grouping in Accordance With Utterance Planning Facilitates Speech Production.

Research on language production has focused on the process of utterance planning and involved studying the synchronization between visual gaze and the production of sentences that refer to objects in the immediate visual environment. However, it remains unclear how the visual grouping of these objects might influence this process. To shed light on this issue, the present research examined the effects of the visual grouping of objects in a visual display on utterance planning in two experiments. Participants produced utterances of the form “The snail and the necklace are above/below/on the left/right side of the toothbrush” for objects containing these referents (e.g., a snail, a necklace and a toothbrush). These objects were grouped using classic Gestalt principles of color similarity (Experiment 1) and common region (Experiment 2) so that the induced perceptual grouping was congruent or incongruent with the required phrasal organization. The results showed that speech onset latencies were shorter in congruent than incongruent conditions. The findings therefore reveal that the congruency between the visual grouping of referents and the required phrasal organization can influence speech production. Such findings suggest that, when language is produced in a visual context, speakers make use of both visual and linguistic cues to plan utterances.

Visual Gaze Patterns Are Again Associated with Polyp Detection

Many factors correlate with polyp detection during colonoscopy, including bowel preparation quality, withdrawal time, examination technique, and a

The effect of assuming static or dynamic gaze behaviour on the estimated background luminance of drivers

Application of the Commission Internationale de l’Eclairage system for mesopic photometry requires that an estimate is made of the observers’ state of luminance adaptation. This paper addresses an assumption made when estimating background luminance, a component of adaptation luminance. Specifically, using spatial sampling of the visual field we compare background luminances determined from assumptions of static or dynamic visual gaze, the former being a simplification, the latter being a better representation based on eye movements when driving. The comparison was undertaken with luminance images of urban scenes at night on three roads, two real and one simulated. It was found that background luminances were significantly higher when estimated using the dynamic assumption. It was also found that scene luminances at the point of foveal fixations tend to be higher than those luminances influencing peripheral regions of the retina. Compared with the background luminance estimated for a dynamic peripheral field, a horizon-centred 10° circle led to a slightly higher estimate and the road surface luminance to a slightly lower estimate of background luminance.

Gaze patterns hold key to unlocking successful search strategies and increasing polyp detection rate in colonoscopy.

The adenoma detection rate (ADR) is an important quality indicator in colonoscopy. The aim of this study was to evaluate the changes in visual gaze patterns (VGPs) with increasing polyp detection rate (PDR), a surrogate marker of ADR. 18 endoscopists participated in the study. VGPs were measured using eye-tracking technology during the withdrawal phase of colonoscopy. VGPs were characterized using two analyses - screen and anatomy. Eye-tracking parameters were used to characterize performance, which was further substantiated using hidden Markov model (HMM) analysis. Subjects with higher PDRs spent more time viewing the outer ring of the 3 × 3 grid for both analyses (screen-based: r = 0.56, P = 0.02; anatomy: r = 0.62, P < 0.01). Fixation distribution to the "bottom U" of the screen in screen-based analysis was positively correlated with PDR (r = 0.62, P = 0.01). HMM demarcated the VGPs into three PDR groups. This study defined distinct VGPs that are associated with expert behavior. These data may allow introduction of visual gaze training within structured training programs, and have implications for adoption in higher-level assessment.

Eye-tracking technology in medical education: A systematic review

Background: Eye-tracking technology is an established research tool within allied industries such as advertising, psychology and aerospace. This review aims to consolidate literature describing the evidence for use of eye-tracking as an adjunct to traditional teaching methods in medical education.Methods: A systematic literature review was conducted in line with STORIES guidelines. A search of EMBASE, OVID MEDLINE, PsycINFO, TRIP database, and Science Direct was conducted until January 2017. Studies describing the use of eye-tracking in the training, assessment, and feedback of clinicians were included in the review.Results: Thirty-three studies were included in the final qualitative synthesis. Three studies were based on the use of gaze training, three studies on the changes in gaze behavior during the learning curve, 17 studies on clinical assessment and six studies focused on the use of eye-tracking methodology as a feedback tool. The studies demonstrated feasibility and validity in the use of eye-tracking as a training and assessment method.Conclusions: Overall, eye-tracking methodology has contributed significantly to the training, assessment, and feedback practices used in the clinical setting. The technology provides reliable quantitative data, which can be interpreted to give an indication of clinical skill, provide training solutions and aid in feedback and reflection. This review provides a detailed summary of evidence relating to eye-tracking methodology and its uses as a training method, changes in visual gaze behavior during the learning curve, eye-tracking methodology for proficiency assessment and its uses as a feedback tool.

P 144 Longer-term spatial navigation deficits in transient global amnesia – a follow-up study

Introduction Transient global amnesia (TGA) is a neurological syndrome that typically presents with sudden and severe anterograde amnesia and spatial orientation deficits. Although acute symptoms last for less than 24 h, subtle deficits may persist longer. The aim of the current study was to investigate spatial orientation performance in a real world environment longitudinally after onset of TGA. Methods We present the results of 19 healthy subjects and 21 TGA patients, who completed a spatial navigation task in a real-space environment. The TGA patients performed the task on two occasions, firstly within ∼ 3.15 days of their attack (post-acute TGA), secondly a follow up repetition (follow up) some months later. The spatial navigation task consisted of 5 distributed target items in a novel environment. In the exploration phase (10 min) of the task, the examiner took the subject to the exact location of each target item. In the navigation phase (10 min) the subject had to find each target item in a pseudo-randomised order. Throughout, the subjects wore a gaze-controlled head camera, allowing recording of individual ”navigograms” for visual exploration, trajectory, and gaze behaviour. An MRI was done in all patients during the acute phase of the TGA. Results Post-acute TGA patients showed a significantly worse spatial navigation performance when compared to healthy controls ( p p Discussion The higher number of error rates and non-use of short cuts during the task suggest that post-acute TGA patients are unable to generate cognitive maps of their environment. Spatial navigation deficits are - although to a lesser extent - still present several months after the initial presentation of symptoms, despite subjective memory symptoms have gone. These data challenge the concept of TGA as a short-lasting transient syndrome and indicate a longer-lasting hippocampal pathology in this disorder.

Watch Out! Or do we? A Review of Literature on Human Locomotion, Visual Gaze, and Expectation of Trip Hazards

Pedestrian trip and fall events, a major source of accidental injury and death, are often the result of a vertical perturbation that goes undetected by a pedestrian. Inattention is frequently blamed as the underlying root cause of the fall. However, research suggests that a pedestrian’s visual gaze is typically focused out towards the horizon and not towards the ground in the immediate vicinity of their feet, leaving them with poor visual acuity to detect subtle details on the walking surface. To the extent a vertical perturbation is unexpected, the likelihood of a trip event is increased. Relevant research on gait cycles, toe clearance, visual gaze, and expectation is examined to determine why pedestrians may fail to detect walkway hazards that are physically possible to see. Synthesis and future research are also discussed.

Driver Hazard Detection and Avoidance Performance as a Function of Eyes-Off-Road Interval Under Partially Automated Driving

There has been concern that automation may impair a person’s ability to detect and appropriately respond to environmental hazards. This study sought to examine a driver’s ability to not only detect hazards, but also to implement appropriate responses following extended Eyes-Off-Road (EOR) intervals. Thirty-six drivers were exposed to a series of filmed roadway vignettes with discernible hazards embedded within a range of traffic situations. Driver’s visual gaze was directed “off-road” for 2, 7 or 20 seconds, and then re-oriented to the roadway. Drivers were tasked to respond appropriately as quickly as possible once their gaze returned to the forward road scene by steering and/or braking in response to the hazards. Results found no significant differences among the EOR intervals in terms of the driver’s ability to detect and appropriately respond to the hazard when it emerged after the driver’s eyes were directed forward.

The Impact of Accompanying Text on Visual Processing and Hedonic Evaluation of Art

This study explores how accompanying text affects the way an individual views and interprets a painting. We randomly assigned participants to view 20 paintings from the classical era with factual information, contextualized background information, or no information displayed next to them. We then recorded their visual gaze using an eye-tracking device and asked them to evaluate the paintings. The results show that how people view a painting and how they evaluate a painting are two distinct cognitive processes. The contextual information serves to orient the viewing process. The accompanying text influences the visual attention and gaze pattern but has limited impact on the hedonic evaluation of paintings. Instead, hedonic evaluation is more of a taste acquired through education and socialization. This study offers an empirical footnote to discussions on the cognitive assumptions in sociological studies of art and cultural phenomena.

The impact of visual gaze direction on auditory object tracking

Subjective experience suggests that we are able to direct our auditory attention independent of our visual gaze, e.g when shadowing a nearby conversation at a cocktail party. But what are the consequences at the behavioural and neural level? While numerous studies have investigated both auditory attention and visual gaze independently, little is known about their interaction during selective listening. In the present EEG study, we manipulated visual gaze independently of auditory attention while participants detected targets presented from one of three loudspeakers. We observed increased response times when gaze was directed away from the locus of auditory attention. Further, we found an increase in occipital alpha-band power contralateral to the direction of gaze, indicative of a suppression of distracting input. Finally, this condition also led to stronger central theta-band power, which correlated with the observed effect in response times, indicative of differences in top-down processing. Our data suggest that a misalignment between gaze and auditory attention both reduce behavioural performance and modulate underlying neural processes. The involvement of central theta-band and occipital alpha-band effects are in line with compensatory neural mechanisms such as increased cognitive control and the suppression of task irrelevant inputs.

Direct and indirect effects of attention and visual function on gait impairment in Parkinson's disease: influence of task and turning (Commentary on Stuart et al. (2017)).

Direct and indirect effects of attention and visual function on gait impairment in Parkinson's disease: influence of task and turning (Commentary on Stuart et al. (2017)).

Sa1071 A Look in the Right Place: Optimal Gaze Strategy for Colonoscopy: A Systematic Review and Comparative Analysis

Recent research has revealed differences in the gaze control strategies of experienced and novice laparoscopic surgeons. When acquiring endoscopic techniques, the gastroenterologists must overcome environmental constraints on visuomotor control, such as loss of depth perception, poor tactile feedback, and the use of long instruments that amplify movement error and exhibit a fulcrum effect. Little is known about the visual gaze pattern (VGP) that may impact colonoscopy effectiveness. This study examined the optimal gaze strategy for colonoscopy and effect on adenoma detection through a systematic review and comparative analysis.

On the Role of Color in Visual Saliency

Visual attention refers to the cognitive mechanism that allows us to select and process only the relevant information arriving at our eyes. Therefore, eye movements will have a significant dependency on visual attention. Saliency models, trying to simulate visual gaze and consequently, visual attention, have been continuously developed over the last years. Color information has been shown to play an important role in visual attention, and it is used in saliency computations. However, psychophysical evidence explaining the relationship between color and saliency is lacking. The results of the experiment will be presented aiming at studying and quantifying saliency of colors of different hues and lightness specified in CIELab coordinates. In the experiment, 12 observers were asked to report the number of color patches presented at random locations on a masking gray background. Eye movements were recorded using an SMI remote eye tracking system and being used to validate the reported data. In the presentation, we will compare the reported data and visual gaze data for different colors and discuss implications for our understanding of color saliency and color processing.

Metoclopramide-induced oculogygyric crisis: Report of three cases

Metoclopramide is a dopamine (D2) receptor blocker, which is widely used in emergency departments (EDs) in the treatment of patients with nausea and emesis. The primary side effect is acute extrapyramidal reactions requiring acute intervention. Acute dystonic reaction incidence is 0.5-1% in children while incidence in young adults and elders is 25%. Oculogyric crisis is an acute dystonic reaction of the ocular muscles characterized by bilateral dystonic elevation of visual gaze lasting from seconds to hours. There have been few published cases on oculogyric crisis in general. This reaction is most commonly explained as an adverse reaction to drugs such as antiemetics, antipsychotics, antidepressants, antiepileptics, and antimalarials. It is important to be able to recognize this ocular side effect because, without a thorough patient history, symptoms can be confused with other diseases (such as versive seizures, encephalopathy). In this paper, we report three cases of oculogyric crisis induced by metoclopramide.

Commentary: Visual Fixation in Human Newborns Correlates with Extensive White Matter Networks and Predicts Long-Term Neurocognitive Development.

Commentary: Visual Fixation in Human Newborns Correlates with Extensive White Matter Networks and Predicts Long-Term Neurocognitive Development.

Looking towards objective quality evaluation in colonoscopy: Analysis of visual gaze patterns.

There are currently limited training and assessment tools available to novice endoscopists. A potential tool for the objective assessment of endoscopist visual search strategy is eye-tracking technology. The aim of this study is to assess whether eye-tracking technology can be used to differentiate the visual gaze patterns (VGP) of experienced and novice endoscopists, and to characterize any differences arising between the two groups. With the use of portable eye-tracking glasses, VGP of novice (n = 20) and experienced (n = 14) endoscopists were compared while viewing a colonoscopy withdrawal through the hepatic flexure. Analysis was performed by comparing the central versus peripheral distribution and the horizontal distribution of novice and experienced endoscopist fixations, along with comparison of basic eye-tracking metrics. This study found that experienced endoscopists had a significantly higher percentage of fixations within the periphery of the screen (13.4% vs 23.0%, P = 0.013). Experienced endoscopists also had a significantly greater percentage of fixations on the left side of the screen (18.6% vs 33.5%, P = 0.005) that displayed the poorly visualized "inside bend" of the hepatic flexure. This study has detailed specific VGP acquired through expertise, which can potentially explain why adenomas are regularly missed at the hepatic flexure during colonoscopy. These may be useful for the training of novice colonoscopists, and further validation may utilize VGP in the development of an objective proficiency based curriculum to improve the detection of pathology and overall quality in endoscopy.

Gaze-estimation for consumer-grade cameras using a Gaussian process latent variable model

Commercial gaze-tracking devices provide accurate measurements of the visual gaze and are applied to a broad range of problems in marketing, human-computer interaction, and health care technology. In some applications commercial systems are either unavailable or unaffordable. Therefore, developing low cost solutions using off the shelf components is worthwhile. In the paper at hand, we apply a hierarchy of Gaussian processes, a class of probabilistic function regressors, to the problem of visual gaze-tracking for consumer grade cameras. Gaussian process latent variable models lead to a lower dimensional manifold which represents the gaze space. Finally, a Gaussian process mapping from screen coordinates to gaze manifold enables us to seek for the users visual gaze point given a previously unseen eye-patch. In our experiments, we achieve mean errors of approximately 2 cm for a consumer grade webcam that is positioned 30-40 cm in front of the user.

Mothers' Attention‐getting Utterances During Shared Book Reading: Links to Low‐income Preschoolers' Verbal Engagement, Visual Attention, and Early Literacy

This study examined associations among low‐income mothers' use of attention‐getting utterances during shared book reading, preschoolers' verbal engagement and visual attention to reading, and their early literacy skills (N = 51). Mother–child shared book reading sessions were videotaped and coded for each utterance, including attention talk, contextualized talk, de‐contextualized talk, and print talk. Preschoolers' attention during book reading was assessed using two measures: verbal engagement by using a rating scale of preschoolers' involvement in book reading discussion and visual attention by coding visual gaze. Findings indicated that mothers' attention talk was positively associated with children's verbal engagement, while maternal print talk was associated with children's visual attention. Further, low‐level maternal print talk was associated with higher early decoding scores of children when it was accompanied with high‐level maternal attention talk. Copyright © 2015 John Wiley &amp; Sons, Ltd.