Changes In Gaze Direction Research Articles

Role-taking in two micro-community sign languages: Phylogenetic insights

This study aimed to examine the use of role-taking constructions in two micro-community sign languages (SLs): Yucatec Maya SL (YMSL), and Soure SL. Various role-taking types were quantified and changes in gaze direction were annotated on data regarding these two SLs. The results showed (i) a greater diversity of role-taking produced in YMSL, including complex role-taking (i.e., multiple perspectives or role-taking produced alongside lexical units), and (ii) changes in gaze direction before and after the production of role-taking constructions in both SLs. First, this suggests a phylogenetic development between the two SLs partly observable from the study of role-taking through the conceptual lens of bifurcation of the signer's intent. More broadly, an analysis of the phylogenesis of SLs would benefit from this kind of examination using analytical concepts relevant to SLs. Second, results seem to indicate that the phylogenesis of SLs would share similarities with the ontogenesis of SLs. Indeed, the less socially integrated and analyzed SL (Soure SL) displayed a less advanced degree of bifurcation since few complex role-taking constructions were observed. This is in line with the way these structures are acquired late by SL learners. Further studies on other micro-community SLs would be needed to validate the results of this study.

Pre-exposure to Ambiguous Faces Modulates Top-Down Control of Attentional Orienting to Counterpredictive Gaze Cues.

Understanding and reacting to others’ nonverbal social signals, such as changes in gaze direction (i.e., gaze cue), are essential for social interactions, as it is important for processes such as joint attention and mentalizing. Although attentional orienting in response to gaze cues has a strong reflexive component, accumulating evidence shows that it can be top-down controlled by context information regarding the signals’ social relevance. For example, when a gazer is believed to be an entity “with a mind” (i.e., mind perception), people exert more top-down control on attention orienting. Although increasing an agent’s physical human-likeness can enhance mind perception, it could have negative consequences on top-down control of social attention when a gazer’s physical appearance is categorically ambiguous (i.e., difficult to categorize as human or nonhuman), as resolving this ambiguity would require using cognitive resources that otherwise could be used to top-down control attention orienting. To examine this question, we used mouse-tracking to explore if categorically ambiguous agents are associated with increased processing costs (Experiment 1), whether categorically ambiguous stimuli negatively impact top-down control of social attention (Experiment 2), and if resolving the conflict related to the agent’s categorical ambiguity (using exposure) would restore top-down control to orient attention (Experiment 3). The findings suggest that categorically ambiguous stimuli are associated with cognitive conflict, which negatively impact the ability to exert top-down control on attentional orienting in a counterpredicitive gaze-cueing paradigm; this negative impact, however, is attenuated when being pre-exposed to the stimuli prior to the gaze-cueing task. Taken together, these findings suggest that manipulating physical human-likeness is a powerful way to affect mind perception in human-robot interaction (HRI) but has a diminishing returns effect on social attention when it is categorically ambiguous due to drainage of cognitive resources and impairment of top-down control.

Does Context Matter? Effects of Robot Appearance and Reliability on Social Attention Differs Based on Lifelikeness of Gaze Task

Social signals, such as changes in gaze direction, are essential cues to predict others’ mental states and behaviors (i.e., mentalizing). Studies show that humans can mentalize with nonhuman agents when they perceive a mind in them (i.e., mind perception). Robots that physically and/or behaviorally resemble humans likely trigger mind perception, which enhances the relevance of social cues and improves social-cognitive performance. The current experiments examine whether the effect of physical and behavioral influencers of mind perception on social-cognitive processing is modulated by the lifelikeness of a social interaction. Participants interacted with robots of varying degrees of physical (humanlike vs. robot-like) and behavioral (reliable vs. random) human-likeness while the lifelikeness of a social attention task was manipulated across five experiments. The first four experiments manipulated lifelikeness via the physical realism of the robot images (Study 1 and 2), the biological plausibility of the social signals (Study 3), and the plausibility of the social context (Study 4). They showed that humanlike behavior affected social attention whereas appearance affected mind perception ratings. However, when the lifelikeness of the interaction was increased by using videos of a human and a robot sending the social cues in a realistic environment (Study 5), social attention mechanisms were affected both by physical appearance and behavioral features, while mind perception ratings were mainly affected by physical appearance. This indicates that in order to understand the effect of physical and behavioral features on social cognition, paradigms should be used that adequately simulate the lifelikeness of social interactions.

Effect of brain stimulation on mechanisms of social cognition is modulated by individual preferences for human versus robot agents

When we interact with others, we use nonverbal behavior such as changes in gaze direction to make inferences about what people think or what they want to do next – a process called mentalizing. Previous studies have shown that how we react to others’ gaze signals depends on how much “mind” we ascribe to the gazer, and that this process of mind perception is related to activation in brain areas that process social information (i.e., social brain). Although brain stimulation studies have identified prefrontal structures like the ventromedial prefrontal cortex (vmPFC) as the potential neural substrate through which mind perception modulates social-cognitive processes like attentional orienting to gaze cues (i.e., gaze following), little is known about whether and how individual differences in preferences for human versus robot agents modulate this relationship. To address this question, the current study examines how transcranial direct current stimulation (tDCS) of left prefrontal versus left temporo-parietal areas affects attentional orienting to gaze signals as a function of the participants’ preferences for human ( Human Gaze Followers, HGF) versus robot ( Robot Gaze Followers; RGF) agents at baseline (prior to brain stimulation). Results show that prefrontal (but not temporo-parietal) stimulation positively affected attentional orienting to gaze signals for HGFs for the human but not the robot gazer; RGFs showed no effect of brain stimulation in neither of the stimulation conditions. These findings inform how preferences for human versus nonhuman agent types can influence subsequent interactions and communications in human-robot interaction.

Automatic evaluation of eye gestural reactions to sound in video sequences

Hearing loss is a common disorder that often intensifies with age. In some cases, especially in the elderly population, the hearing loss may decrease in physical, mental and social well-being capacities. In particular, patients with signs of cognitive impairment typically present specific clinical–pathological conditions, which complicates the analysis and diagnosis of the type and severity of hearing loss by clinical specialists. In these patients, unconscious changes in gaze direction may indicate a certain perception of sound through their auditory system. In this context, this work presents a new system that supports clinical experts in the identification and classification of eye gestures that are associated with reactions to auditory stimuli by patients with different levels of cognitive impairment. The proposed system was validated using the public Video Audiometry Sequence Test (VAST) dataset, providing a global accuracy of 97.12% for the classification of eye gestures and a 100% for gestural reactions to auditory stimuli. The proposed system offers a complete analysis of audiometric video sequences, being applicable in daily clinical practice, improving the well-being and quality of life of the patients.

The effect of prism adaptation on state estimates of eye position in the orbit

Prism adaptation (PA) after-effects are assessed using tests that measure changes in sensorimotor systems. After-effects on pointing without feedback to a visual target (open loop pointing – OLP) are traditionally described as being larger than those measured by straight ahead pointing (SAP) with eyes closed, and the difference between them is attributed to a shift in visual localisation. However, neither differences between OLP and SAP, nor shifts in perceptual judgement of visual straight ahead (VSA), are consistently reported. Moreover, since very few studies have directly recorded direction of gaze, an effect of PA on the state estimate of gaze direction has not been reliably documented. The current research aimed to isolate the effects of PA on state estimates of eye position. We measured sensorimotor after-effects through common (OLP, SAP, and VSA) measures, and also recorded eye position and additional after-effect measures to interrogate changes to the oculomotor system and how these might relate to other measures of sensorimotor change. To ascertain if PA's effects on estimates of eye position could be attributed to eye muscle potentiation, we compared the effects of PA to sustained gaze deviation without adaptation. PA induced no effect on visual straight-ahead and no change in direction of gaze, when measured while positioning a target, looking straight ahead in the dark, or looking toward the passively positioned and occluded unexposed hand. We also found that after-effects measured by SAP with the eyes open were larger than SAP with the eyes closed and equal to those observed with OLP. The findings challenge the concept that total adaptation after-effect is a direct sum of arm proprioceptive and visual after-effects as conventionally measured, and suggest that the oculomotor system is altered by prism adaptation only in interaction with an arm motor command when vision is available.

Effects of embodiment on social attention mechanisms in human-robot interaction

Effects of embodiment on social attention mechanisms in human-robot interaction

Perceived robot personality affects social attention in real-time human-robot interaction

Perceived robot personality affects social attention in real-time human-robot interaction

Mind perception modulates social attention in real-time human-robot interaction

Mind perception modulates social attention in real-time human-robot interaction

Social Saliency of the Cue Slows Attention Shifts.

Eye gaze is a powerful cue that indicates where another person’s attention is directed in the environment. Seeing another person’s eye gaze shift spontaneously and reflexively elicits a shift of one’s own attention to the same region in space. Here, we investigated whether reallocation of attention in the direction of eye gaze is modulated by personal familiarity with faces. On the one hand, the eye gaze of a close friend should be more effective in redirecting our attention as compared to the eye gaze of a stranger. On the other hand, the social relevance of a familiar face might itself hold attention and, thereby, slow lateral shifts of attention. To distinguish between these possibilities, we measured the efficacy of the eye gaze of personally familiar and unfamiliar faces as directional attention cues using adapted versions of the Posner paradigm with saccadic and manual responses. We found that attention shifts were slower when elicited by a perceived change in the eye gaze of a familiar individual as compared to attention shifts elicited by unfamiliar faces at short latencies (100 ms). We also measured simple detection of change in direction of gaze in personally familiar and unfamiliar faces to test whether slower attention shifts were due to slower detection. Participants detected changes in eye gaze faster for familiar faces than for unfamiliar faces. Our results suggest that personally familiar faces briefly hold attention due to their social relevance, thereby slowing shifts of attention, even though the direction of eye movements are detected faster in familiar faces.

Evaluation of Driver Distraction with Changes in Gaze Direction Based on a Vestibulo-Ocular Reflex Model

With the aim of improving parameter identification and, eventually, evaluating driver distraction with changes in gaze direction, we applied a genetic algorithm (GA) method to identify parameters for an existing vestibulo-ocular reflex (VOR) model. By changing the initial inputs to the GA and fixing two parameters pertaining to the horizontal direction, we achieved improved parameter identification with a lower mean-square error. The influence of driver distraction on eye movement with changes in gaze direction was evaluated from the difference between the predicted and observed VOR in the vertical axis. When a driver was given an additional mental workload, the mean-square error between the measured and simulated values was bigger than that in the absence of the mental workload. This confirmed the relationship between driver distraction and eye movement in the vertical direction. We hope that this method can be applied in evaluating driver distraction.

Sex differences for attentional capture by changes in gaze direction

Sex differences for attentional capture by changes in gaze direction

Perception of own and robot engagement in human–robot interactions and their dependence on robotics knowledge

Communication between socially assistive robots and humans might be facilitated by intuitively understandable mechanisms. To investigate the effects of some key nonverbal gestures on a human’s own engagement and robot engagement experienced by humans, participants read a series of instructions to a robot that responded with nods, blinks, changes in gaze direction, or a combination of these. Unbeknown to the participants, the robot had no form of speech processing or gesture recognition, but simply measured speech volume levels, responding with gestures whenever a lull in sound was detected. As measured by visual analogue scales, engagement of participants was not differentially affected by the different responses of the robot. However, their perception of the robot’s engagement in the task, its likability and its understanding of the instructions depended on the gesture presented, with nodding being the most effective response. Participants who self-reported greater robotics knowledge reported higher overall engagement and greater success at developing a relationship with the robot. However, self-reported robotics knowledge did not differentially affect the impact of robot gestures. This suggests that greater familiarity with robotics may help to maximise positive experiences for humans involved in human–robot interactions without affecting the impact of the type of signal sent by the robot.

Influence of eye make-up on the perception of gaze direction

Make-up has a beautifying effect on facial attractiveness. However, little is known about the role of make-up in other than social perception. Does eye make-up applied to a gazing face influence the perception of this gaze by others? Eye make-up might make an individual's gaze more recognizable by emphasizing the contrast of the eyes. Or make-up might make gaze less recognizable by transforming eyes to unnatural shapes. There were two stimulus conditions: eye make-up (no-eyeliner, thin-eyeliner and thick-eyeliner) and gaze direction (same gaze direction and different gaze directions). Participants had to make a recognition response indicating whether the gaze directions were the same or different. Results indicated that in the same gaze-direction condition, thin and thick-eyeliner made recognition of a persisting gaze direction easier, whereas in the different gaze-direction condition thick-eyeliner made it more difficult to recognize the changes in gaze direction. These findings suggested a significant role of make-up in facial processing.

A Cognitive Model of Saliency, Attention, and Picture Scanning

To view and understand the visual world, we shift our gaze from one location to another about three times per second. These rapid changes in gaze direction result from very fast eye movements called saccades. Visual information is acquired only during fixations, stationary periods between saccades. Active visual search of pictures is the process of active scanning of the visual environment for a particular target among distracters or for the extraction of its meaning. This article discusses a cognitive model of saliency, overt attention, and natural picture scanning that unravels the neurocomputational mechanisms of how human gaze control operates during active real-world scene viewing.

Unsupervised learning of head-centered representations in a network of spiking neurons

Movement planning based on visual information requires a transformation from a retina-centered into a heador body-centered frame of reference. It has been shown that such transformations can be achieved via basis function networks [1,2]. We investigated whether basis functions for coordinate transformations can be learned by a biologically plausible neural network. We employed a model network of spiking neurons that learns invariant representations based on spatio-temporal stimulus correlations [3]. The model consists of a three-stage network of leaky integrate-and-fire neurons with biologically realistic conductances. The network has two input layers, corresponding to neurons representing the retinal image and neurons representing the direction of gaze. These inputs are represented in the map layer via excitatory or modulatory connections, respectively, that exhibit Hebbian-like spiketiming dependent plasticity (STDP). Neurons within the map layer are connected via short-range lateral excitatory connections and unspecific lateral inhibition. We trained the network with stimuli corresponding to typical viewing situations when a visual scene is explored by saccadic eye movements, with gaze direction changing on a faster time scale than object positions in space. After learning, each neuron in the map layer was selective for a small subset of the stimulus space, with excitatory and modulatory connections adapted to achieve a topographic map of the inputs. Neurons in the output layer with a localized receptive field in the map layer were selective for positions in head-centered space, invariant to changes in retinal image due to changes in gaze direction. Our results show that coordinate transformations via basis function networks can be learned in a biologically plausible way by exploiting the spatio-temporal correlations between visual stimulation and eye position signals under natural viewing conditions.

Unsupervised learning of gain-field like interactions to achieve head-centered representations

Event Abstract Back to Event Unsupervised learning of gain-field like interactions to achieve head-centered representations Sebastian Thomas-Philipp1*, Frank Michler2 and Thomas Wachtler3 1 Ludwig-Maximilian-University, Computational Neuroscience, Department Biologie II, Germany 2 Philipps-Universitat, Neurophysik, Germany 3 Ludwig-Maximilians University Munich, Fakultat fur Biologie, Germany Movement planning based on visual information requires a transformation from a retina-centered into a head- or body-centered frame of reference. It has been shown that such transformations can be achieved via basis function networks [1,2]. We investigated whether basis functions for coordinate transformations can be learned by a biologically plausible neural network. We employed a model network of spiking neurons that learns invariant representations based on spatio-temporal stimulus correlations [3]. The model consists of a three-stage network of leaky integrate-and-fire neurons with biologically realistic conductances. The network has two input layers, corresponding to neurons representing the retinal image and neurons representing the direction of gaze. These inputs are represented in the map layer via excitatory or modulatory connections, respectively, that exhibit Hebbian-like spike-timing dependent plasticity. Neurons within the map layer are connected via short range lateral excitatory connections and unspecific lateral inhibition. We trained the network with stimuli corresponding to typical viewing situations when a visual scene is explored by saccadic eye movements, with gaze direction changing on a faster time scale than object positions in space. After learning, each neuron in the map layer was selective for a small subset of the stimulus space, with excitatory and modulatory connections adapted to achieve a topographic map of the inputs. Neurons in the output layer with a localized receptive field in the map layer were selective for positions in head-centered space, invariant to changes in retinal image due to changes in gaze direction. Our results show that coordinate transformations via basis function networks can be learned in a biologically plausible way by exploiting the spatio-temporal correlations between visual stimulation and eye position signals under natural viewing conditions. Acknowledgements:Supported by DFG Forschergruppe 560 and BCCN Munich

Depth perception during saccades

A number of studies have investigated the localization of briefly flashed targets during saccades to understand how the brain perceptually compensates for changes in gaze direction. Typical version saccades, i.e., saccades between two points of the horopter, are not only associated with changes in gaze direction, but also with large transient changes of ocular vergence. These transient changes in vergence have to be compensated for just as changes in gaze direction. We investigated depth judgments of perisaccadically flashed stimuli relative to continuously present references and report several novel findings. First, disparity thresholds increased around saccade onset. Second, for horizontal saccades, depth judgments were prone to systematic errors: Stimuli flashed around saccade onset were perceived in a closer depth plane than persistently shown references with the same retinal disparity. Briefly before and after this period, flashed stimuli tended to be perceived in a farther depth plane. Third, depth judgments for upward and downward saccades differed substantially: For upward, but not for downward saccades we observed the same pattern of mislocalization as for horizontal saccades. Finally, unlike localization in the fronto-parallel plane, depth judgments did not critically depend on the presence of visual references. Current models fail to account for the observed pattern of mislocalization in depth.

Driven from distraction: How infants respond to parents’ attempts to elicit and re-direct their attention

This experiment examined how parents’ verbal and non-verbal behavioral cues cause infants to shift and share attention within environments where many objects compete for infants’ attention. Fifteen- and 21-month-old infants played with toys while their parent periodically shifted attention to a distal object within a larger array. Parents’ attention-shifts were indicated by a change in direction of gaze, a pointing gesture, and/or verbalizations. Verbalizations were either attention-eliciting or attention-directing. In some trials parents covered their eyes to occlude line-of-gaze. Both ages seldom followed simple gaze shifts, but frequently followed gaze with-points or gaze-with-directing verbalizations. Attention-eliciting verbalizations increased infants’ looks to the parent. Gaze occlusion reduced infants’ responses to directing verbalizations. Responses to eliciting verbalizations increased with age. Infant receptive vocabulary did not predict attention-sharing, even when parents named objects (i.e., directing verbalizations). Implications for development of attention-sharing, language and understanding of visual attention are discussed.

Regarding Scenes

When we view the visual world, our eyes flit from one location to another about three times each second. These frequent changes in gaze direction result from very fast saccadic eye movements. Useful visual information is acquired only during fixations, periods of relative gaze stability. Gaze control is defined as the process of directing fixation through a scene in real time in the service of ongoing perceptual, cognitive, and behavioral activity. This article discusses current approaches and new empirical findings that are allowing investigators to unravel how human gaze control operates during active real-world scene perception.