Agent's Gaze Research Articles

Gaze communicates both cue direction and agent mental states.

Although it is well established that humans spontaneously attend to where others are looking, it remains debated whether this gaze following behavior occurs because gaze communicates directional information (i.e., where an agent is looking) or because gaze communicates an agent's inferred mental content (i.e., what an agent perceives), both of which rely on the processes involved in the general Theory of Mind ability. To address this question, in two Experiments we used a novel task to measure how spatially dissociated and spatially combined effects of an agent's gaze direction and perceived mental content influence target performance. We also contrasted performance for social directional cues and nonsocial arrows. Our data revealed that performance was compromised when cue direction and mental content dissociated relative to when they combined. Performance for dissociated components was especially prominent when a social avatar served as a cue relative to a comparison arrow. Together, these data show that a typical gaze signal communicates information about both where an agent is attending and what they are attending to.

Inferring Interactivity From Gaze Patterns During Triadic Person-Object-Agent Interactions.

Observing others’ gaze informs us about relevant matters in the environment. Humans’ sensitivity to gaze cues and our ability to use this information to focus our own attention is crucial to learning, social coordination, and survival. Gaze can also be a deliberate social signal which captures and directs the gaze of others toward an object of interest. In the current study, we investigated whether the intention to actively communicate one’s own attentional focus can be inferred from the dynamics of gaze alone. We used a triadic gaze interaction paradigm based on the recently proposed classification of attentional states and respective gaze patterns in person-object-person interactions, the so-called “social gaze space (SGS).” Twenty-eight participants interacted with a computer controlled virtual agent while they assumed to interact with a real human. During the experiment, the virtual agent engaged in various gaze patterns which were determined by the agent’s attentional communicative state, as described by the concept of SGS. After each interaction, participants were asked to judge whether the other person was trying to deliberately interact with them. Results show that participants were able to infer the communicative intention solely from the agent’s gaze behavior. The results substantiate claims about the pivotal role of gaze in social coordination and relationship formation. Our results further reveal that social expectations are reflected in differential responses to the displayed gaze patterns and may be crucial for impression formation during gaze-based interaction. To the best of our knowledge, this is the first study to document the experience of interactivity in continuous and contingent triadic gaze interactions.

Power of looking together: an analysis of social facilitation by Agent's mutual gaze

The authors investigated the effect of gaze tracking of humans by an agent. Psychological studies have shown that the presence of others facilitates task with implying partnership. This effect also occurs if the presence of others is replaced with a robot. Furthermore, the previous research showed that being touched by a robot motivates a person. However, direct contact such as the touch of a robot has a risk of disturbing human work. The authors used gaze tracking for social facilitation because gaze tracking can also facilitate human efforts and implemented a social facilitation robot agent by using gaze tracking. The agent projects its face on a surface to show quickly changing expressions, which is difficult to achieve with ordinary robots. The agent can also express gaze tracking by implementing eye trackers. The authors prepared three conditions with monotonous tasks to compare the effects of gaze tracking: the agent gazes at the instructed point moves its gaze randomly and traces the gaze point of a participant. The results showed that the participants’ motivation increased if the agent is tracking gazed point by a human, even though there are no differences between the achievement scores of the task and the continuation time.

In natural interaction with embodied robots, we prefer it when they follow our gaze: a gaze-contingent mobile eyetracking study.

Initiating joint attention by leading someone's gaze is a rewarding experience which facilitates social interaction. Here, we investigate this experience of leading an agent's gaze while applying a more realistic paradigm than traditional screen-based experiments. We used an embodied robot as our main stimulus and recorded participants' eye movements. Participants sat opposite a robot that had either of two ‘identities’—‘Jimmy’ or ‘Dylan’. Participants were asked to look at either of two objects presented on screens to the left and the right of the robot. Jimmy then looked at the same object in 80% of the trials and at the other object in the remaining 20%. For Dylan, this proportion was reversed. Upon fixating on the object of choice, participants were asked to look back at the robot's face. We found that return-to-face saccades were conducted earlier towards Jimmy when he followed the gaze compared with when he did not. For Dylan, there was no such effect. Additional measures indicated that our participants also preferred Jimmy and liked him better. This study demonstrates (a) the potential of technological advances to examine joint attention where ecological validity meets experimental control, and (b) that social reorienting is enhanced when we initiate joint attention.This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’.

Spontaneous Facial Mimicry is Modulated by Joint Attention and Autistic Traits.

Joint attention (JA) and spontaneous facial mimicry (SFM) are fundamental processes in social interactions, and they are closely related to empathic abilities. When tested independently, both of these processes have been usually observed to be atypical in individuals with autism spectrum conditions (ASC). However, it is not known how these processes interact with each other in relation to autistic traits. This study addresses this question by testing the impact of JA on SFM of happy faces using a truly interactive paradigm. Sixty‐two neurotypical participants engaged in gaze‐based social interaction with an anthropomorphic, gaze‐contingent virtual agent. The agent either established JA by initiating eye contact or looked away, before looking at an object and expressing happiness or disgust. Eye tracking was used to make the agent's gaze behavior and facial actions contingent to the participants' gaze. SFM of happy expressions was measured by Electromyography (EMG) recording over the Zygomaticus Major muscle. Results showed that JA augments SFM in individuals with low compared with high autistic traits. These findings are in line with reports of reduced impact of JA on action imitation in individuals with ASC. Moreover, they suggest that investigating atypical interactions between empathic processes, instead of testing these processes individually, might be crucial to understanding the nature of social deficits in autism. Autism Res 2016, 9: 781–789. © 2015 The Authors Autism Research published by Wiley Periodicals, Inc. on behalf of International Society for Autism Research

Why we interact: On the functional role of the striatum in the subjective experience of social interaction

There is ample evidence that human primates strive for social contact and experience interactions with conspecifics as intrinsically rewarding. Focusing on gaze behavior as a crucial means of human interaction, this study employed a unique combination of neuroimaging, eye-tracking, and computer-animated virtual agents to assess the neural mechanisms underlying this component of behavior. In the interaction task, participants believed that during each interaction the agent's gaze behavior could either be controlled by another participant or by a computer program. Their task was to indicate whether they experienced a given interaction as an interaction with another human participant or the computer program based on the agent's reaction. Unbeknownst to them, the agent was always controlled by a computer to enable a systematic manipulation of gaze reactions by varying the degree to which the agent engaged in joint attention. This allowed creating a tool to distinguish neural activity underlying the subjective experience of being engaged in social and non-social interaction. In contrast to previous research, this allows measuring neural activity while participants experience active engagement in real-time social interactions. Results demonstrate that gaze-based interactions with a perceived human partner are associated with activity in the ventral striatum, a core component of reward-related neurocircuitry. In contrast, interactions with a computer-driven agent activate attention networks. Comparisons of neural activity during interaction with behaviorally naïve and explicitly cooperative partners demonstrate different temporal dynamics of the reward system and indicate that the mere experience of engagement in social interaction is sufficient to recruit this system.

An investigation of social factors related to online mentalizing in a human‐robot competitive game

Abstract“Mentalizing” is the ability to attribute mental states to other agents. The lack of online mentalizing, which is required in actual social contexts, may cause serious social disorders such as autism. However, the mechanism of online mentalizing is still unclear. In this study, we found that behavioral entropy (which indicates the randomness of decision making) was an efficient behavioral index for online mentalizing in a human‐human competitive game. Further participants played the game with a humanoid robot; the results indicated that the entropy was significantly higher in participants whose gaze followed the robot's head turn than in those who did not, although the explicit human‐likeness of the robot did not correlate with behavioral entropy. These results implied that mentalizing could be divided into two separate processes: an explicit, logical reasoning process and an implicit, intuitive process driven by perception of the other agent's gaze. We hypothesize that the latter is a core process for online mentalizing, and we argue that the social problems of autistic people are caused by dysfunction of this process.

Can Infants Use a Nonhuman Agent's Gaze Direction to Establish Word-Object Relations?

Adopting a procedure developed with human speakers, we examined infants' ability to follow a nonhuman agent's gaze direction and subsequently to use its gaze to learn new words. When a programmable robot acted as the speaker (Experiment 1), infants followed its gaze toward the word referent whether or not it coincided with their own focus of attention, but failed to learn a new word. When the speaker was human, infants correctly mapped the words (Experiment 2). Furthermore, when the robot interacted contingently, this did not facilitate infants' word mapping (Experiment 3). These findings suggest that gaze following upon hearing a novel word is not sufficient to learn the referent of the word when the speaker is nonhuman.

Genealogy of the Australian Homocriminal Subject

By conducting close readings of a wide range of texts where knowledge of homosexual deviance was constructed and deployed, this article seeks to chart how two competing models evolved to account for the emergence of the ‘homocriminal’ subject in Australia. In the first model, that of importation, ‘homocriminality’ was imagined as emanating from abroad — an exotic perversion imported from England. In contrast, a second model posited that ‘homocriminality’ was simultaneously present within Australia. Exploring these models offers a valuable insight into the ways that homosexuality was imagined as deviant, threatening and offensive prior to the advent of the decriminalisation era in Australia. Furthermore, the article highlights how the gaze of agents of the law was implicated in constructing and representing homosexual desire as disgusting and unruly, warranting ongoing surveillance and regulation by the law.