Robot Gaze Research Articles

Perceiving social gaze produces the reversed congruency effect.

Numerous studies have shown that the gaze of others produces a special attentional process, such as the eye contact effect or joint attention. This study investigated the attentional process triggered by various types of gaze stimuli (i.e., human, cat, fish, koala, and robot gaze). A total of 300 university students participated in five experiments. They performed a spatial Stroop task in which five types of gaze stimuli were presented as targets. Participants were asked to judge the direction of the target (left or right) irrespective of its location (left or right). The results showed that the social gaze targets (i.e., human and cat gaze) produced a reversed congruency effect. In contrast to the social gaze targets, the non-social gaze (i.e., fish and robot) target did not produce the reversed congruency effect (Experiments 2, 2B, 3, and 4). These results suggest that attention to the gaze of socially communicable beings (i.e., humans and cats) is responsible for the reversed congruency effect. Our findings support the notion that the theory of mind or social interaction plays an important role in producing specific attentional processes in response to gaze stimuli.

Does a Robot's Gaze Behavior Affect Entrainment in HRI?

Does a Robot's Gaze Behavior Affect Entrainment in HRI?

The Methodology of Quantitative Social Intention Evaluation and Robot Gaze Behavior Control in Multi-Objects Scenario

The Methodology of Quantitative Social Intention Evaluation and Robot Gaze Behavior Control in Multi-Objects Scenario

Can the robot "see" what I see? Robot gaze drives attention depending on mental state attribution.

Mentalizing, where humans infer the mental states of others, facilitates understanding and interaction in social situations. Humans also tend to adopt mentalizing strategies when interacting with robotic agents. There is an ongoing debate about how inferred mental states affect gaze following, a key component of joint attention. Although the gaze from a robot induces gaze following, the impact of mental state attribution on robotic gaze following remains unclear. To address this question, we asked forty-nine young adults to perform a gaze cueing task during which mental state attribution was manipulated as follows. Participants sat facing a robot that turned its head to the screen at its left or right. Their task was to respond to targets that appeared either at the screen the robot gazed at or at the other screen. At the baseline, the robot was positioned so that participants would perceive it as being able to see the screens. We expected faster response times to targets at the screen the robot gazed at than targets at the non-gazed screen (i.e., gaze cueing effect). In the experimental condition, the robot's line of sight was occluded by a physical barrier such that participants would perceive it as unable to see the screens. Our results revealed gaze cueing effects in both conditions although the effect was reduced in the occluded condition compared to the baseline. These results add to the expanding fields of social cognition and human-robot interaction by suggesting that mentalizing has an impact on robotic gaze following.

Gaze Point Tracking Based on a Robotic Body-Head-Eye Coordination Method.

When the magnitude of a gaze is too large, human beings change the orientation of their head or body to assist their eyes in tracking targets because saccade alone is insufficient to keep a target at the center region of the retina. To make a robot gaze at targets rapidly and stably (as a human does), it is necessary to design a body-head-eye coordinated motion control strategy. A robot system equipped with eyes and a head is designed in this paper. Gaze point tracking problems are divided into two sub-problems: in situ gaze point tracking and approaching gaze point tracking. In the in situ gaze tracking state, the desired positions of the eye, head and body are calculated on the basis of minimizing resource consumption and maximizing stability. In the approaching gaze point tracking state, the robot is expected to approach the object at a zero angle. In the process of tracking, the three-dimensional (3D) coordinates of the object are obtained by the bionic eye and then converted to the head coordinate system and the mobile robot coordinate system. The desired positions of the head, eyes and body are obtained according to the object's 3D coordinates. Then, using sophisticated motor control methods, the head, eyes and body are controlled to the desired position. This method avoids the complex process of adjusting control parameters and does not require the design of complex control algorithms. Based on this strategy, in situ gaze point tracking and approaching gaze point tracking experiments are performed by the robot. The experimental results show that body-head-eye coordination gaze point tracking based on the 3D coordinates of an object is feasible. This paper provides a new method that differs from the traditional two-dimensional image-based method for robotic body-head-eye gaze point tracking.

Robot Gaze During Autonomous Navigation and Its Effect on Social Presence

As robots have become increasingly common in human-rich environments, it is critical that they are able to exhibit social cues to be perceived as a cooperative and socially-conformant team member. We investigate the effect of robot gaze cues on people’s subjective perceptions of a mobile robot as a socially present entity in three common hallway navigation scenarios. The tested robot gaze behaviors were path-oriented (looking at its own future path), or human-oriented (looking at the nearest person), with fixed-gaze as the control. We conduct a real-world study with 36 participants who walked through the hallway, and an online study with 233 participants who were shown simulated videos of the same scenarios. Our results suggest that the preferred gaze behavior is scenario-dependent. Human-oriented gaze behaviors which acknowledge the presence of the human are generally preferred when the robot and human cross paths. However, this benefit is diminished in scenarios that involve less implicit interaction between the robot and the human.

Does a robot's gaze aversion affect human gaze aversion?

Gaze cues serve an important role in facilitating human conversations and are generally considered to be one of the most important non-verbal cues. Gaze cues are used to manage turn-taking, coordinate joint attention, regulate intimacy, and signal cognitive effort. In particular, it is well established that gaze aversion is used in conversations to avoid prolonged periods of mutual gaze. Given the numerous functions of gaze cues, there has been extensive work on modelling these cues in social robots. Researchers have also tried to identify the impact of robot gaze on human participants. However, the influence of robot gaze behavior on human gaze behavior has been less explored. We conducted a within-subjects user study (N = 33) to verify if a robot's gaze aversion influenced human gaze aversion behavior. Our results show that participants tend to avert their gaze more when the robot keeps staring at them as compared to when the robot exhibits well-timed gaze aversions. We interpret our findings in terms of intimacy regulation: humans try to compensate for the robot's lack of gaze aversion.

Pocketable-Bones: Self-Augment Mobile Robot Mediating our Sociality

Discussions on human-robot interactions abound in the fields of human-agent interaction and human-robot interaction. Although studies on communication through conversations and physical actions have been conducted, many of these studies focus on robot gaze while neglecting human gaze. In this study, we developed Pocketable-Bones, a mobile robot that can move with the human gaze in mind. This robot can fit in a breast pocket and follows and turns in the same direction a person faces. Notably, we have been investigating the realization of joint gazing in which two persons share interests and concerns through gazing. This study’s experiment results showed that Pocketable-Bones’ gazing behaviors satisfied the components of self-determination theory in well-being, notably autonomy, competence, and relationship.

Robotic gaze and human views: A systematic exploration of robotic gaze aversion and its effects on human behaviors and attitudes.

Similar to human-human interaction (HHI), gaze is an important modality in conversational human-robot interaction (HRI) settings. Previously, human-inspired gaze parameters have been used to implement gaze behavior for humanoid robots in conversational settings and improve user experience (UX). Other robotic gaze implementations disregard social aspects of gaze behavior and pursue a technical goal (e.g., face tracking). However, it is unclear how deviating from human-inspired gaze parameters affects the UX. In this study, we use eye-tracking, interaction duration, and self-reported attitudinal measures to study the impact of non-human inspired gaze timings on the UX of the participants in a conversational setting. We show the results for systematically varying the gaze aversion ratio (GAR) of a humanoid robot over a broad parameter range from almost always gazing at the human conversation partner to almost always averting the gaze. The main results reveal that on a behavioral level, a low GAR leads to shorter interaction durations and that human participants change their GAR to mimic the robot. However, they do not copy the robotic gaze behavior strictly. Additionally, in the lowest gaze aversion setting, participants do not gaze back as much as expected, which indicates a user aversion to the robot gaze behavior. However, participants do not report different attitudes toward the robot for different GARs during the interaction. In summary, the urge of humans in conversational settings with a humanoid robot to adapt to the perceived GAR is stronger than the urge of intimacy regulation through gaze aversion, and a high mutual gaze is not always a signof high comfort, as suggested earlier. This result can be used as a justification to deviate from human-inspired gaze parameters when necessary for specific robot behavior implementations.

Effects of robot gaze and voice human-likeness on users’ subjective perception, visual attention, and cerebral activity in voice conversations

Robot gaze and voice are essential anthropomorphic features to promote users' engagement in voice conversations. Earlier research chiefly examined how robot gaze and voice human-likeness separately influenced users' subjective perception. When implementing gaze on robots with different human-like voices, there has little evidence of their possible interaction effects, particularly on users' visual attention and cerebral activity, which could help to understand the perceptual and cognitive processing of anthropomorphic features. Therefore, a within-subject experiment of voice conversations with diverse robot gaze (gaze versus no gaze) and human-like voices (high human-like versus low human-like) using subjective reporting, eye-tracker, and fNIRS was conducted. The results showed that the robot with gaze or a high human-like voice evoked more pleasure, higher arousal, more perceived likability, and less negative attitudes. Robot gaze significantly increased users' average fixation durations and total fixation time, while voice human-likeness prolonged first fixation durations. Moreover, the robot with a high human-like voice (or gaze) induced increased activity in the left DLPFC and decreased activity in the right Broca's area than that had no gaze (or a low human-like voice). The results suggest that robot gaze might chiefly capture users' sustained attention, voice human-likeness might attract users' initial attention, and they might jointly influence users' perceptual processing of prosodic features and emotional processing.

Robotic Gaze Responsiveness in Multiparty Teamwork

Robotic Gaze Responsiveness in Multiparty Teamwork

Human but not robotic gaze facilitates action prediction

SummaryDo people ascribe intentions to humanoid robots as they would to humans or non-human-like animated objects? In six experiments, we compared people’s ability to extract non-mentalistic (i.e., where an agent is looking) and mentalistic (i.e., what an agent is looking at; what an agent is going to do) information from gaze and directional cues performed by humans, human-like robots, and a non-human-like object. People were faster to infer the mental content of human agents compared to robotic agents. Furthermore, although the absence of differences in control conditions rules out the use of non-mentalizing strategies, the human-like appearance of non-human agents may engage mentalizing processes to solve the task. Overall, results suggest that human-like robotic actions may be processed differently from humans’ and objects’ behavior. These findings inform our understanding of the relevance of an object’s physical features in triggering mentalizing abilities and its relevance for human–robot interaction.

Human Preferences for Robot Eye Gaze in Human-to-Robot Handovers.

This paper investigates human’s preferences for a robot’s eye gaze behavior during human-to-robot handovers. We studied gaze patterns for all three phases of the handover process: reach, transfer, and retreat, as opposed to previous work which only focused on the reaching phase. Additionally, we investigated whether the object’s size or fragility or the human’s posture affect the human’s preferences for the robot gaze. A public data-set of human-human handovers was analyzed to obtain the most frequent gaze behaviors that human receivers perform. These were then used to program the robot’s receiver gaze behaviors. In two sets of user studies (video and in-person), a collaborative robot exhibited these gaze behaviors while receiving an object from a human. In the video studies, 72 participants watched and compared videos of handovers between a human actor and a robot demonstrating each of the three gaze behaviors. In the in-person studies, a different set of 72 participants physically performed object handovers with the robot and evaluated their perception of the handovers for the robot’s different gaze behaviors. Results showed that, for both observers and participants in a handover, when the robot exhibited Face-Hand-Face gaze (gazing at the giver’s face and then at the giver’s hand during the reach phase and back at the giver’s face during the retreat phase), participants considered the handover to be more likable, anthropomorphic, and communicative of timing (p < 0.0001). However, we did not find evidence of any effect of the object’s size or fragility or the giver’s posture on the gaze preference.

Development of a listening system by guiding the robot's gaze to the nodding images of spoken words

For the elderly, the act of “speaking” is important to maintain their physical function, and it is known that listening with nods and giving responses are important to encourage utterances. Also, in elderly nursing homes, conversations are often started when they look at images. So far, we proposed a listening system that promotes utterances by recognizing the user’s speech contents by voice input, and presenting images related to the words in speech on the display, and giving nodding reaction from the listener’s InterActor and images, and demonstrated the effectiveness. In this study, we have developed a listening system that guides gaze to the image presented by utterance using a robot that is familiar as a dialogue agent. In this development, we have developed two methods, one is to guide the user to gaze at the image while communicating face-to-face with the robot to the user, and the other is to guide the robot to gaze at the screen by the robot taking a position diagonally to the right of the user.

From Social Gaze to Indirect Speech Constructions: How to Induce the Impression That Your Companion Robot Is a Conscious Creature

We implemented different modes of social gaze behavior in our companion robot, F-2, to evaluate the impression of the gaze behaviors on humans in three symmetric communicative situations: (a) the robot telling a story, (b) the person telling a story to the robot, and (c) both parties communicating about objects in the real world while solving a Tangram puzzle. In all the situations the robot localized the human’s eyes and directed its gaze between the human, the environment, and the object of interest in the problem space (if it existed). We examined the balance between different gaze directions as the novel key element to maintaining a feeling of social connection with the robot in humans. We extended the computer model of the robot in order to simulate realistic gaze behavior in the robot and create the impression of the robot changing its internal cognitive states. Other novel results include the implicit, rather than explicit, character of the robot gaze perception for many of our subjects and the role of individual differences, especially the level of emotional intelligence, in terms of human sensitivity to the robotic gaze. Therefore, in this study, we used an iterative approach, extending the applied cognitive architecture in order to simulate the balance between different behavioral reactions and to test it in the experiments. In such a way, we came to a description of the key behavioral cues that suggest to a person that the particular robot can be perceived as an emotional and even conscious creature.

Mutual gaze with a robot affects human neural activity and delays decision-making processes.

In most everyday life situations, the brain needs to engage not only in making decisions but also in anticipating and predicting the behavior of others. In such contexts, gaze can be highly informative about others’ intentions, goals, and upcoming decisions. Here, we investigated whether a humanoid robot’s gaze (mutual or averted) influences the way people strategically reason in a social decision-making context. Specifically, participants played a strategic game with the robot iCub while we measured their behavior and neural activity by means of electroencephalography (EEG). Participants were slower to respond when iCub established mutual gaze before their decision, relative to averted gaze. This was associated with a higher decision threshold in the drift diffusion model and accompanied by more synchronized EEG alpha activity. In addition, we found that participants reasoned about the robot’s actions in both conditions. However, those who mostly experienced the averted gaze were more likely to adopt a self-oriented strategy, and their neural activity showed higher sensitivity to outcomes. Together, these findings suggest that robot gaze acts as a strong social signal for humans, modulating response times, decision threshold, neural synchronization, as well as choice strategies and sensitivity to outcomes. This has strong implications for all contexts involving human-robot interaction, from robotics to clinical applications.

Cognitive Impact of Anthropomorphized Robot Gaze

Attentional control does not have fix functioning and can be strongly impacted by the presence of other human beings or humanoid robots. In two studies, this phenomenon was investigated while focusing exclusively on robot gaze as a potential determinant of attentional control along with the role of participants’ anthropomorphic inferences toward the robot. In study 1, we expected and found higher interference in trials including a direct robot gaze compared to an averted gaze on a task measuring attentional control (Eriksen flanker task). Participants’ anthropomorphic inferences about the social robot mediated this interference. In study 2, we found that averted gazes congruent with the correct answer (same task as study 1) facilitated performance. Again, this effect was mediated by anthropomorphic inferences. These two studies show the importance of anthropomorphic robotic gaze on human cognitive processing, especially attentional control, and also open new avenues of research in social robotics.

Optimization and improvement of a robotics gaze control system using LSTM networks

Gaze control represents an important issue in the interaction between a robot and humans. Specifically, deciding who to pay attention to in a multi-party conversation is one way to improve the naturalness of a robot in human-robot interaction. This control can be carried out by means of two different models that receive the stimuli produced by the participants in an interaction, either an on-center off-surround competitive network or a recurrent neural network. A system based on a competitive neural network is able to decide who to look at with a smooth transition in the focus of attention when significant changes in stimuli occur. An important aspect in this process is the configuration of the different parameters of such neural network. The weights of the different stimuli have to be computed to achieve human-like behavior. This article explains how these weights can be obtained by solving an optimization problem. In addition, a new model using a recurrent neural network with LSTM layers is presented. This model uses the same set of stimuli but does not require its weighting. This new model is easier to train, avoiding manual configurations, and offers promising results in robot gaze control. The experiments carried out and some results are also presented.

Robot Gaze Behavior Affects Honesty in Human-Robot Interaction.

As the use of humanoid robots proliferates, an increasing amount of people may find themselves face-to-“face” with a robot in everyday life. Although there is a plethora of information available on facial social cues and how we interpret them in the field of human-human social interaction, we cannot assume that these findings flawlessly transfer to human-robot interaction. Therefore, more research on facial cues in human-robot interaction is required. This study investigated deception in human-robot interaction context, focusing on the effect that eye contact with a robot has on honesty toward this robot. In an iterative task, participants could assist a humanoid robot by providing it with correct information, or potentially secure a reward for themselves by providing it with incorrect information. Results show that participants are increasingly honest after the robot establishes eye contact with them, but only if this is in response to deceptive behavior. Behavior is not influenced by the establishment of eye contact if the participant is actively engaging in honest behavior. These findings support the notion that humanoid robots can be perceived as, and treated like, social agents, since the herein described effect mirrors one present in human-human social interaction.

Repetitive Robot Behavior Impacts Perception of Intentionality and Gaze-Related Attentional Orienting.

Gaze behavior is an important social signal between humans as it communicates locations of interest. People typically orient their attention to where others look as this informs about others' intentions and future actions. Studies have shown that humans can engage in similar gaze behavior with robots but presumably more so when they adopt the intentional stance toward them (i.e., believing robot behaviors are intentional). In laboratory settings, the phenomenon of attending toward the direction of others' gaze has been examined with the use of the gaze-cueing paradigm. While the gaze-cueing paradigm has been successful in investigating the relationship between adopting the intentional stance toward robots and attention orienting to gaze cues, it is unclear if the repetitiveness of the gaze-cueing paradigm influences adopting the intentional stance. Here, we examined if the duration of exposure to repetitive robot gaze behavior in a gaze-cueing task has a negative impact on subjective attribution of intentionality. Participants performed a short, medium, or long face-to-face gaze-cueing paradigm with an embodied robot while subjective ratings were collected pre and post the interaction. Results show that participants in the long exposure condition had the smallest change in their intention attribution scores, if any, while those in the short exposure condition had a positive change in their intention attribution, indicating that participants attributed more intention to the robot after short interactions. The results also show that attention orienting to robot gaze-cues was positively related to how much intention was attributed to the robot, but this relationship became more negative as the length of exposure increased. In contrast to subjective ratings, the gaze-cueing effects (GCEs) increased as a function of the duration of exposure to repetitive behavior. The data suggest a tradeoff between the desired number of trials needed for observing various mechanisms of social cognition, such as GCEs, and the likelihood of adopting the intentional stance toward a robot.