Robot Gaze Research Articles

Interactions between a quiz robot and multiple participants

This paper reports on a quiz robot experiment in which we explore similarities and differences in human participant speech, gaze, and bodily conduct in responding to a robot’s speech, gaze, and bodily conduct across two languages. Our experiment involved three-person groups of Japanese and English-speaking participants who stood facing the robot and a projection screen that displayed pictures related to the robot’s questions. The robot was programmed so that its speech was coordinated with its gaze, body position, and gestures in relation to transition relevance places (TRPs), key words, and deictic words and expressions (e.g. this, this picture) in both languages. Contrary to findings on human interaction, we found that the frequency of English speakers’ head nodding was higher than that of Japanese speakers in human-robot interaction (HRI). Our findings suggest that the coordination of the robot’s verbal and non-verbal actions surrounding TRPs, key words, and deictic words and expressions is important for facilitating HRI irrespective of participants’ native language. Keywords: coordination of verbal and non-verbal actions; robot gaze comparison between English and Japanese; human-robot interaction (HRI); transition relevance place (TRP); conversation analysis

Effects of Robot Gaze and Proxemic Behavior on Perceived Social Presence during a Hallway Navigation Scenario

Robots are increasingly being introduced into task environments that require the ability to exhibit appropriate social functionality. The present study is an examination of how social cues conveyed by a robot, during a brief interaction, affect the perception of the robot as a socially present agent. Participants were exposed to one of three gaze conditions and two proxemic behavioral programs during a number of experimental trials involving path-crossing in a hallway setting. Results indicated that participants perceived the robot as more socially present when it exhibited a passive proxemic behavior and more socially present over time; though, findings varied at the sub-scale level. Design recommendations are presented for roboticists.

Making eyes with robots

One of the most important aims of social robotics is to improve Human-Robot Interaction by providing robots with means to understand observed behavior and to predict upcoming actions of their in-teraction partners. The most reliable source for inferring the action goals of interaction partners is their gaze direction. Hence, to anticipate upcoming actions, it is necessary to identify where others are currently look-ing at and to shift the attentional focus to the same location. Interestingly, it has been shown that observing robot gaze direction also induces attentional shifts to the location that is gazed-at by the robot. Given this, gaze direction can be actively used by the robot to direct the attentional focus of interaction partners to im-portant events in the world. In this paper, we review findings from two studies indicating that the readiness to engage attentional resources in interactions with robots is modulated by the degree to which intentionali-ty can be attributed to the robot: Robots believed to behave similar to humans cause stronger gaze-cueing effects than robots perceived as machines, independently of their physical appearance. Based on these find-ings, and on results from a pilot study with a sample of patients diagnosed with autism spectrum disorder (ASD), we derive guidelines for improving human-robot interaction by emulating social gaze behavior.

The Repertoire of Robot Behavior: Designing Social Behaviors to Support Human-Robot Joint Activity

In social interaction, people draw on a large repertoire of social acts tailoring their use of these acts to meet the demands of the social situation and to achieve the goals of the interaction. This paper presents an approach to creating such a repertoire of social acts for robots and enabling designers to specify the social situation to which robots may adapt their behaviors. Drawing on principles of Activity Theory and social-scientific findings on human social behavior, this paper introduces an implementation of this approach---the Robot Behavior Toolkit---and two studies that use a limited, proof-of-concept repertoire of specifications for gaze cues to demonstrate the feasibility of this approach for controlling robot gaze behavior. The first study assessed the feasibility of the use of this repertoire, comparing it to alternative, baseline repertoires in two human-robot interaction tasks, and found that it enabled the robot to more effectively support the interaction goals. The second study investigated the feasibility of the robot adapting its use of the repertoire to a social situation by comparing different goal specifications in two human-robot interaction tasks. The results showed that these specifications enabled the robot to achieve some of its task and communicative goals, although participant gender strongly affected whether the robot elicited these interaction outcomes.

Infants understand the referential nature of human gaze but not robot gaze

Infants can acquire much information by following the gaze direction of others. This type of social learning is underpinned by the ability to understand the relationship between gaze direction and a referent object (i.e., the referential nature of gaze). However, it is unknown whether human gaze is a privileged cue for information that infants use. Comparing human gaze with nonhuman (robot) gaze, we investigated whether infants’ understanding of the referential nature of looking is restricted to human gaze. In the current study, we developed a novel task that measured by eye-tracking infants’ anticipation of an object from observing an agent’s gaze shift. Results revealed that although 10- and 12-month-olds followed the gaze direction of both a human and a robot, only 12-month-olds predicted the appearance of objects from referential gaze information when the agent was the human. Such a prediction for objects reflects an understanding of referential gaze. Our study demonstrates that by 12months of age, infants hold referential expectations specifically from the gaze shift of humans. These specific expectations from human gaze may enable infants to acquire various information that others convey in social learning and social interaction.

A Teleoperation Approach for Mobile Social Robots Incorporating Automatic Gaze Control and Three-Dimensional Spatial Visualization

The teleoperation of mobile social robots requires operators to understand facial gestures and other nonverbal communication from a person interacting with the robot. It is also critical for the operator to comprehend the surrounding environment in order to facilitate both navigation and human-robot interaction. Allowing the operator to control the robot's gaze direction can help the operator observe a person's nonverbal communication; however, manually actuating a gaze increases the operator's workload and conflicts with the use of the robot's camera for navigation. To address these problems, the authors developed a teleoperation system that combines automatic control of the robot's gaze and a 3-D graphical representation of the surrounding environment, such as location of items and configuration of a shop. A study where a robot plays the role of a shopkeeper was conducted to validate the effectiveness of the proposed gaze-control technique and control interface. It was demonstrated that the combination of automatic gaze control and representations of spatial relationships improved the quality of the robot's interaction with the customer.

Toward understanding social cues and signals in human–robot interaction: effects of robot gaze and proxemic behavior

As robots are increasingly deployed in settings requiring social interaction, research is needed to examine the social signals perceived by humans when robots display certain social cues. In this paper, we report a study designed to examine how humans interpret social cues exhibited by robots. We first provide a brief overview of perspectives from social cognition in humans and how these processes are applicable to human–robot interaction (HRI). We then discuss the need to examine the relationship between social cues and signals as a function of the degree to which a robot is perceived as a socially present agent. We describe an experiment in which social cues were manipulated on an iRobot AvaTM mobile robotics platform in a hallway navigation scenario. Cues associated with the robot’s proxemic behavior were found to significantly affect participant perceptions of the robot’s social presence and emotional state while cues associated with the robot’s gaze behavior were not found to be significant. Further, regardless of the proxemic behavior, participants attributed more social presence and emotional states to the robot over repeated interactions than when they first interacted with it. Generally, these results indicate the importance for HRI research to consider how social cues expressed by a robot can differentially affect perceptions of the robot’s mental states and intentions. The discussion focuses on implications for the design of robotic systems and future directions for research on the relationship between social cues and signals.

I Reach Faster When I See You Look: Gaze Effects in Human-Human and Human-Robot Face-to-Face Cooperation.

Human–human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human–human cooperation experiment demonstrating that an agent’s vision of her/his partner’s gaze can significantly improve that agent’s performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human–robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human–robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.

Can infants predict objects from human and robot's gaze cue?

Can infants predict objects from human and robot's gaze cue?

Mutual Gaze: Implications for Human-Robot Interaction

Mutual Gaze: Implications for Human-Robot Interaction

Robot Head Pose Detection and Gaze Direction Determination Using Local Invariant Features

Gaze direction determination can be a powerful anticipatory perceptual mechanism for determining the next action of other individuals, humans or robots. It can allow cooperation, synchronization or competition between robots. This is of particular importance in the case of anthropomorphic robots, which in addition of having a human-like body, should behave as humans and have similar attention mechanisms for tracking and gazing other individuals and objects. We address this problem by proposing a gaze direction determination system for robots. This system is based primarily on a robot head pose detection system that consists of two processing stages: computation of scale-invariant local descriptors of the scene and matching of these descriptors against descriptors of robot head prototypes already stored in a database. These prototypes correspond to images of robot heads taken under different view angles. After the robot head pose is detected, the robot gaze direction is determined by a composed coordinate transformation that considers the three-dimensional pose of the observing robot's camera, the detected robot head pose with respect to the observing camera, and the head model of the observed robot. Results of the successful application of the proposed system in real robots are presented.

Bottom-Up Gaze Shifts and Fixations Learning by Imitation

The ability to follow the gaze of conspecifics is a critical component in the development of social behaviors, and many efforts have been directed to studying the earliest age at which it begins to develop in infants. Developmental and neurophysiological studies suggest that imitative learning takes place once gaze-following abilities are fully established and joint attention can support the shared behavior required by imitation. Accordingly, gaze-following acquisition should be precursory to most machine learning tasks, and imitation learning can be seen as the earliest modality for acquiring meaningful gaze shifts and for understanding the structural substrate of fixations. Indeed, if some early attentional process, based on a suitable combination of gaze shifts and fixations, could be learned by the robot, then several demonstration learning tasks would be dramatically simplified. In this paper, we describe a methodology for learning gaze shifts based on imitation of gaze following with a gaze machine, which we purposefully introduced to make the robot gaze imitation conspicuous. The machine allows the robot to share and imitate gaze shifts and fixations of a caregiver through a mutual vergence. This process is then suitably generalized by learning both the scene salient features toward which the gaze is directed and the way saccadic programming is attained. Salient features are modeled by a family of Gaussian mixtures. These together with learned transitions are generalized via hidden Markov models to account for humanlike gaze shifts allowing to discriminate salient locations.

Acquisition of joint attention through natural interaction utilizing motion cues

Joint attention is one of the most important cognitive functions for the emergence of communication not only between humans, but also between humans and robots. In previous work, we have demonstrated how a robot can acquire primary joint attention behavior (gaze following) without external evaluation. However, this method needs the human to tell the robot when to shift its gaze. This paper presents a method that does not need such a constraint by introducing an attention selector based on a measure consisting of saliencies of object features and motion cues. In order to realize natural interaction, a self-organizing map for real-time face pattern separation and contingency learning for gaze following without external evaluation are utilized. The attention selector controls the robot gaze to switch often from the human face to an object and vice versa, and pairs of a face pattern and a gaze motor command are input to the contingency learning. The motion cues are expected to reduce the number of incorrect training data pairs due to the asynchronous interaction that affects the convergence of the contingency learning. The experimental result shows that gaze shift utilizing motion cues enables a robot to synchronize its own motion with human motion and to learn joint attention efficiently in about 20 min.

Robot gaze stabilization based on mimesis of oculomotor dynamics and vestibulocerebellar learning

Robot gaze stabilization based on mimesis of oculomotor dynamics and vestibulocerebellar learning

Robot gaze stabilization based on mimesis of oculomotor dynamics and vestibulocerebellar learning

(1998). Robot gaze stabilization based on mimesis of oculomotor dynamics and vestibulocerebellar learning. Advanced Robotics: Vol. 13, No. 3, pp. 351-352.