Child-robot Interaction Research Articles

Social engagement of children with autism spectrum disorder (ASD) in imitating a humanoid robot: a case study

Our study aims to compare the social engagement behaviors of the children with autism spectrum disorder (ASD) in interaction with a kid-size humanoid robot and a human partner in a motor imitation task on two children with ASD with high functioning. The evaluation measures (e.g., eye gaze direction, gaze shifting, completeness/correctness of the movements, initiation with/without prompt, verbal instruction, and time differences between starting/finishing the robot’s movement and the child’s movement) are extracted from the recorded videos of the trial sessions and analyzed to assess the engagement of the children in interaction with the robot or a human partner. The results indicate that the eye gaze duration to the robot, the frequency of the initiation without prompt, and the frequency of the complete and correct movements for Child 1 in the robot condition are higher compared to the human condition, while these criteria for Child 2 in the robot condition are slightly less compared to the human condition. Also, the frequency of the gaze shifting for Child 1 in the robot condition is lower compared to the human condition, while for Child 2, it is slightly higher compared to the human condition. Moreover, both children show desirable results in the case of the frequency of the required verbal instructions, and finished the movements before the robot and started after the robot most time. Finally, the study can suggest child–robot interaction to improve the social engagement of some children with ASD.

Toward an Automated Measure of Social Engagement for Children With Autism Spectrum Disorder-A Personalized Computational Modeling Approach.

Social engagement is a key indicator of an individual's socio-emotional and cognitive states. For a child with Autism Spectrum Disorder (ASD), this serves as an important factor in assessing the quality of the interactions and interventions. So far, qualitative measures of social engagement have been used extensively in research and in practice, but a reliable, objective, and quantitative measure is yet to be widely accepted and utilized. In this paper, we present our work on the development of a framework for the automated measurement of social engagement in children with ASD that can be utilized in real-world settings for the long-term clinical monitoring of a child's social behaviors as well as for the evaluation of the intervention methods being used. We present a computational modeling approach to derive the social engagement metric based on a user study with children between the ages of 4 and 12 years. The study was conducted within a child-robot interaction setting that targets sensory processing skills in children. We collected video, audio and motion-tracking data from the subjects and used them to generate personalized models of social engagement by training a multi-channel and multi-layer convolutional neural network. We then evaluated the performance of this network by comparing it with traditional classifiers and assessed its limitations, followed by discussions on the next steps toward finding a comprehensive and accurate metric for social engagement in ASD.

Trouble and Repair in Child-Robot Interaction: A Study of Complex Interactions With a Robot Tutee in a Primary School Classroom.

Today, robots are studied and expected to be used in a range of social roles within classrooms. Yet, due to a number of limitations in social robots, robot interactions should be expected to occasionally suffer from troublesome situations and breakdowns. In this paper, we explore this issue by studying how children handle interaction trouble with a robot tutee in a classroom setting. The findings have implications not only for the design of robots, but also for evaluating their benefit in, and for, educational contexts. In this study, we conducted video analysis of children's group interactions with a robot tutee in a classroom setting, in order to explore the nature of these troubles in the wild. Within each group, children took turns acting as the primary interaction partner for the robot within the context of a mathematics game. Specifically, we examined what types of situations constitute trouble in these child–robot interactions, the strategies that individual children employ to cope with this trouble, as well as the strategies employed by other actors witnessing the trouble. By means of Interaction Analysis, we studied the video recordings of nine group interaction sessions (n = 33 children) in primary school grades 2 and 4. We found that sources of trouble related to the robot's social norm violations, which could be either active or passive. In terms of strategies, the children either persisted in their attempts at interacting with the robot by adapting their behavior in different ways, distanced themselves from the robot, or sought the help of present adults (i.e., a researcher in a teacher role, or an experimenter) or their peers (i.e., the child's classmates in each group). In terms of the witnessing actors, they addressed the trouble by providing guidance directed at the child interacting with the robot, or by intervening in the interaction. These findings reveal the unspoken rules by which children orient toward social robots, the complexities of child–robot interaction in the wild, and provide insights on children's perspectives and expectations of social robots in classroom contexts.

Shall I Trust You? From Child-Robot Interaction to Trusting Relationships.

Studying trust in the context of human–robot interaction is of great importance given the increasing relevance and presence of robotic agents in the social sphere, including educational and clinical. We investigated the acquisition, loss, and restoration of trust when preschool and school-age children played with either a human or a humanoid robot in vivo. The relationship between trust and the representation of the quality of attachment relationships, Theory of Mind, and executive function skills was also investigated. Additionally, to outline children’s beliefs about the mental competencies of the robot, we further evaluated the attribution of mental states to the interactive agent. In general, no substantial differences were found in children’s trust in the play partner as a function of agency (human or robot). Nevertheless, 3-year-olds showed a trend toward trusting the human more than the robot, as opposed to 7-year-olds, who displayed the reverse pattern. These findings align with results showing that, for 3- and 7-year-olds, the cognitive ability to switch was significantly associated with trust restoration in the human and the robot, respectively. Additionally, supporting previous findings, we found a dichotomy between attributions of mental states to the human and robot and children’s behavior: while attributing to the robot significantly lower mental states than the human, in the Trusting Game, children behaved in a similar way when they related to the human and the robot. Altogether, the results of this study highlight that similar psychological mechanisms are at play when children are to establish a novel trustful relationship with a human and robot partner. Furthermore, the findings shed light on the interplay – during development – between children’s quality of attachment relationships and the development of a Theory of Mind, which act differently on trust dynamics as a function of the children’s age as well as the interactive partner’s nature (human vs. robot).

Intentional acceptance of social robots: Development and validation of a self-report measure for children

Abstract As the field of child-robot interaction (CRI) is still young, little consensus exists on the conceptual and methodological approach to the study of children's intentional acceptance of social robots, an important antecedent and an outcome of CRI. Against this background, the goal of this study was to develop and validate a self-report measure of intentional social robot acceptance (i.e., intention to use) for children. Partly based on measures for adults, we developed a four-item scale for children's intentional acceptance of social robots. We administered the measure along with the validation measures (i.e., enjoyment, social presence, and social anxiety) among 87 Dutch children aged 7 to 11 years. Our measure reliably and validly assessed children's intentional acceptance of social robots. As children are increasingly likely to encounter social robots in their lives, our measure may help to improve the study of children's acceptance of social robots and its pertinent processes.

Young Children's Indiscriminate Helping Behavior Toward a Humanoid Robot.

Young children help others in a range of situations, relatively indiscriminate of the characteristics of those they help. Recent results have suggested that young children’s helping behavior extends even to humanoid robots. However, it has been unclear how characteristics of robots would influence children’s helping behavior. Considering previous findings suggesting that certain robot features influence adults’ perception of and their behavior toward robots, the question arises of whether young children’s behavior and perception would follow the same principles. The current study investigated whether two key characteristics of a humanoid robot (animate autonomy and friendly expressiveness) would affect children’s instrumental helping behavior and their perception of the robot as an animate being. Eighty-two 3-year-old children participated in one of four experimental conditions manipulating a robot’s ostensible animate autonomy (high/low) and friendly expressiveness (friendly/neutral). Helping was assessed in an out-of-reach task and animacy ratings were assessed in a post-test interview. Results suggested that both children’s helping behavior, as well as their perception of the robot as animate, were unaffected by the robot’s characteristics. The findings indicate that young children’s helping behavior extends largely indiscriminately across two important characteristics. These results increase our understanding of the development of children’s altruistic behavior and animate-inanimate distinctions. Our findings also raise important ethical questions for the field of child-robot interaction.

Special Issue on Human-Robot Interaction in Close Distance

As social robot research is advancing, the interaction distance between people and robots is decreasing. Indeed, although we were once required to maintain a certain physical distance from traditional industrial robots for safety, we can now interact with social robots in such a close distance that we can touch them. The physical existence of social robots will be essential to realize natural and acceptable interactions with people in daily environments. Because social robots function in our daily environments, we must design scenarios where robots interact closely with humans by considering various viewpoints. Interactions that involve touching robots influence the changes in the behavior of a person strongly. Therefore, robotics researchers and developers need to design such scenarios carefully. Based on these considerations, this special issue focuses on close human-robot interactions. This special issue on “Human-Robot Interaction in Close Distance” includes a review paper and 11 other interesting papers covering various topics such as social touch interactions, non-verbal behavior design for touch interactions, child-robot interactions including physical contact, conversations with physical interactions, motion copying systems, and mobile human-robot interactions. We thank all the authors and reviewers of the papers and hope this special issue will help readers better understand human-robot interaction in close distance.

GEARing smart environments for pediatric motor rehabilitation

BackgroundThere is a lack of early (infant) mobility rehabilitation approaches that incorporate natural and complex environments and have the potential to concurrently advance motor, cognitive, and social development. The Grounded Early Adaptive Rehabilitation (GEAR) system is a pediatric learning environment designed to provide motor interventions that are grounded in social theory and can be applied in early life. Within a perceptively complex and behaviorally natural setting, GEAR utilizes novel body-weight support technology and socially-assistive robots to both ease and encourage mobility in young children through play-based, child-robot interaction. This methodology article reports on the development and integration of the different system components and presents preliminary evidence on the feasibility of the system.MethodsGEAR consists of the physical and cyber components. The physical component includes the playground equipment to enrich the environment, an open-area body weight support (BWS) device to assist children by partially counter-acting gravity, two mobile robots to engage children into motor activity through social interaction, and a synchronized camera network to monitor the sessions. The cyber component consists of the interface to collect human movement and video data, the algorithms to identify the children’s actions from the video stream, and the behavioral models for the child-robot interaction that suggest the most appropriate robot action in support of given motor training goals for the child. The feasibility of both components was assessed via preliminary testing. Three very young children (with and without Down syndrome) used the system in eight sessions within a 4-week period.ResultsAll subjects completed the 8-session protocol, participated in all tasks involving the selected objects of the enriched environment, used the BWS device and interacted with the robots in all eight sessions. Action classification algorithms to identify early child behaviors in a complex naturalistic setting were tested and validated using the video data. Decision making algorithms specific to the type of interactions seen in the GEAR system were developed to be used for robot automation.ConclusionsPreliminary results from this study support the feasibility of both the physical and cyber components of the GEAR system and demonstrate its potential for use in future studies to assess the effects on the co-development of the motor, cognitive, and social systems of very young children with mobility challenges.

Collaborative and Inclusive Process with the Autism Community: A Case Study in Colombia About Social Robot Design

One of the most promising areas in which social assistive robotics has been introduced is therapeutic intervention for children with autism spectrum disorders (CwASD). Even though there are promising results in therapeutic contexts, there is a lack of guidelines on how to select the appropriate robot and how to design and implement the child–robot interaction. The use of participatory design (PD) methods in the design of technology-based processes for CwASD is a recognition of the stakeholders as “experts” in their fields. This work explores the benefits brought by the use of PD methods in the design of a social robot, with a specific focus on their use in autism spectrum disorders therapies on the Colombian autism community. Based on what proved to be effective in our previous research, we implemented participatory methods for both the CwASD and the stakeholders. The process leverages the active role of participants using a focus group approach with parents and specialists, and scene cards, narrative and handmade generative methods with the children. To overcome some challenges of traditional PD processes, where not all community actors are considered, we included a Colombian community consisting of therapists, nurses, caregivers and parents. The proposed PD process provides an opportunity to learn from several community actors (and thus different cultural and social aspects of developing countries), improving traditional robot design methods. In this way, the findings are summarized through a set of guidelines regarding the design of a social robot-device suitable to be implemented for robot-assisted therapy for CwASD.

Analysis of Attention in Child–Robot Interaction Among Children Diagnosed with Cognitive Impairment

Interacting with social robots has been reported as potentially beneficial for children with social communication difficulties, with one of the promising applications being the practising of social skills, such as joint attention. We present the analysis of attention skills in children with cognitive impairments over a series of child–robot interaction sessions. Here, an interaction consists of five different modules. The first module introduces the child to the robot. The next three modules are the task modules during which children are expected to improve their attention skills during the completion of a series of social tasks. The final module is a free style interaction, where the duration of interaction between the child and robot was used as a proxy to indicate the attention of the child towards a robot. Our analysis showed that the majority of the children reduced their task completion time in modules two to four, indicating an improvement in attention. Moreover, most of the children showed positive engagement towards the robot and spent an average of 120 s during the free style interaction in module five. The positive response suggests that the robot, via child–robot interaction could be a useful and engaging tool to improve attention skills of the children with cognitive impairment.

Children's reliance on the non-verbal cues of a robot versus a human.

Robots are used for language tutoring increasingly often, and commonly programmed to display non-verbal communicative cues such as eye gaze and pointing during robot-child interactions. With a human speaker, children rely more strongly on non-verbal cues (pointing) than on verbal cues (labeling) if these cues are in conflict. However, we do not know how children weigh the non-verbal cues of a robot. Here, we assessed whether four- to six-year-old children (i) differed in their weighing of non-verbal cues (pointing, eye gaze) and verbal cues provided by a robot versus a human; (ii) weighed non-verbal cues differently depending on whether these contrasted with a novel or familiar label; and (iii) relied differently on a robot’s non-verbal cues depending on the degree to which they attributed human-like properties to the robot. The results showed that children generally followed pointing over labeling, in line with earlier research. Children did not rely more strongly on the non-verbal cues of a robot versus those of a human. Regarding pointing, children who perceived the robot as more human-like relied on pointing more strongly when it contrasted with a novel label versus a familiar label, but children who perceived the robot as less human-like did not show this difference. Regarding eye gaze, children relied more strongly on the gaze cue when it contrasted with a novel versus a familiar label, and no effect of anthropomorphism was found. Taken together, these results show no difference in the degree to which children rely on non-verbal cues of a robot versus those of a human and provide preliminary evidence that differences in anthropomorphism may interact with children’s reliance on a robot’s non-verbal behaviors.

. Social Robots For Reinforcing Attention And Forming Emotional Knowledge Of Children With Special Educational Needs

Emotional child-robot interaction helps catching quickly child attention and enhance information perception during learning and verbalization in children with Special Educational Needs (SEN). This will improve the pedagogical rehabilitation for these children and also develop their emotional knowledge and memory by play-like activities mediated by emotion-expressive social robots. The designed EEG-based portable Brain-Computer Interface (BCI) measures and features in real time the brain electrical activity in order to analyze the correlated attentional or emotional states of a child. The output performance scores are used either by special educators for assessment of the emotional states and cognitive performance of a child or as inputs for robot control in the play-learning scenarios. BCI is a new technology for human-robot interaction and it can evolve into technology for self-regulatory training of attention and emotional skills via neurofeedback exposed on the robot. This study marks a first step towards how to use the advantages of educational theater in Social Robotics to facilitate the perceptual processing of a child and at the same time to augment the emotional talent of an actor to an emotion-expressive robot. A low resolution EMOTIV brain-listening headset is used to translate the head/face movements or child brain activity into robot commands. Then, they are wirelessly transmitted to robot sensors, modules and controllers. Since the attention or emotional responses of children with SEN make robots to act, these skills are naturally reinforced in the play.

Fusing Body Posture With Facial Expressions for Joint Recognition of Affect in Child–Robot Interaction

In this paper we address the problem of multi-cue affect recognition in challenging scenarios such as child-robot interaction. Towards this goal we propose a method for automatic recognition of affect that leverages body expressions alongside facial ones, as opposed to traditional methods that typically focus only on the latter. Our deep-learning based method uses hierarchical multi-label annotations and multi-stage losses, can be trained both jointly and separately, and offers us computational models for both individual modalities, as well as for the whole body emotion. We evaluate our method on a challenging child-robot interaction database of emotional expressions collected by us, as well as on the GEMEP public database of acted emotions by adults, and show that the proposed method achieves significantly better results than facial-only expression baselines.

Children Teach Handwriting to a Social Robot with Different Learning Competencies

As robots are entering into educational fields to enhance children’s learning, it becomes relevant to explore different methods of learning in the area of child–robot interaction. In this article, we present an autonomous educational system incorporating a social robot to enhance children’s handwriting skills. The system provides a one-to-one learning scenario based on the learning-by-teaching approach where a tutor-child assess the handwriting skills of a learner-robot. The robot’s writing was generated by an algorithm incorporating human-inspired movements and could reproduce a set of writing errors. We tested the system by conducting two multi-session studies. In the first study, we assigned the robot two contrasting competencies: ‘learning’ and ‘non-learning’. We measured the differences in children’s learning gains and changes in their perceptions of the learner-robot. The second study followed a similar interaction scenario and research questions, but this time the robot performed three learning competencies: ‘continuous-learning’; ‘non-learning’ and ‘personalised-learning’. The findings of these studies show that the children learnt with the robot that exhibits learning competency and children’s learning and perceptions of the robot changed as interactions unfold, confirming the need for longitudinal studies. This research supports that the contrasting learning competencies of social robots can impact children’s learning differently in peer-learning scenarios.

Bridging Multilevel Time Scales in HRI

In this article, we present a multi-level time scales framework for the analysis of human-robot interaction (HRI). Such a framework allows HRI scientists to model the inter-relation between measures and factors of an experiment. Our final goal with the introduction of this framework is to unify scientific practice in the HRI community for better reproducibility. Our new approach transposes Newell’s framework of human actions to model human-robot interaction. Measures from the interaction are sorted into categories (time scales) corresponding to the temporal constraints proposed by Newell. According to this sorting, a bottom-up or top-down analysis can then be performed to correlate variables which allows a better understanding and explanation of the interaction. The utilization of our method within two experimental use cases is then presented. The first one, a child-robot interaction, involves two robots and one child playing a memory game. The second is based on an analysis of the PInSoRo dataset, involving 30 child-robot pairs in a freeplay interaction. Finally, we introduce clear guidelines to re-use the framework.

Design and Validation of MOMDP Models for Child–Robot Interaction Within Tasks of Robot-Assisted ASD Diagnostic Protocol

The existing procedures for autism spectrum disorder diagnosis are time-consuming and challenging both for evaluators and children being evaluated. Occurrence of low agreement rates between different clinicians when evaluating a child suggests that there exists a need for a more objective approach to diagnostics. To that end, we developed a robot-assisted ASD diagnostic protocol. In this work the focus is on robot reasoning for tasks of the protocol. We propose the mixed observability Markov decision process models for tasks which infer information about the state of a child based on observations of child’s behavior. In order to formulate observation probabilities of task models, ASD experts are surveyed and their knowledge is encoded in the observation probabilities of task models. Expert knowledge also allowed for implementation of child behavioral models which are used to validate and tune developed models. Following the successful validation through simulations of child–robot interaction using child behavioral models, task models are validated through experimental sessions with six typically developing children and eight children with ASD. Results obtained through experiments show that the robot is capable of correctly identifying the behavior of the child within the diagnostic tasks.

A video-based assessment of likeability and usability in a play application of robots for children

Purpose The purpose of this paper is to conduct research with children, who have different abilities from adults, in terms of language understanding and level of attention, is a challenging task, especially concerning novel interactive systems such as social robots. Consequently, self-reporting methods are often replaced or supplemented by observational methods that are usually carried out taking advantage of video recordings. However, some limitations make this approach challenging for studies conducted with groups of children in real-world environments, whose relevance is being addressed more and more frequently in human-robot interaction (HRI) research. Thus, there is a growing need for rigorous observation approaches in unstructured test environments. Design/methodology/approach This paper presents an alternative analysis approach, in relation to an experimental child-robot interaction (CRI) application, which was developed at the Academy of Arts and Design, Tsinghua University, China. The proposed methodology is based on the analysis of video recordings of in-wild activities of children with a robot. The methodology has the aim of providing a framework to facilitate knowledge identification and structuring. It was implemented for experiment evaluation and validation purposes and to propose a reference structure for the organization of new experiments and the stimulation of new ideas and activities in the design process. Findings This methodology provides a logical structure, which can be used to identify the effectiveness or limits of design choices, pertaining to such aspects as the morphology or movement of robots or the choice of their specific role in education, all of which play crucial roles in the design process and could be improved to achieve better results. This structured identification is a practical implication for the design process, above all when it is oriented toward social robots and their interaction with children or elderly senile people. In this case, the outcomes were the identification of important elements of an experiment (psychological profiles of the involved children and possible problems or risks) and their impact on the design process. Originality/value The methodological approach, which structures and uses cognitive maps to elaborate multicriteria evaluation models, is not new to the operations research field (where it is defined as a multimethodology application of Soft OR), but it has not yet been applied in the field of HRI studies, to analyze children’s perception of a robot and to identify the factors that can affect a good CRI or to structure knowledge that can be shared to guide the design process of robots for the experience of children playing.

Visual and Thermal Image Processing for Facial Specific Landmark Detection to Infer Emotions in a Child-Robot Interaction.

Child-Robot Interaction (CRI) has become increasingly addressed in research and applications. This work proposes a system for emotion recognition in children, recording facial images by both visual (RGB—red, green and blue) and Infrared Thermal Imaging (IRTI) cameras. For this purpose, the Viola-Jones algorithm is used on color images to detect facial regions of interest (ROIs), which are transferred to the thermal camera plane by multiplying a homography matrix obtained through the calibration process of the camera system. As a novelty, we propose to compute the error probability for each ROI located over thermal images, using a reference frame manually marked by a trained expert, in order to choose that ROI better placed according to the expert criteria. Then, this selected ROI is used to relocate the other ROIs, increasing the concordance with respect to the reference manual annotations. Afterwards, other methods for feature extraction, dimensionality reduction through Principal Component Analysis (PCA) and pattern classification by Linear Discriminant Analysis (LDA) are applied to infer emotions. The results show that our approach for ROI locations may track facial landmarks with significant low errors with respect to the traditional Viola-Jones algorithm. These ROIs have shown to be relevant for recognition of five emotions, specifically disgust, fear, happiness, sadness, and surprise, with our recognition system based on PCA and LDA achieving mean accuracy (ACC) and Kappa values of 85.75% and 81.84%, respectively. As a second stage, the proposed recognition system was trained with a dataset of thermal images, collected on 28 typically developing children, in order to infer one of five basic emotions (disgust, fear, happiness, sadness, and surprise) during a child-robot interaction. The results show that our system can be integrated to a social robot to infer child emotions during a child-robot interaction.

Robot’s adaptive emotional feedback sustains children’s social engagement and promotes their vocabulary learning: a long-term child–robot interaction study

In this article, we present an emotion and memory model for a social robot. The model allowed the robot to create a memory account of a child’s emotional events over four individual sessions. The robot then adapted its behaviour based on the developed memory. The model was applied on the NAO robot to teach vocabulary to children while playing the popular game ‘Snakes and Ladders’. We conducted an exploratory evaluation of our model with 24 children at a primary school for 2 weeks to verify its impact on children’s long-term social engagement and overall vocabulary learning. Our preliminary results showed that the behaviour generated based on our model was able to sustain social engagement. In addition, it also helped children to improve their vocabulary. We also evaluated the impact of the positive, negative and neutral emotional feedback of the NAO robot on children’s vocabulary learning. Three groups of children (eight per group) interacted with the robot on four separate occasions over a period of 2 weeks. Our results showed that the condition where the robot displayed positive emotional feedback had a significantly positive effect on the child’s vocabulary learning performance as compared to the two other conditions: negative feedback and neutral feedback.

Robot-Assisted Autism Spectrum Disorder Diagnostic Based on Artificial Reasoning

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects people from birth, whose symptoms are found in the early developmental period. The ASD diagnosis is usually performed through several sessions of behavioral observation, exhaustive screening, and manual coding behavior. The early detection of ASD signs in naturalistic behavioral observation may be improved through Child-Robot Interaction (CRI) and technological-based tools for automated behavior assessment. Robot-assisted tools using CRI theories have been of interest in intervention for children with Autism Spectrum Disorder (CwASD), elucidating faster and more significant gains from the diagnosis and therapeutic intervention when compared to classical methods. Additionally, using computer vision to analyze child’s behaviors and automated video coding to summarize the responses would help clinicians to reduce the delay of ASD diagnosis. In this article, a CRI to enhance the traditional tools for ASD diagnosis is proposed. The system relies on computer vision and an unstructured and scalable network of RGBD sensors built upon Robot Operating System (ROS) and machine learning algorithms for automated face analysis. Also, a proof of concept is presented, with participation of three typically developing (TD) children and three children in risk of suffering from ASD.