NAO Robot Research Articles

A New Knowledge Primitive of Digits Recognition for NAO Robot Using MNIST Dataset and CNN Algorithm for Children’s Visual Learning Enhancement

Aim/Purpose: Our study is focused on prototyping, development, testing, and deployment of a new knowledge primitive for the humanoid robot assistant NAO, in order to enhance student visual learning by establishing a human-robot interaction. Background: This new primitive, utilizing a convolutional neural network (CNN), enables real-time recognition of handwritten digits captured by the NAO robot, a humanoid robot assistant developed by SoftBank Robotics. It is equipped with advanced capabilities, including a wide range of sensors, cameras, and interactive features. By integrating the proposed primitive, the NAO robot gains the ability to accurately recognize handwritten digits, contributing to improved student visual learning experiences. Methodology: Our developed primitive consists of the use of a convolutional neural network (CNN) so that the robot is able to recognize the handwriting of the digits present in the input image received in real-time. The NAO robot establishes interaction with the learners through a scenario based on a predefined assignment. In this scenario, NAO captures the digit handwritten by the learner via its camera, recognizes the digit using the deep learning model generated by the MNIST dataset, and announces to the learner the handwritten digit in the input image. The prototype is realized using the concept of a distributed system allowing the distribution of tasks in four different computing nodes. Contribution: Our research makes a significant contribution by equipping the humanoid robot NAO with a cognitive intelligence system through the integration of a new knowledge primitive based on handwriting digit recognition (HWDR). Our approach used to create and implement this primitive in the NAO robot is interesting and innovative, and presents a promising provision for enhancing the visual learning experience of children and young students with special needs, based on the use of distributed systems that divide the work using various components distributed over several nodes, coordinating their efforts to perform tasks more efficiently than a single device besides the NAO robot. Findings: We designed our model using specific parameters and a fully convolutional neural network architecture, which includes three residual depthwise separable convolutions, each followed by batch normalization and ReLU activation. To evaluate the performance of our model, we tested it on the MNIST dataset, where we achieved a remarkable accuracy, F1 score, and recall of 99%. An experiment was conducted to test our implemented primitive and see the effectiveness of this invention for enhancing visual learning in children with special needs. We developed a visual learning strategy based on the creation of engaging activities mediated by the NAO robot in an educational context. The results showed that participants achieved a strong commitment to the NAO robot, appreciating its ability to recognize handwritten digits and highlighting its promising potential to enrich visual learning experiences. Participants expressed a strong preference for teaching methods integrating assistive learning technologies, demonstrating the positive impact of our humanoid assistant robot on improving learning and visual intelligence in an educational environment. Recommendations for Practitioners: Encourage creativity and innovation in the field of robotics and special needs. This can lead to new and effective solutions that improve the lives of students with special needs. Recommendation for Researchers: Test and evaluate the proposed robotics solutions to ensure they are effective and making a positive impact. Use feedback from users, educators, and parents to refine and improve your solutions. Also, ensure that the robotics solutions are accessible to students with a range of abilities. This may involve designing solutions that are adjustable or providing alternative means of access. Impact on Society: As there are several ways to educate, there are multiple forms of learning. With the help of this learning procedure and strategy, the human teacher collaborates with the robot assistance NAO to improve visual learning among students. The findings of this research can serve as an application for the implementation of various pedagogical methods that will assist in meeting the needs of the majority of learners. Future Research: Our future research will concentrate on addressing the educational needs of students with special needs, enabling them to overcome their challenges and reach academic excellence in an inclusive environment. To achieve this goal, we plan to leverage the capabilities of social robots, which have emerged as a significant contributor to the field of human-robot interaction, particularly in facilitating inclusive education. These agents have proven to be effective in providing support to students with special needs, thereby enabling them to receive the education they need to succeed.

Exploring space for robot mistakes in child robot interactions

Abstract Understanding the impact of robot errors in child-robot-interactions (CRI) is critical, as current technological systems are still limited and may randomly present a variety of mistakes during interactions with children. In this study we manipulate a task-based error of a NAO robot during a semi-autonomous computational thinking task implemented with the Cozmo robot. Data from 72 children aged 7–10 were analysed regarding their attitudes towards NAO (social trust, competency trust, liking, and perceived agency), their behaviour towards the robot (self-disclosure, following recommendations), as well as their task performance. We did not find quantitative effects of the robot’s error on children’s self-reported attitudes, behaviour, or task performance. Age was also not significantly related to either social attitudes or behaviours towards NAO, although there were some age-related differences in task performance. Potential reasons behind the lack of statistical effects and limitations of the study with regards to the manipulation of robot errors are discussed and insights into the design of future CRI studies provided.

Data learning‐based model‐free adaptive control and application to an NAO robot

AbstractIn this article, an improved model‐free adaptive control method based on the controller dynamic linearization technique combined with the locally weighted regression‐based lazy learning method (cMFAC‐LL) is presented, and it is applied to solve the path‐tracking control problem for an NAO robot. In the proposed cMFAC‐LL method, two dynamic linearization techniques are first applied on the controlled plant, and then the cMFAC‐LL controller is further designed with the time‐varying parameters estimated using a novel locally weighted regression‐based lazy learning (LWR‐LL) technique. The greatest advantage of the cMFAC‐LL method is that it is a pure data‐driven control method, and the designed controller makes full use of both the online and offline measurement data of the plant. Moreover, the introduction of the local learning (LL) method gives the cMFAC‐LL method strong data learning ability and satisfactory control performance. The stability of cMFAC‐LL is proven via rigorous mathematical analysis. Furthermore, cMFAC‐LL is applied to the path‐tracking control of the NAO robot under different walking environments, by which the applicability of cMFAC‐LL is further verified.

Follow control of upper limb rehabilitation training based on Kinect and NAO robot

Gesture imitation is a common rehabilitation strategy in limb rehabilitation training. In traditional rehabilitation training, patients need to complete training actions under the guidance of rehabilitation physicians. However, due to the limited resources of the hospital, it cannot meet the training and guidance needs of all patients. In this paper, we proposed a following control method based on Kinect and NAO robot for the gesture imitation task in rehabilitation training. The method realized the joint angles mapping from Kinect coordination to NAO robot coordination through inverse kinematics algorithm. Aiming at the deflection angle estimation problem of the elbow joint, a virtual space plane was constructed and realized the accurate estimation of deflection angle. Finally, a comparative experiment for deflection angle of the elbow joint angle was conducted. The experimental results showed that the root mean square error of the angle estimation value of this method in right elbow transverse deflection and vertical deflection directions was 2.734° and 2.159°, respectively. It demonstrates that the method can follow the human movement in real time and stably using the NAO robot to show the rehabilitation training program for patients.

Design and Evaluation of a Personal Robot Playing a Self-Management for Children with Obesity

The preponderance of obesity and being overweight among children has increased significantly during the last two decades in Saudi Arabia and United Arab Emirates (UAE) with overwhelming consequences to public health. Most recommended approaches have paid attention to a healthier diet and physical activity (PA) to reduce obesity. Recent research shows that the use of social robots could play a vital role in encouraging children to improve their skills in self-management. As children need to be surprised and feel a sense of enjoyment when involved in any activity where they can spend time and actively engage in activities, social robots could be an effective intervention for this purpose. In this context, the current project aimed to build an innovation social robot system to offer a set of activities to help obese children improve their capabilities to manage their selves properly and increase their obesity knowledge. This study aimed to determine the perceptions of obese children towards the NAO robot, a new medical technology, and analyze their responses to the robot’s advice and education-related activities. A proposed model of the intervention using the NAO robot is discussed in this study, and a pilot study was conducted to assess the performance of the proposed system. The obtained results showed an average acceptability of 89.37% for social robots to be involved in obesity management.

Improvement of the Sensor Capability of the NAO Robot by the Integration of a Laser Rangefinder

This paper focuses on integrating a laser rangefinder system with an anthropomorphic robot (NAO6—Aldebaran, United Robotics Group) to improve its sensory and operational capabilities, as part of a larger project concerning the use of these systems in “assisted living” activities. This additional sensor enables the robot to reconstruct its surroundings by integrating new information with that identified by the on-board sensors. Thus, it can identify more objects in a scene and detect any obstacles along its navigation path. This feature will improve the efficiency of navigation algorithms, increasing movement competence in environments where people live and work. Indeed, these environments are characterized by details and specificities within a range of distances that best suit the new robot design. The paper presents a laser finder integration project that consists of two different parts, which are as follows: the former, the mechanical part, provided the NAO robot’s head; the latter, the software, provided the robot with proper software drivers to enable integration of the new sensor with its acquisition system. Some experimental results in an actual environment are presented.

A brain-inspired intention prediction model and its applications to humanoid robot.

With the development of artificial intelligence and robotic technology in recent years, robots are gradually integrated into human daily life. Most of the human-robot interaction technologies currently applied to home service robots are programmed by the manufacturer first, and then instruct the user to trigger the implementation through voice commands or gesture commands. Although these methods are simple and effective, they lack some flexibility, especially when the programming program is contrary to user habits, which will lead to a significant decline in user experience satisfaction. To make that robots can better serve human beings, adaptable, simple, and flexible human-robot interaction technology is essential. Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction model to enable the robot to perform actions according to the user's intention. With the spike-timing-dependent plasticity (STDP) mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by (N 2 - N)/4, where N is the number of intentions.

Expecting, understanding, relating, and interacting-older, middle-aged and younger adults' perspectives on breakdown situations in human-robot dialogues.

Purpose: The purpose of this study is to explore how older, middle aged and younger adults perceive breakdown situations caused by lack of or inconsistent knowledge, sudden focus shifts, and conflicting intentions in dialogues between a human and a socially intelligent robot in a home environment, and how they perceive strategies to manage breakdown situations. Methods: Scenarios embedding dialogues on health-related topics were constructed based on activity-theoretical and argumentation frameworks. Different reasons for breakdown situations and strategies to handle these were embedded. The scenarios were recorded in a Wizard-of-Oz setup, with a human actor and a Nao robot. Twenty participants between 23 and 72years of age viewed the recordings and participated in semi-structured interviews conducted remotely. Data were analyzed qualitatively using thematic analysis. Results: Four themes relating to breakdown situations emerged: expecting, understanding, relating, and interacting. The themes span complex human activity at different complementary levels and provide further developed understanding of breakdown situations in human-robot interaction (HRI). Older and middle-aged adults emphasized emphatic behavior and adherence to social norms, while younger adults focused on functional aspects such as gaze, response time, and length of utterances. A hierarchical taxonomy of aspects relating to breakdown situations was formed, and design implications are provided, guiding future research. Conclusion: We conclude that a socially intelligent robot agent needs strategies to 1) construct and manage its understanding related to emotions of the human, social norms, knowledge, and motive on a higher level of meaningful human activity, 2) act accordingly, for instance, adhering to transparent social roles, and 3) resolve conflicting motives, and identify reasons to prevent and manage breakdown situations at different levels of collaborative activity. Furthermore, the novel methodology to frame the dynamics of human-robot dialogues in complex activities using Activity Theory and argumentation theory was instrumental in this work and will guide the future work on tailoring the robot's behavior.

NAO robots as context to teach numerical methods

This article examines the contextual role that NAO humanoid robots play in the learning environment of a numerical methods course through a design-based research approach. Under the design of intentional and reflective planning that emanates from the socio-critical paradigm in education, the feasibility of using this type of robot in the classroom as a motivation element to engage students in understanding numerical analysis issues is addressed. In particular, the active learning of engineering students is achieved by studying such robots' forward and inverse kinematics. The didactical sequence to teach the generalized Newton–Raphson method for solving non-linear equation systems is validated as part of a Hypothetical Learning Trajectory proposed. Besides, this research provides a brief theoretical discussion on reflection-in-action of the teacher of Higher Education level.

The effect of a social robot mediator on speech characteristics of children with autism spectrum disorder

Robot-mediated interventions have been investigated for the treatment of social skill deficits amongst children with Autism Spectrum Disorder (ASD). Does the use of a Nao robot as a mediator increase vocal interaction between children with ASD? The present study examined the vocalization and turn-taking rate in six children with ASD (mean age = 11.4 years, SD = 0.86 years) interacting with and without a Nao robot for 10 sessions, order counterbalanced. Each session lasted nine minutes. In the Robot condition, the robot provided vocal prompts; in the No Robot condition, children interacted freely. Child vocalization and turn-taking rate defined as the number of utterances/turns per second were measured. Results demonstrated that three children produced higher vocalization and turn-taking rates when a robot was present, and two when it was absent. One participant produced higher vocalization rates when the robot was not present, but more conversational turns when the robot was present. The findings suggest that the use of a Nao robot as a social mediator increases vocalization and turn-taking rates among children with ASD, but large individual variability is observed. The effect of the robot as a mediator on lexical diversity of child speech will also be investigated.

Automatic Aesthetics Evaluation of Robotic Dance Poses Based on Hierarchical Processing Network.

Vision plays an important role in the aesthetic cognition of human beings. When creating dance choreography, human dancers, who always observe their own dance poses in a mirror, understand the aesthetics of those poses and aim to improve their dancing performance. In order to develop artificial intelligence, a robot should establish a similar mechanism to imitate the above human dance behaviour. Inspired by this, this paper designs a way for a robot to visually perceive its own dance poses and constructs a novel dataset of dance poses based on real NAO robots. On this basis, this paper proposes a hierarchical processing network-based approach to automatic aesthetics evaluation of robotic dance poses. The hierarchical processing network first extracts the primary visual features by using three parallel CNNs, then uses a synthesis CNN to achieve high-level association and comprehensive processing on the basis of multi-modal feature fusion, and finally makes an automatic aesthetics decision. Notably, the design of this hierarchical processing network is inspired by the research findings in neuroaesthetics. Experimental results show that our approach can achieve a high correct ratio of aesthetic evaluation at 82.3%, which is superior to the existing methods.

Conceptualization and development of an autonomous and personalized early literacy content and robot tutor behavior for preschool children

Personalized learning has a higher impact on students’ progress than traditional approaches. However, current resources required to implement personalization are scarce. This research aims to conceptualize and develop an autonomous robot tutor with personalization policy for preschool children aged between three to five years old. Personalization is performed by automatically adjusting the difficulty level of the lesson delivery and assessment, as well as adjusting the feedback based on the reaction of children. This study explores three child behaviors for the personalization policy: (i) academic knowledge (measured by the correctness of the answer), (ii) executive functioning of attention (measured by the orientation and the gaze direction of child’s body), and (iii) working memory or hesitation (measured by the time lag before the answer). Moreover, this study designed lesson content through interviews with teachers and deployed the personalization interaction policy through the NAO robot with five children in a case user study method. We qualitatively analyze the session observations and parent interviews, as well as quantitatively analyze knowledge gain through pre- and posttests and a parent questionnaire. The findings of the study reveal that the personalized interaction with the robot showed a positive potential in increasing the children’s learning gains and attracting their engagement. As general guidelines based on this pilot study, we identified additional personalization strategies that could be used for autonomous personalization policies based on each child’s behavior, which could have a considerable impact on child learning.

Target Recognition and Navigation Path Optimization Based on NAO Robot

The NAO robot integrates sensors, vision systems, and control systems. Its monocular vision system is adopted to locate the target object in the three-dimensional space of robots. Firstly, a positioning model based on monocular vision is established according to the principle of small hole perspective. Then, the position coordinates of the target center are obtained in the image coordinate system. In the model mentioned above, the relationship between position coordinates and image coordinates is established at a certain space height. According to this relationship, the two-dimensional coordinates in the image are converted into the three-dimensional coordinates in the robot coordinate system. After getting the target location, we establish the navigation map and find the optimal path under the unknown environment. Based on the simultaneous localization and the mapping (SLAM) theory, the sonar sensor of the NAO robot is used to detect the distance between the robot and the obstacles or between the robot and the end landmark. Moreover, the sonar sensor and the camera are used to distinguish the obstacle and the landmark. After the navigation map is built, the bi-directional parallel search strategy and the simulated annealing algorithm are introduced to improve the traditional artificial bee colony algorithm, and the improved artificial bee colony algorithm is proposed to find an optimal path in the navigation map. Finally, the experimental results show that based on the built environment map, the robot can find an optimal path from the origin to the landmark on the premise of avoiding obstacles.

The Use of Social Robots in the Diagnosis of Autism in Preschool Children

The present study contributes to the research problem of applying social robots in autism diagnosis. There is a common belief that existing diagnostic methods for autistic spectrum disorder are not effective. Advances in Human–Robot Interactions (HRI) provide potential new diagnostic methods based on interactive robots. We investigated deficits in turn-taking in preschool children by observing their interactions with the NAO robot during two games: (Dance with me vs. Touch me). We compared children’s interaction profiles with the robot (five autistic vs. five typically developing young children). Then, to investigate turn-taking deficits, we adopted a rating procedure to indicate differences between both groups of children based on an observational scale. A statistical analysis based on ratings of the children’s interactions with the NAO robot indicated that autistic children presented a deficient level of turn-taking behaviors. Our study provides evidence for the potential of designing and implementing an interactive dyadic game between a child and a social robot that can be used to detect turn-taking deficits based on objective measures. We also discuss our results in the context of existing studies and propose guidelines for a robotic-enabled autism diagnosis system.

Exploring How People with Expressive Aphasia Interact with and Perceive a Social Robot

People with aphasia need high-intensive language training to significantly improve their language skills, however practical barriers arise. Socially assistive robots have been proposed as a possibility to provide additional language training. However, it is yet unknown how people with aphasia perceive interacting with a social robot, and which factors influence this interaction. The aim of this study was to gain insight in how people with mild to moderate chronic expressive aphasia perceived interacting with the social robot NAO, and to explore what needs and requisites emerged. A total of 11 participants took part in a single online semi-structured interaction, which was analysed using observational analysis, thematic analysis, and post-interaction questionnaire. The findings show that participants overall felt positive towards using the social robot NAO. Moreover, they perceived NAO as enjoyable, useful, and to a lesser extent easy to use. This exploratory study provides a tentative direction for the intention of people with mild to moderate chronic expressive aphasia to use social robots. Design implications and directions for future research are proposed.

Efficacy of a Robot-Assisted Intervention in Improving Learning Performance of Elementary School Children with Specific Learning Disorders.

(1) Background: There has been significant recent interest in the potential role of social robots (SRs) in special education. Specific Learning Disorders (SpLDs) have a high prevalence in the student population, and early intervention with personalized special educational programs is crucial for optimal academic achievement. (2) Methods: We designed an intense special education intervention for children in the third and fourth years of elementary school with a diagnosis of a SpLD. Following confirmation of eligibility and informed consent, the participants were prospectively and randomly allocated to two groups: (a) the SR group, for which the intervention was delivered by the humanoid robot NAO with the assistance of a special education teacher and (b) the control group, for which the intervention was delivered by the special educator. All participants underwent pre- and post-intervention evaluation for outcome measures. (3) Results: 40 children (NAO = 19, control = 21, similar baseline characteristics) were included. Pre- and post-intervention evaluation showed comparable improvements in both groups in cognition skills (decoding, phonological awareness and reading comprehension), while between-group changes favored the NAO group only for some phonological awareness exercises. In total, no significant changes were found in any of the groups regarding the emotional/behavioral secondary outcomes. (4) Conclusion: NAO was efficient as a tutor for a human-supported intervention when compared to the gold-standard intervention for elementary school students with SpLDs.

The Quantitative Case-by-Case Analyses of the Socio-Emotional Outcomes of Children with ASD in Robot-Assisted Autism Therapy

With its focus on robot-assisted autism therapy, this paper presents case-by-case analyses of socio-emotional outcomes of 34 children aged 3–12 years old, with different cases of Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD). We grouped children by the following characteristics: ASD alone (n = 22), ASD+ADHD (n = 12), verbal (n = 11), non-verbal (n = 23), low-functioning autism (n = 24), and high-functioning autism (n = 10). This paper provides a series of separate quantitative analyses across the first and last sessions, adaptive and non-adaptive sessions, and parent and no-parent sessions, to present child experiences with the NAO robot, during play-based activities. The results suggest that robots are able to interact with children in social ways and influence their social behaviors over time. Each child with ASD is a unique case and needs an individualized approach to practice and learn social skills with the robot. We, finally, present specific child–robot intricacies that affect how children engage and learn over time as well as across different sessions.

Comparing Social Robot Embodiment for Child Musical Education.

The present research focuses in the comparison of two social robot models running the same Human-Robot Interaction (HRI) applications targeting the context of music education for children aged 9-11, with the objective of underlying the design choices favored by the target audience on the running tasks. The Guitar Tuner consists of two main functionalities: tuning process and performance evaluation, which we implemented using the NAO and Zenbo robots. User evaluation included 20 children and assessed their perceived robot embodiment preferences (e.g., shape, robot motion, displays, and emotional expressivity) and perceived usability aspects. The evaluation used an experimental remote protocol supporting collecting online feedback with users during the COVID-19 pandemic. Empirical results supported performing quantitative and qualitative evaluations of the HRI application and highlighting the perceived differences of robot embodiment features. The discussions center on improving a future version of the HRI application, plus children’s considerations about their preferred robot embodiment features during the observation sessions. Finally, we propose recommendations for robot embodiment design for children and learning based on this case study and discuss protocol limitations during the social distancing context, that we believe as a valid alternative to move forward with experimental designs, particularly in robotics, becoming a great contribution to other researchers facing similar hurdles.

Could NAO Robot Function as Model Demonstrating Joint Attention Skills for Children with Autism Spectrum Disorder? An Exploratory Study

Previous studies reported that children with autism spectrum disorder (ASD) show a certain interest in social robots. This makes social robots potential to be a model to teach social skills. This exploratory study aims to investigate whether three types of joint attention skills (i.e., eye-contact, pointing, gaze-following) could be improved for five preschoolers with ASD using an evidence-based robot-modeling intervention with a humanoid social robot NAO. Our observation shows that these children were motivated when interacting with NAO by following and responding correctly to NAO’s joint attention behaviors. Although some improvements were found, no pattern or systematic effect could be revealed. In the future, more evidence-based studies are needed to investigate the benefits of robot-assisted therapy more deeply.

Emotion Recognition in Human–Robot Interaction Using the NAO Robot

Affective computing can be implemented across many fields in order to provide a unique experience by tailoring services and products according to each person’s needs and interests. More specifically, digital learning and robotics in education can benefit from affective computing with a redesign of the curriculum’s contents based on students’ emotions during teaching. This key feature is observed during traditional learning methods, and robot tutors are adapting to it gradually. Following this trend, this work focused on creating a game that aims to raise environmental awareness by using the social robot NAO as a conversation agent. This quiz-like game supports emotion recognition with DeepFace, allowing users to review their answers if a negative emotion is detected. A version of this game was tested during real-life circumstances and produced favorable results, both for emotion analysis and overall user enjoyment.