NAO Humanoid Robot Research Articles

Localización de robots basada en red neural utilizando características visuales

This paper outlines the development of a module capable of constructing a map-building algorithm using inertial odometry and visual features. It incorporates an object recognition module that leverages local features and unsupervised artificial neural networks to identify non-dynamic elements in a room and assign them positions. The map is modeled using a neural network, where each neuron corresponds to an absolute position in the room. Once the map is constructed, capturing just a couple of images of the environment is sufficient to estimate the robot's location. The experiments were conducted using both simulation and a real robot. The Webots environment with the virtual humanoid robot NAO was used for the simulations. Concurrently, results were obtained using a real NAO robot in a setting with various objects. The results demonstrate notable precision in localization within the two-dimensional maps, achieving an accuracy of ± (0.06, 0.1) m in simulations contrasted with the natural environment, where the best value achieved was ± (0.25, 0.16) m.

Managing social-educational robotics for students with autism spectrum disorder through business model canvas and customer discovery.

Social-educational robotics, such as NAO humanoid robots with social, anthropomorphic, humanlike features, are tools for learning, education, and addressing developmental disorders (e.g., autism spectrum disorder or ASD) through social and collaborative robotic interactions and interventions. There are significant gaps at the intersection of social robotics and autism research dealing with how robotic technology helps ASD individuals with their social, emotional, and communication needs, and supports teachers who engage with ASD students. This research aims to (a) obtain new scientific knowledge on social-educational robotics by exploring the usage of social robots (especially humanoids) and robotic interventions with ASD students at high schools through an ASD student-teacher co-working with social robot-social robotic interactions triad framework; (b) utilize Business Model Canvas (BMC) methodology for robot design and curriculum development targeted at ASD students; and (c) connect interdisciplinary areas of consumer behavior research, social robotics, and human-robot interaction using customer discovery interviews for bridging the gap between academic research on social robotics on the one hand, and industry development and customers on the other. The customer discovery process in this research results in eight core research propositions delineating the contexts that enable a higher quality learning environment corresponding with ASD students' learning requirements through the use of social robots and preparing them for future learning and workforce environments.

Evaluating speech-in-speech perception via a humanoid robot.

Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. As a first step toward the use of NAO in psychophysical testing of speech- in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Toward Robot Scale (NARS) and quantification of behavioural cues (backchannels). Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a relatively positive attitude toward "situations of interactions" with robots prior to the experiment, but otherwise showed neutral attitudes toward the "social influence" of and "emotions in interaction" with robots. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Overall, the study presents the potential of the NAO for presenting speech materials and collecting psychophysical measurements for speech-in-speech perception.

Tailoring the motion planning of humanoids in complex arena: a regression-firefly-based approach

AbstractIn the current investigation, a two-stage hybridization model has been used for the motion planning of humanoids in complex environmental conditions using regression analysis and the firefly algorithm. In the first step, sensory outputs are fed to the regression model, and an initial turning angle (ITA) is obtained. In the second step, the ITA is again fed as input to the firefly model along with other required inputs, and the final turning angle (FTA) is obtained. The FTA serves as the guiding parameter for the humanoids to reach their desired destinations. The developed motion planning scheme has been implemented on NAO humanoid robots in simulation and experimental platforms. A Petri-Net control strategy has been integrated along with the hybrid scheme while negotiating multiple humanoids in a common platform. The results obtained from the motion planning analysis in simulation and experimental arenas are compared against each other in terms of selected navigational parameters and observed satisfactory agreements. Finally, the proposed hybrid controller is also tested against another standard navigational model and substantial enhancement in the performance has been noticed.

Interactive mirrOring Games wIth sOCial rObot (IOGIOCO): a pilot study on the use of intransitive gestures in a sample of Italian preschool children with autism spectrum disorder.

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by persistent deficits in social communication, social interaction, and restricted behaviors. The importance of early intervention has been widely demonstrated, and developmental trajectories in ASD emphasize the importance of nonverbal communication, such as intransitive gesture production, as a possible positive prognostic factor for language development. The use of technological tools in the therapy of individuals with ASD has also become increasingly important due to their higher engagement and responsiveness to technological objects, such as robots. We developed a training protocol using the humanoid robot NAO, called IOGIOCO (Interactive mirroring Games wIth sOCial rObot), based on the use of intransitive gestures embedded in naturalistic dialogues, stimulating a triadic interaction between child, robot and therapist. The training was divided into six levels; the first 2 levels were called "familiarization levels," and the other 4 were "training levels". The technological setup includes different complexity levels, from mirroring tasks to building spontaneous interactions. We tested the protocol on 10 preschool children with ASD (aged 2-6 years) for 14 weeks. We assessed them at recruitment (T0), at the end of training (T1), and after 6 months (T2). We demonstrated the tolerability of the protocol. We found that one group (n=4, males and 2 females) reached the training level, while another and group (n=6 males) remained at a familiarization level (mirroring), we analyzed the results for the two groups. In the group that reached the training levels, we found promising results, such as an improvement in the Social Adaptive Domain of the ABAS-II questionnaire between T0 and T2. While current results will need a Randomized Controlled Trial to be confirmed, the present work sets an important milestone in using social robots for ASD treatment, aimed at impacting social and communication skills in everyday life.

New Fixed-Time Observer-Based Model-Free Fixed-Time Sliding Mode of Joint Angle Commanded NAO Humanoid Robot

New Fixed-Time Observer-Based Model-Free Fixed-Time Sliding Mode of Joint Angle Commanded NAO Humanoid Robot

Walking position commanded NAO robot using nonlinear disturbance observer-based fixed-time terminal sliding mode

The walking stability of a humanoid robot is a fundamental problem due to the complex nonlinear dynamic model of the robot’s legs. This work introduces the performance tracking control for the humanoid NAO robot by using a Nonlinear Disturbance Observer (NDO)-based Fixed-time Terminal Sliding Mode (FTSM). The influence of uncertain external disturbance is considered while implementing the control strategy to improve the walking motion of the NAO robot. An NDO is adapted to estimate the uncertainties and external disturbances. A novel FTSM surface is proposed to drive the tracking errors to zero in fixed-time. The designed NDO-based FTSM control law achieves robustness while reducing the chattering phenomenon. The Lyapunov’s stability theory is used to establish the fixed-time stability of the sliding surface and system states under the proposed control method. To validate the performance of the proposed NDO-based FTSM control, a real-time experiment was conducted on a humanoid NAO robot to demonstrate the improved tracking performance in the presence of the uncertain perturbation effect. The effectiveness of the proposed controller design is validated on a flat, upward inclined surface, and compared to another controller.

Use of a humanoid robot for auditory psychophysical testing.

Tasks in psychophysical tests can at times be repetitive and cause individuals to lose engagement during the test. To facilitate engagement, we propose the use of a humanoid NAO robot, named Sam, as an alternative interface for conducting psychophysical tests. Specifically, we aim to evaluate the performance of Sam as an auditory testing interface, given its potential limitations and technical differences, in comparison to the current laptop interface. We examine the results and durations of two voice perception tests, voice cue sensitivity and voice gender categorisation, obtained from both the conventionally used laptop interface and Sam. Both tests investigate the perception and use of two speaker-specific voice cues, fundamental frequency (F0) and vocal tract length (VTL), important for characterising voice gender. Responses are logged on the laptop using a connected mouse, and on Sam using the tactile sensors. Comparison of test results from both interfaces shows functional similarity between the interfaces and replicates findings from previous studies with similar tests. Comparison of test durations shows longer testing times with Sam, primarily due to longer processing times in comparison to the laptop, as well as other design limitations due to the implementation of the test on the robot. Despite the inherent constraints of the NAO robot, such as in sound quality, relatively long processing and testing times, and different methods of response logging, the NAO interface appears to facilitate collecting similar data to the current laptop interface, confirming its potential as an alternative psychophysical test interface for auditory perception tests.

A novel visual tracking scheme for unstructured indoor environments

In the ever-expanding sphere of assistive robotics, the pressing need for advanced methods capable of accurately tracking individuals within unstructured indoor settings has been magnified. This research endeavours to devise a realtime visual tracking mechanism that encapsulates high performance attributes while maintaining minimal computational requirements. Inspired by the neural processes of the human brain’s visual information handling, our innovative algorithm employs a pattern image, serving as an ephemeral memory, which facilitates the identification of motion within images. This tracking paradigm was subjected to rigorous testing on a Nao humanoid robot, demonstrating noteworthy outcomes in controlled laboratory conditions. The algorithm exhibited a remarkably low false detection rate, less than 4%, and target losses were recorded in merely 12% of instances, thus attesting to its successful operation. Moreover, the algorithm’s capacity to accurately estimate the direct distance to the target further substantiated its high efficacy. These compelling findings serve as a substantial contribution to assistive robotics. The proficient visual tracking methodology proposed herein holds the potential to markedly amplify the competencies of robots operating in dynamic, unstructured indoor settings, and set the foundation for a higher degree of complex interactive tasks.

Can You Dance? A Study of Child–Robot Interaction and Emotional Response Using the NAO Robot

This retrospective study presents and summarizes our long-term efforts in the popularization of robotics, engineering, and artificial intelligence (STEM) using the NAO humanoid robot. By a conservative estimate, over a span of 8 years, we engaged at least a couple of thousand participants: approximately 70% were preschool children, 15% were elementary school students, and 15% were teenagers and adults. We describe several robot applications that were developed specifically for this task and assess their qualitative performance outside a controlled research setting, catering to various demographics, including those with special needs (ASD, ADHD). Five groups of applications are presented: (1) motor development activities and games, (2) children’s games, (3) theatrical performances, (4) artificial intelligence applications, and (5) data harvesting applications. Different cases of human–robot interactions are considered and evaluated according to our experience, and we discuss their weak points and potential improvements. We examine the response of the audience when confronted with a humanoid robot featuring intelligent behavior, such as conversational intelligence and emotion recognition. We consider the importance of the robot’s physical appearance, the emotional dynamics of human–robot engagement across age groups, the relevance of non-verbal cues, and analyze drawings crafted by preschool children both before and after their interaction with the NAO robot.

Designing Behaviors of Robots Based on the Artificial Emotion Expression Method in Human–Robot Interactions

How to express emotions through motion behaviors of robots (mainly for robotic arms) to achieve human–robot emotion interactions is the focus of this paper. An artificial emotion expression method that accords with human emotion that can deal with external stimuli and has the capability of emotion decision-making was proposed based on the motion behaviors of robot. Firstly, a three-dimensional emotion space was established based on the motion indexes (deviation coefficient, acceleration, and interval time). Then, an artificial emotion model, which was divided into three parts (the detection and processing of external events, the generation and modification of emotion response vectors, and the discretization of emotions) was established in the three-dimensional emotion space. Then emotion patterns (love, excited, happy, anxiety, hate) and emotion intensity were calculated based on the artificial emotion model in human–robot interaction experiments. Finally, the influence of motion behaviors of humanoid robot NAO on the emotion expression of experimenters was studied through human–robot emotion interaction experiments based on the emotion patterns and emotion intensity. The positive emotion patterns (love, excited, happy) and negative emotion patterns (anxiety, hate) of the experimenters were evaluated. The experimental results showed that the personalized emotion responses could be generated autonomously for external stimuli, and the change process of human emotions could be simulated effectively according to the established artificial emotion model. Furthermore, the experimenters could recognize the emotion patterns expressed by the robot according to the motion behaviors of the robot, and whether experimenters were familiar with robots did not influence the recognition of different emotion patterns.

Design, Calibration, and Control of Compliant Force-Sensing Gripping Pads for Humanoid Robots

Abstract This paper introduces a pair of low-cost, light-weight, and compliant force-sensing gripping pads used for manipulating box-like objects with smaller-sized humanoid robots. These pads measure normal gripping forces and center of pressure (CoP). A calibration method is developed to improve the CoP measurement accuracy. A hybrid force-alignment-position control framework is proposed to regulate the gripping forces and to ensure the surface alignment between the grippers and the object. Limit surface theory is incorporated as a contact friction modeling approach to determine the magnitude of gripping forces for slippage avoidance. The integrated hardware and software system is demonstrated with an NAO humanoid robot. Experiments show the effectiveness of the overall approach.

Voice cue perception and voice gender categorisation via a humanoid robot interface

Two voice cues, fundamental frequency (F0) and vocal-tract length (VTL), are important for characterising voice gender. Due to their often repetitive nature, psychophysical tests evaluating perception of these vocal cues can at times cause individuals to lose engagement during the test. To help with this, we propose the use of an interactive humanoid NAO robot as an alternative to the conventionally used laptop interface. As a first step, we compare the performance of two psychophysical tests between the robot and laptop interfaces. Experiment I measured F0 and VTL cue perception through an adaptive test in just noticeable differences (JNDs). Experiment II measured voice gender categorisation using F0 and VTL manipulated stimuli. The robot implementation made use of the tactile sensors on the hands and head of the robot for response logging. Performance accuracy between the computer and robot interfaces was functionally similar, confirming data reliability. Test duration comparison showed that both experiments were longer on the NAO robot than the laptop. Despite potential design limitations of the robot interface, both interfaces showed that the F0 and VTL JNDs are very small in normal-hearing listeners, and that normal-hearing listeners make effective use of F0 and VTL cues for voice gender categorisation.

Convolutional Neural Network Based Hand Gesture Recognition in Sophisticated Background for Humanoid Robot Control

Hand gesture recognition is a preferred way for human-robot interactions. Conventional approaches are generally based on image processing and recognition of hand poses with simple backgrounds. In this paper, we propose deep learning models, and humanoid robot integration for offline and online (real-time) recognition and control using hand gestures. One thousand and two hundred of hand images belonging to four participants are collected to construct the hand gesture database. Five class (forward, backward, right, left and stop) images in six sophisticated backgrounds with different illumination levels are obtained for four participants, and then one participant's images are kept as testing data. A lightweight Convolutional Neural Network (CNN), and transfer learning techniques using VGG16, and Mobilenetv2 are performed on this database to evaluate user independent performance of the hand gesture system. After offline training, real-time implementation is designed using a mobile phone (Wi-Fi and camera), Wi-Fi router, computer with embedded deep learning algorithms, and NAO humanoid robot. Streamed video by the mobile phone is processed and recognized using the proposed deep algorithm in the computer, and then command is transferred to robot via TCP/IP protocol. Thus, the NAO humanoid robot control using hand gesture in RGB and HSV color spaces is evaluated in sophisticated background, and the implementation of the system is presented. In our simulations, 95% and 100% accuracy rates are yielded for the lightweight CNN, and transfer learning, respectively.

A New Knowledge Primitive of Digits Recognition for NAO Robot Using MNIST Dataset and CNN Algorithm for Children’s Visual Learning Enhancement

Aim/Purpose: Our study is focused on prototyping, development, testing, and deployment of a new knowledge primitive for the humanoid robot assistant NAO, in order to enhance student visual learning by establishing a human-robot interaction. Background: This new primitive, utilizing a convolutional neural network (CNN), enables real-time recognition of handwritten digits captured by the NAO robot, a humanoid robot assistant developed by SoftBank Robotics. It is equipped with advanced capabilities, including a wide range of sensors, cameras, and interactive features. By integrating the proposed primitive, the NAO robot gains the ability to accurately recognize handwritten digits, contributing to improved student visual learning experiences. Methodology: Our developed primitive consists of the use of a convolutional neural network (CNN) so that the robot is able to recognize the handwriting of the digits present in the input image received in real-time. The NAO robot establishes interaction with the learners through a scenario based on a predefined assignment. In this scenario, NAO captures the digit handwritten by the learner via its camera, recognizes the digit using the deep learning model generated by the MNIST dataset, and announces to the learner the handwritten digit in the input image. The prototype is realized using the concept of a distributed system allowing the distribution of tasks in four different computing nodes. Contribution: Our research makes a significant contribution by equipping the humanoid robot NAO with a cognitive intelligence system through the integration of a new knowledge primitive based on handwriting digit recognition (HWDR). Our approach used to create and implement this primitive in the NAO robot is interesting and innovative, and presents a promising provision for enhancing the visual learning experience of children and young students with special needs, based on the use of distributed systems that divide the work using various components distributed over several nodes, coordinating their efforts to perform tasks more efficiently than a single device besides the NAO robot. Findings: We designed our model using specific parameters and a fully convolutional neural network architecture, which includes three residual depthwise separable convolutions, each followed by batch normalization and ReLU activation. To evaluate the performance of our model, we tested it on the MNIST dataset, where we achieved a remarkable accuracy, F1 score, and recall of 99%. An experiment was conducted to test our implemented primitive and see the effectiveness of this invention for enhancing visual learning in children with special needs. We developed a visual learning strategy based on the creation of engaging activities mediated by the NAO robot in an educational context. The results showed that participants achieved a strong commitment to the NAO robot, appreciating its ability to recognize handwritten digits and highlighting its promising potential to enrich visual learning experiences. Participants expressed a strong preference for teaching methods integrating assistive learning technologies, demonstrating the positive impact of our humanoid assistant robot on improving learning and visual intelligence in an educational environment. Recommendations for Practitioners: Encourage creativity and innovation in the field of robotics and special needs. This can lead to new and effective solutions that improve the lives of students with special needs. Recommendation for Researchers: Test and evaluate the proposed robotics solutions to ensure they are effective and making a positive impact. Use feedback from users, educators, and parents to refine and improve your solutions. Also, ensure that the robotics solutions are accessible to students with a range of abilities. This may involve designing solutions that are adjustable or providing alternative means of access. Impact on Society: As there are several ways to educate, there are multiple forms of learning. With the help of this learning procedure and strategy, the human teacher collaborates with the robot assistance NAO to improve visual learning among students. The findings of this research can serve as an application for the implementation of various pedagogical methods that will assist in meeting the needs of the majority of learners. Future Research: Our future research will concentrate on addressing the educational needs of students with special needs, enabling them to overcome their challenges and reach academic excellence in an inclusive environment. To achieve this goal, we plan to leverage the capabilities of social robots, which have emerged as a significant contributor to the field of human-robot interaction, particularly in facilitating inclusive education. These agents have proven to be effective in providing support to students with special needs, thereby enabling them to receive the education they need to succeed.

A brain-inspired intention prediction model and its applications to humanoid robot.

With the development of artificial intelligence and robotic technology in recent years, robots are gradually integrated into human daily life. Most of the human-robot interaction technologies currently applied to home service robots are programmed by the manufacturer first, and then instruct the user to trigger the implementation through voice commands or gesture commands. Although these methods are simple and effective, they lack some flexibility, especially when the programming program is contrary to user habits, which will lead to a significant decline in user experience satisfaction. To make that robots can better serve human beings, adaptable, simple, and flexible human-robot interaction technology is essential. Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction model to enable the robot to perform actions according to the user's intention. With the spike-timing-dependent plasticity (STDP) mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by (N 2 - N)/4, where N is the number of intentions.

NAO robots as context to teach numerical methods

This article examines the contextual role that NAO humanoid robots play in the learning environment of a numerical methods course through a design-based research approach. Under the design of intentional and reflective planning that emanates from the socio-critical paradigm in education, the feasibility of using this type of robot in the classroom as a motivation element to engage students in understanding numerical analysis issues is addressed. In particular, the active learning of engineering students is achieved by studying such robots' forward and inverse kinematics. The didactical sequence to teach the generalized Newton–Raphson method for solving non-linear equation systems is validated as part of a Hypothetical Learning Trajectory proposed. Besides, this research provides a brief theoretical discussion on reflection-in-action of the teacher of Higher Education level.

Efficacy of a Robot-Assisted Intervention in Improving Learning Performance of Elementary School Children with Specific Learning Disorders.

(1) Background: There has been significant recent interest in the potential role of social robots (SRs) in special education. Specific Learning Disorders (SpLDs) have a high prevalence in the student population, and early intervention with personalized special educational programs is crucial for optimal academic achievement. (2) Methods: We designed an intense special education intervention for children in the third and fourth years of elementary school with a diagnosis of a SpLD. Following confirmation of eligibility and informed consent, the participants were prospectively and randomly allocated to two groups: (a) the SR group, for which the intervention was delivered by the humanoid robot NAO with the assistance of a special education teacher and (b) the control group, for which the intervention was delivered by the special educator. All participants underwent pre- and post-intervention evaluation for outcome measures. (3) Results: 40 children (NAO = 19, control = 21, similar baseline characteristics) were included. Pre- and post-intervention evaluation showed comparable improvements in both groups in cognition skills (decoding, phonological awareness and reading comprehension), while between-group changes favored the NAO group only for some phonological awareness exercises. In total, no significant changes were found in any of the groups regarding the emotional/behavioral secondary outcomes. (4) Conclusion: NAO was efficient as a tutor for a human-supported intervention when compared to the gold-standard intervention for elementary school students with SpLDs.

Could NAO Robot Function as Model Demonstrating Joint Attention Skills for Children with Autism Spectrum Disorder? An Exploratory Study

Previous studies reported that children with autism spectrum disorder (ASD) show a certain interest in social robots. This makes social robots potential to be a model to teach social skills. This exploratory study aims to investigate whether three types of joint attention skills (i.e., eye-contact, pointing, gaze-following) could be improved for five preschoolers with ASD using an evidence-based robot-modeling intervention with a humanoid social robot NAO. Our observation shows that these children were motivated when interacting with NAO by following and responding correctly to NAO’s joint attention behaviors. Although some improvements were found, no pattern or systematic effect could be revealed. In the future, more evidence-based studies are needed to investigate the benefits of robot-assisted therapy more deeply.

"Sequencing Matters": Investigating Suitable Action Sequences in Robot-Assisted Autism Therapy.

Social robots have been shown to be promising tools for delivering therapeutic tasks for children with Autism Spectrum Disorder (ASD). However, their efficacy is currently limited by a lack of flexibility of the robot’s social behavior to successfully meet therapeutic and interaction goals. Robot-assisted interventions are often based on structured tasks where the robot sequentially guides the child towards the task goal. Motivated by a need for personalization to accommodate a diverse set of children profiles, this paper investigates the effect of different robot action sequences in structured socially interactive tasks targeting attention skills in children with different ASD profiles. Based on an autism diagnostic tool, we devised a robotic prompting scheme on a NAO humanoid robot, aimed at eliciting goal behaviors from the child, and integrated it in a novel interactive storytelling scenario involving screens. We programmed the robot to operate in three different modes: diagnostic-inspired (Assess), personalized therapy-inspired (Therapy), and random (Explore). Our exploratory study with 11 young children with ASD highlights the usefulness and limitations of each mode according to different possible interaction goals, and paves the way towards more complex methods for balancing short-term and long-term goals in personalized robot-assisted therapy.