NAO Humanoid Robot Research Articles

Sustaining multi-partner engaged research: principles, processes and tensions

ABSTRACT This paper presents a reflective analysis of a multi-partner, transdisciplinary research project where Nao humanoid robots were integrated into children’s learning experiences at a community school. Using critical reflection as a method of inquiry principles of engaged research are conceptualised showing how they were articulated throughout the study, including how they were represented in the issues that characterised the project. A process of content analysis drawn from critical incidents told via a story-telling strategy revealed that the supportive and restrictive narratives that shaped our experience were embedded around the core principles of meeting community needs, reciprocity and boundary crossing. The complex and intersecting engagement processes necessitated skills in managing human and human-system relationships and interdependencies, temporal and spatial connections, social ecologies, ethical and practical decisions, and knowledge and inquiry tensions. The study concluded that securing institutional commitments and streamlining operational procedures are critical conditions for advancing complex, multi-partner engaged research. Additionally, guidelines to foster the development of novice educational researchers (e.g. higher degrees research students) as engaged scholars are outlined.

Synthesis of a hybrid brain for a humanoid robot

This article comprehends the design of a Brain-Based Robot (BBR) using hybrid techniques that incorporate both Brain-Based Device (BBD) and computational algorithms. BBDs are biologically inspired machines which have its behavior guided by a simulated nervous system. This nervous system follows detailed neuroanatomy of different brain areas. BBDs tend to have a nervous system with a large number of neurons and synapses. Thus, a huge computational power is required to simulate the nervous system of a BBD. Nevertheless, some of the tasks carried out by the simulated nervous system can be accomplished using computational algorithms which can help reducing the required computational power greatly. In this article, a BBR is built which combines some subsystems from BBD with computer vision algorithms. Computer vision algorithms are applied using OpenCV to extract some features from images, while neuronal-areas are connected together based on a detailed neuroanatomical structure to mimic the human learning process. Nengo python package is used for simulating neuronal areas in the system and monitoring activities of neuronal units. Moreover, the successful integration of the BBD’s subsystems with computer vision leads to the perceptual categorization based on invariant object-recognition of various visual cues. To make a fair comparison with BBD, the nervous system of a BBD is built on the same computer used to build the hybrid brain for the proposed BBR. The proposed hybrid brain is then applied to a Nao humanoid robot in V-REP simulation environment to test it. The results obtained through this article prove that the proposed hybrid brain possesses the same intelligence of the BBD and requires much less computational power that it can run on an on-board computer of a robot, which makes it plausible for engineering applications.

Design and validation of two embodied mirroring setups for interactive games with autistic children using the NAO humanoid robot.

Socially assistive robots have shown potential benefits in therapy of child and elderly patients with social and cognitive deficits. In particular, for autistic children, humanoid robots could enhance engagement and attention, thanks to their simplified toy-like appearance and the reduced set of possible movements and expressions. The recent focus on autism-related motor impairments has increased the interest on developing new robotic tools aimed at improving not only the social capabilities but also the motor skills of autistic children. To this purpose, we have designed two embodied mirroring setups using the NAO humanoid robot. Two different tracking systems were used and compared: Inertial Measurement Units and the Microsoft Kinect, a marker-less vision based system. Both platforms were able to mirror upper limb basic movements of two healthy subjects, an adult and a child. However, despite the lower accuracy, the Kinect-based setup was chosen as the best candidate for embodied mirroring in autism treatment, thanks to the lower intrusiveness and reduced setup time. A prototype of an interactive mirroring game was developed and successfully tested with the Kinect-based platform, paving the way to the development of a versatile and powerful tool for clinical use with autistic children.

Desarrollo de herramientas para favorecer el aprendizaje del inglés

The objective of this work is the creation of tools for learning the English language using augmented reality and humanoid robots, in order to interact with people autonomously in the learning process. For this, the contents of the English language programs and programs at the higher level were reviewed, language area heads and teachers were interviewed, in order to finish cable issues in language learning. At present, having a second language in the field of education today is considered a necessity. Currently, higher level institutions use English language learning techniques based on books, flash-cards, slides, audios, videos, etc. From the above it is proposed to develop learning tools capable of interacting with users autonomously. This leads to solving major learning problems using human computer and robotic human interaction. What is presented are two tools for learning the English language, the first based on augmented reality and the second using Nao humanoid robot.

An Adaptive Augmented Vision-Based Ellipsoidal SLAM for Indoor Environments.

In this paper, the problem of Simultaneous Localization And Mapping (SLAM) is addressed via a novel augmented landmark vision-based ellipsoidal SLAM. The algorithm is implemented on a NAO humanoid robot and is tested in an indoor environment. The main feature of the system is the implementation of SLAM with a monocular vision system. Distinguished landmarks referred to as NAOmarks are employed to localize the robot via its monocular vision system. We henceforth introduce the notion of robotic augmented reality (RAR) and present a monocular Extended Kalman Filter (EKF)/ellipsoidal SLAM in order to improve the performance and alleviate the computational effort, to provide landmark identification, and to simplify the data association problem. The proposed SLAM algorithm is implemented in real-time to further calibrate the ellipsoidal SLAM parameters, noise bounding, and to improve its overall accuracy. The augmented EKF/ellipsoidal SLAM algorithms are compared with the regular EKF/ellipsoidal SLAM methods and the merits of each algorithm is also discussed in the paper. The real-time experimental and simulation studies suggest that the adaptive augmented ellipsoidal SLAM is more accurate than the conventional EKF/ellipsoidal SLAMs.

Multi‐target detection and grasping control for humanoid robot NAO

SummaryGraspirng objects is an important capability for humanoid robots. Due to complexity of environmental and diversity of objects, it is difficult for the robot to accurately recognize and grasp multiple objects. In response to this problem, we propose a robotic grasping method that uses the deep learning method You Only Look Once v3 for multi‐target detection and the auxiliary signs to obtain target location. The method can control the movement of the robot and plan the grasping trajectory based on visual feedback information. It is verified by experiments that this method can make the humanoid robot NAO grasp the object effectively, and the success rate of grasping can reach 80% in the experimental environment.

A Socially Assistive Robotic Platform for Upper-Limb Rehabilitation: A Longitudinal Study With Pediatric Patients

In motor rehabilitation therapies, especially with pediatric patients, there is often a loss of interest and motivation due to the lack of variety in exercises and therapeutic activities. NAOTherapist (NT) is a socially assistive robotics (SAR) platform designed to support hands-off rehabilitation sessions through two interactive games: mirror and Simon. It integrates a cognitive architecture with an NAO humanoid robot that acts as a coach, as well as a red-green-blue-depth (RGB-D) sensor that eases body capture. This work aims to deploy and evaluate the developed technology for a long-term experience.

NAO robot fuzzy obstacle avoidance in virtual environment

The fuzzy inference system for obstacle avoidance developed in this paper is designed for NAO humanoid robot. The fuzzy obstacle avoidance (Fuzzy OA) has been tested in Webots virtual environment and the results showed that this method is almost two times faster than the Naoqi framework obstacle avoidance (Naoqi OA) while the robot is much more stable. Because the fuzzy inference system is a method that relies on trial an error and experience, the obstacle avoidance algorithm is subject to improvements. Future developments will take into account these results and will add other fuzzy inference systems for navigation, in order to get more autonomy for Nao robot.

Target recognition of indoor trolley for humanoid robot based on piecewise fitting method

SummaryAlthough several studies have considered the problem of humanoid robots pushing carts, only a few have focused on the problem of robots moving heavy objects under monocular vision. This study proposes a target recognition and positioning method and a control method for a robot pushing a loaded trolley. A control system based on the humanoid robot NAO is developed and a monocular visual ranging method with segmented fitting is proposed to realize hardware control and target search and positioning for NAO. The ability of NAO to push a small cart with various weights using visual positioning is tested. The experimental results show that the average error of the monocular distance measurement method is 1.7 mm and that the target search and positioning is accurate. NAO can push a loaded cart that is 6.5 times its own weight.

Using socially assistive robots for monitoring and preventing frailty among older adults: a study on usability and user experience challenges

Socially assistive robots can play an important role in the monitoring and training of health of older adults. But before their benefits can be reaped, proper usability and a positive user experience need to be ensured. In this study, we tested the usability and user experience of a socially assistive robot (the NAO humanoid robot) to monitor and train the health of frail older adults. They were asked to complete a set of health monitoring and physical training tasks, once provided by the NAO robot, and once provided by a Tablet PC application (as a reference technology). After using each technology, they completed the System Usability Scale for usability, and a set of rating scales for perceived usefulness, enjoyment, and control. Finally, we questioned the participants’ preference for one of the technologies. All interactions were recorded on video and scrutinized for usability issues. Twenty older adults participated. They awarded both technologies ‘average’ usability scores. Perceived usefulness and enjoyment were rated as very positive for both modalities; control was scored positively. Main usability issues for NAO for these tasks were related to speech interaction (e.g., NAO’s limited speech library, NAO’s difficulty to cope with Dutch dialect), older adults’ difficulties with taking their proper role in human-robot interaction, and a lack of affordances of NAO. Seven participants preferred NAO: it was easier to use and more personal. Social robots have the potential to monitor and train the health of frail older adults, but some critical usability challenges need to be overcome first.

Desarrollo de una aplicación en el robot NAO H25 para la enseñanza y comprobación de la operación multiplicación con el método Maya

This research document shows the development of an application implemented in the humanoid robot NAO H25, using the methodology of Analysis, Design, Development, Implementation and Evaluation (ADDIE). Currently, in most public institutions of basic education, there is no robotic technology that the teacher uses for teaching a particular subject. The purpose of the application is that it works as a didactic resource, with which teachers and students of basic education can use as a form of support for the teaching of the multiplication operation. This application is capable of making the NAO H25 robot explain the multiplication operation in an arithmetic way and carry out its verification by means of the Mayan method. When the application is implemented in the NAO H25 robot, it acts as a trainer in the topic of multiplications, following a didactic sequence as a form of support for the development of the explanation. For the moment, tests were carried out on a group of 32 students of the computer engineering degree at this university centre. Where it was found that 94% were interested in the robot acting as a capacitor. 97% would like more applications of this type to be made for learning other subjects. The development of the application is with the motive of obtaining a significant learning in the students of third year of basic education, fomenting the interest by the technology.

The Humanoid Robot NAO as Trainer in a Memory Program for Elderly People with Mild Cognitive Impairment

Many studies on social interaction have used the humanoid robot NAO. In the present paper, we described our project designed to address the growing unmet need for alternative approaches to slowing the progression of cognitive decline in Mild Cognitive Impairment patients. NAO is the experimental platform used in an ecological setting: a center for the treatment of cognitive disorders and dementia of the Italian health service. This paper describes the study addressed to evaluate the effectiveness of human–robot interaction to reinforce therapeutic behavior and treatments adherence and presents the latest findings of functional tests and users investigation recently conducted. The robot was programmed to implement some tasks from the usual memory-training program protocol. In different training conditions, subjects participated in sessions with the support of NAO or only from the psychologist while the interaction was recorded for subsequent exploration. Data indicated that memory training with NAO resulted in an increase of visual gaze from patients and reinforce of therapeutic behavior reducing, in some cases, depressive symptoms. Unexpectedly, significant changes in prose memory and verbal fluency measures were detected. These findings suggest that further research on robotics in ecological settings is necessary to determine the extent to which they can effectively support clinical practice.

Control framework for cooperative object transportation by two humanoid robots

This paper aims at proposing a comprehensive control framework designed for cooperative transportation of a heavy load by two humanoid robots. First, a simplified dynamic model of the cooperative task is developed and the system stability is rigorously analyzed. Next, a centralized controller is formulated, this formulation provides an optimal control of the system by considering the robots dynamical stability while satisfying the robot–object–robot constraints. Finally, the controller is integrated with an arm controller and a local planner module forming a complete framework for cooperative transportation tasks. The approach is thoroughly analyzed and validated in simulation, and experiments are carried out on a team of two Nao humanoid robots transporting a range of objects placed on a small table. The experimental results pointed out the robustness of the approach as the robots successfully accomplished the transportation tasks in a stable way, moreover the transported objects masses were up to half the mass of one of the robot. Besides increasing the robot payload, some of the transported objects are relatively large and it is simply impossible for a single robot to transport them.

A Brain-Robot Interaction System by Fusing Human and Machine Intelligence.

This paper presents a new brain-robot interaction system by fusing human and machine intelligence to improve the real-time control performance. This system consists of a hybrid P300 and steady-state visual evoked potential (SSVEP) mode conveying a human being's intention, and the machine intelligence combining a fuzzy-logic-based image processing algorithm with multi-sensor fusion technology. A subject selects an object of interest via P300, and the classification algorithm transfers the corresponding parameters to an improved fuzzy color extractor for object extraction. A central vision tracking strategy automatically guides the NAO humanoid robot to the destination selected by the subject intentions represented by brainwaves. During this process, human supervises the system at high level, while machine intelligence assists the robot in accomplishing tasks by analyzing image feeding back from the camera, distance monitoring using out-of-gauge alarms from sonars, and collision detecting from bumper sensors. In this scenario, the SSVEP takes over the situations in which the machine intelligence cannot make decisions. The experimental results show that the subjects can control the robot to a destination of interest, with fewer commands than only using a brain-robot interface. Therefore, the fusion of human and machine intelligence greatly alleviates the brain load and enhances the robot executive efficiency of a brain-robot interaction system.

Developing a Robot-Guided Interactive Simon Game for Physical and Cognitive Training

Enveloping cognitive or physical rehabilitation into a game highly increases the patients’ commitment with their treatment. Specially with children, keeping them motivated is a very time-consuming work, so therapists are demanding tools to help them with this task. NAOTherapist is a generic robotic architecture that uses Automated Planning techniques to autonomously drive noncontact upper-limb rehabilitation sessions for children with a humanoid NAO robot. Our aim is to develop more robotic games for this platform to enrich its variability and possibilities of interaction. The goal of this work is to present our first attempt to develop a different, more complex game that reuses the previous architecture. We contribute with the design description of a novel robotic Simon game that employs upper-limb poses instead of colors and could qualify as a cognitive and physical training. Statistics of evaluation tests with 14 adults and 56 children are displayed and the outcomes are analyzed in terms of human–robot interaction (HRI) quality. The results demonstrate the application-domain generalization capabilities of the NAOTherapist architecture and give an insight to further analyze the therapeutic benefits of the new developed Simon game.

An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC

The ability to classify rooms in a home is one of many attributes that are desired for social robots. In this paper, we address the problem of indoor room classification via several convolutional neural network (CNN) architectures, i.e., VGG16, VGG19, & Inception V3. The main objective is to recognize five indoor classes (bathroom, bedroom, dining room, kitchen, and living room) from a Places dataset. We considered 11600 images per class and subsequently fine-tuned the networks. The simulation studies suggest that cleaning the disparate data produced much better results in all the examined CNN architectures. We report that VGG16 & VGG19 fine-tuned models with training on all layers produced the best validation accuracy, with 93.29% and 93.61% on clean data, respectively. We also propose and examine a combination model of CNN and a multi-binary classifier referred to as error correcting output code (ECOC) with the clean data. The highest validation accuracy of 15 binary classifiers reached up to 98.5%, where the average of all classifiers was 95.37%. CNN and CNN-ECOC, and an alternative form called CNN-ECOC Regression, were evaluated in real-time implementation on a NAO humanoid robot. The results show the superiority of the combination model of CNN and ECOC over the conventional CNN. The implications and the challenges of real-time experiments are also discussed in the paper.

Control of a Humanoid NAO Robot by an Adaptive Bioinspired Cerebellar Module in 3D Motion Tasks.

A bioinspired adaptive model, developed by means of a spiking neural network made of thousands of artificial neurons, has been leveraged to control a humanoid NAO robot in real time. The learning properties of the system have been challenged in a classic cerebellum-driven paradigm, a perturbed upper limb reaching protocol. The neurophysiological principles used to develop the model succeeded in driving an adaptive motor control protocol with baseline, acquisition, and extinction phases. The spiking neural network model showed learning behaviours similar to the ones experimentally measured with human subjects in the same task in the acquisition phase, while resorted to other strategies in the extinction phase. The model processed in real-time external inputs, encoded as spikes, and the generated spiking activity of its output neurons was decoded, in order to provide the proper correction on the motor actuators. Three bidirectional long-term plasticity rules have been embedded for different connections and with different time scales. The plasticities shaped the firing activity of the output layer neurons of the network. In the perturbed upper limb reaching protocol, the neurorobot successfully learned how to compensate for the external perturbation generating an appropriate correction. Therefore, the spiking cerebellar model was able to reproduce in the robotic platform how biological systems deal with external sources of error, in both ideal and real (noisy) environments.

ParlAmI: A Multimodal Approach for Programming Intelligent Environments

The proliferation of Internet of Things (IoT) devices and services and their integration in intelligent environments creates the need for a simple yet effective way of controlling and communicating with them. Towards such a direction, this work presents ParlAmI, a conversational framework featuring a multimodal chatbot that permits users to create simple “if-then” rules to define the behavior of an intelligent environment. ParlAmI delivers a disembodied conversational agent in the form of a messaging application named MAI, and an embodied conversational agent named nAoMI employing the programmable humanoid robot NAO. This paper describes the requirements and architecture of ParlAmI, the infrastructure of the “Intelligent Home” in which ParlAmI is deployed, the characteristics and functionality of both MAI and nAoMI, and finally presents the findings of a user experience evaluation that was conducted with the participation of sixteen users.

INTERPRETATION OF HUMAN AND ROBOT EMBLEMATIC GESTURES: HOW DO THEY DIFFER?

Human gestures are often directly adopted in gesture design for humanoid robots when human-robot interaction is considered, but the interpretations of human and robot gestures can vary due to the structural difference between human and robot and due to people's perceptual difference of human and robot. In this paper, we investigate how people interpret human and robot gestures differently using emblematic gestures, a type of gesture with specific verbal translations. Nine commonly used emblems performed by a human were evaluated by 32 participants, and the nine emblems performed by a NAO humanoid robot were evaluated by another 32 participants. We compared participants' interpretations of the human and robot emblems systematically. The results indicate that the multimessage gesture and multigesture message characteristics of human emblems also exist in robot emblems. However, participants' recognition rates of robot emblems are lower than those of human emblems in many cases. The human and robot emblems can also vary in interpretations and corresponding percentages. Some interpretations of human emblems do not appear in robot emblems and vice versa; even when the interpretations of human and robot emblems are consistent, the corresponding percentage of each interpretation can vary between human and robot. Besides, participants' recognition of robot emblems can be affected by their familiarity with humanoids.

Q-learning trajectory planning based on Takagi–Sugeno fuzzy parallel distributed compensation structure of humanoid manipulator

NAO is the first robot created by SoftBank Robotics. Famous around the world, NAO is a tremendous programming tool and he has especially become a standard in education and research. Aiming at the large error and poor stability of the humanoid robot NAO manipulator during trajectory tracking, a novel framework based on fuzzy controller reinforcement learning trajectory planning strategy is proposed. Firstly, the Takagi–Sugeno fuzzy model based on the dynamic equation of the NAO right arm is established. Secondly, the design and the gain solution of the state feedback controller based on the parallel feedback compensation strategy are studied. Finally, the ideal trajectory of the motion is planned by reinforcement learning algorithm so that the end of the manipulator can track the desired trajectory and realize the valid obstacle avoidance. Simulation and experiment shows that the end of the manipulator based on this scheme has good controllability and stability and can meet the accuracy requirements of trajectory tracking accuracy, which verifies the effectiveness of the proposed framework.