Robot Dance Research Articles

The Design and Usability of Virtual Reality Content Using Eyes for Students with Severe Disabilities

The purposes of this study were to develop the design of virtual reality content using eyes and to investigate the usability of the virtual reality content for nonverbal students with severe disabilities. A 360-degree video in which a narrator introduces scientific activities (momentum, earthquake, gravity, robot dance) and experiences them at Chungnam Science Education Center was produced. Next, the virtual reality content for the video were designed for users to manipulate image buttons with their eyes so that users can select and experience images. Users can select and view images of scientific activity by staring and blinking using the HTC-VIVE Pro Eye, which combines eye tracking technology. The virtual reality content using eyes were designed and developed in accordance with the guidelines for supporting self-determination of students with severe disabilities. In order to investigate the usability of the VR content, 37 special education teachers who had special teacher certificates and had experience in teaching students with severe disabilities answered questions after using the VR content. The teachers evaluated as "neutral" in some usability evaluation categories (e.g., Self-determination, Ease of manipulation, Ease of Understanding, Effectiveness, Efficiency of class preparation, Efficiency in teaching, Appropriateness of content organization). When developing VR contents for students with severe disabilities, developers can design visual images in order for students with severe disabilities to understand easily and establish user interface in users’ eye conditions and operational capabilities. VR contents can be designed in relation to subject areas so that teachers can use them as educational materials.

Influence of social robots and digital media based on edge computing on dance performance entertainment environment

This article proposes a new social robot system that achieves robot dance creation through dance image feature extraction and dance action imitation, thereby promoting the development of digital media dance entertainment culture. This paper uses the edge computing model, and on the basis of the robot’s own computing power, puts some computing tasks related to image processing and action imitation down to the edge of the network to improve the real-time and fluency of dance performance. The study utilized image feature extraction technology to analyze and process dance images, achieving the imitation of dance movements by robots. By imitating and innovating dance movements through robots, unique forms and content can be provided for digital media dance performances. Digital media technology provides rich means and forms of expression, which can make robot dance performances more creative and artistic.

Modeling Robotic Thinking and Creativity: A Classic–Quantum Dialogue

The human mind can be thought of as a black box, where the external inputs are elaborated in an unknown way and lead to external outputs. D’Ariano and Faggin schematized thinking and consciousness through quantum state dynamics. The complexity of mental states can be formalized through the entanglement of the so-called qualia states. Thus, the interaction between the mind and the external world can be formalized as an interplay between classical and quantum-state dynamics. Since quantum computing is more and more often being applied to robots, and robots constitute a benchmark to test schematic models of behavior, we propose a case study with a robotic dance, where the thinking and moving mechanisms are modeled according to quantum–classic decision making. In our research, to model the elaboration of multi-sensory stimuli and the following decision making in terms of movement response, we adopt the D’Ariano–Faggin formalism and propose a case study with improvised dance based on a collection of poses, whose combination is presented in response to external and periodic multi-sensory stimuli. We model the dancer’s inner state and reaction to classic stimuli through a quantum circuit. We present our preliminary results, discussing further lines of development.

An adaptive control framework based multi-modal information-driven dance composition model for musical robots.

Currently, most robot dances are pre-compiled, the requirement of manual adjustment of relevant parameters and meta-action to change the dancing to another type of music would greatly reduce its function. To overcome the gap, this study proposed a dance composition model for mobile robots based on multimodal information. The model consists of three parts. (1) Extraction of multimodal information. The temporal structure feature method of structure analysis framework is used to divide audio music files into music structures; then, a hierarchical emotion detection framework is used to extract information (rhythm, emotion, tension, etc.) for each segmented music structure; calculating the safety of the current car and surrounding objects in motion; finally, extracting the stage color of the robot's location, corresponding to the relevant atmosphere emotions. (2) Initialize the dance library. Dance composition is divided into four categories based on the classification of music emotions; in addition, each type of dance composition is divided into skilled composition and general dance composition. (3) The total path length can be obtained by combining multimodal information based on different emotions, initial speeds, and music structure periods; then, target point planning can be carried out based on the specific dance composition selected. An adaptive control framework based on the Cerebellar Model Articulation Controller (CMAC) and compensation controllers is used to track the target point trajectory, and finally, the selected dance composition is formed. Mobile robot dance composition provides a new method and concept for humanoid robot dance composition.

Center of Pressure Control for Balancing Humanoid Dance Robot Using Load Cell Sensor, Kalman Filter and PID Controller

Balance control on the Lanage Jagad humanoid dance robot is one of the means to create flexible dance movements in the robot movement system to make it more stable and can reduce the frequency of the robot falling or being unable to maintain balance when performing the dance. For the position of the robot, it can use a weight sensor or load cell sensor, the sensor measures the resistance value that can control the weight of 4 weight points on each robot leg which will later be converted into a pressure value at each point, in the study. This test was carried out with the same control behavior using an inertial sensor MPU6050. The balance on the robot uses a balance based on a load cell, which is a situation where the position of the robot in coordinates approaches the center of balance or CoP (Center of Pressure) at coordinates (0,0) or if using MPU6050 it is in a far error value condition so that it can balance the conditions so as not to falls close to the value of the robot state based on ZMP (Zero Moment Point) and CoG (Central of Gravity) as the MPU6050 sensor placement. In this study, for the balance control system using the Arduino MEGA 2560 PRO Board as a complement to the OpenCM 9.04 microcontroller, using 8 load cell sensors to determine the balance point which has been made predictions of pressure from the load cell using a kalman filter also PID control to handle the servo motor. The results from the center point of the robot's pressure have succeeded in determining the center of balance or CoP based on the X coordinates of 0 and the Y coordinates of 0 and the quadrant direction based on the center of gravity, so that the results of the balance system in standing and dancing conditions are based on the center of balance using a load cell, the success rate when standing by 87.5% and balance when dancing by 89%.

Automatic Aesthetics Evaluation of Robotic Dance Poses Based on Hierarchical Processing Network.

Vision plays an important role in the aesthetic cognition of human beings. When creating dance choreography, human dancers, who always observe their own dance poses in a mirror, understand the aesthetics of those poses and aim to improve their dancing performance. In order to develop artificial intelligence, a robot should establish a similar mechanism to imitate the above human dance behaviour. Inspired by this, this paper designs a way for a robot to visually perceive its own dance poses and constructs a novel dataset of dance poses based on real NAO robots. On this basis, this paper proposes a hierarchical processing network-based approach to automatic aesthetics evaluation of robotic dance poses. The hierarchical processing network first extracts the primary visual features by using three parallel CNNs, then uses a synthesis CNN to achieve high-level association and comprehensive processing on the basis of multi-modal feature fusion, and finally makes an automatic aesthetics decision. Notably, the design of this hierarchical processing network is inspired by the research findings in neuroaesthetics. Experimental results show that our approach can achieve a high correct ratio of aesthetic evaluation at 82.3%, which is superior to the existing methods.

Optimization of Humanoid Robot Leg Movement Using Open CM 9.04

The Indonesian Robot Dance Contest (KRSTI) is a branch of the Indonesian Robot Contest (KRI) with the theme of dance. The robot used is a humanoid robot that can dance. Every year at the event, the provisions for robots constantly change, both the type of dance being demonstrated and the requirements for the robot's height. The taller the robot, the more difficult it is to control its walking movements because of the load it carries. This study uses a suitable algorithm to make the walking movement more natural and minimize the robot's falling. Human ROM data is used as a parameter for the range of motion of the servos that act as joints in the robot's legs. The algorithm created serves to determine the initial position of the angle on the servo to avoid the wrong initial movement position between one servo and another. The robot used is the Bioloid Robot’s leg Type A and uses OpenCM 9.04-A as the controller. The results showed that ROM on human feet could not be fully implemented on robot legs due to the robot's structure and the need for a robot that only relies on an algorithm to find the correct fulcrum to maintain balance. The comparison results show that the movement when walking on the ankle (ID servo 15) ranges from 749-567, while the ROM range is only between 580-512. When walking (servo ID 16), movement ranges from 460-291, while the ROM range ranges from 580-512.

Recognition of musical beat and style and applications in interactive humanoid robot.

The musical beat and style recognition have high application value in music information retrieval. However, the traditional methods mostly use a convolutional neural network (CNN) as the backbone and have poor performance. Accordingly, the present work chooses a recurrent neural network (RNN) in deep learning (DL) to identify musical beats and styles. The proposed model is applied to an interactive humanoid robot. First, DL-based musical beat and style recognition technologies are studied. On this basis, a note beat recognition method combining attention mechanism (AM) and independent RNN (IndRNN) [AM-IndRNN] is proposed. The AM-IndRNN can effectively avoid gradient vanishing and gradient exploding. Second, the audio music files are divided into multiple styles using the music signal's temporal features. A human dancing robot using a multimodal drive is constructed. Finally, the proposed method is tested. The results show that the proposed AM-IndRNN outperforms multiple parallel long short-term memory (LSTM) models and IndRNN in recognition accuracy (88.9%) and loss rate (0.0748). Therefore, the AM-optimized LSTM model has gained a higher recognition accuracy. The research results provide specific ideas for applying DL technology in musical beat and style recognition.

Automatic aesthetics assessment of robotic dance motions

Human dancers can understand and judge the aesthetics of their own dance motions from their movement perception. Inspired by this, we propose a novel mechanism of automatic aesthetics assessment of robotic dance motions, which is based on ensemble learning aimed at developing the autonomous judgment ability of robots. In the proposed mechanism, key pose descriptors based higher-order clustering features are designed to characterize robotic dance motion. Then, an ensemble classifier is built to train a machine aesthetics model for the automatic aesthetics assessment on robotic dance motions. The proposed mechanism has been implemented on a simulated robot environment, and experimental results show its feasibility and good performance.

Robot Dance: A mathematical optimization platform for intervention against COVID-19 in a complex network

Robot Dance is a computational optimization platform developed in response to the COVID-19 outbreak, to support the decision-making on public policies at a regional level. The tool is suitable for understanding and suggesting levels of intervention needed to contain the spread of infectious diseases when the mobility of inhabitants through a regional network is a concern. Such is the case for the SARS-CoV-2 virus that is highly contagious and, therefore, makes it crucial to incorporate the circulation of people in the epidemiological compartmental models. Robot Dance anticipates the spread of an epidemic in a complex regional network, helping to identify fragile links where applying differentiated measures of containment, testing, and vaccination is important. Based on stochastic optimization, the model determines efficient strategies on the basis of commuting of individuals and the situation of hospitals in each district. Uncertainty in the capacity of intensive care beds is handled by a chance-constraint approach. Some functionalities of Robot Dance are illustrated in the state of São Paulo in Brazil, using real data for a region with more than forty million inhabitants.

Whole‐Body Multicontact Haptic Human–Humanoid Interaction Based on Leader–Follower Switching: A Robot Dance of the “Box Step”

For physical human–robot interaction (pHRI) where multi‐contacts play a key role, both robustness to achieve robot‐intended motion and adaptability to follow human‐intended motion are fundamental. However, there are tradeoffs during pHRI when their intentions do not match. This paper focuses on bipedal walking control during pHRI, which handles such tradeoff when a human and a humanoid robot having different footsteps locations and durations. To resolve this, a force‐reactive walking controller is proposed by adequately combining ankle and stepping strategies. The ankle strategy maintains the robot's intention based on an analytically‐optimal center of pressure, leading the robot to oppose resistance to multiple contacts from the human. Based on the robot's kinodynamic constraints and/or the confidence of the robot's intention, the stepping strategy updates the robot's footsteps based on the human's intention implied by the multiple contact forces. Consequently, the proposed walking control on pHRI mutually exchanges human–robot intentions in real‐time, thereby achieving coordinated steps. With a full‐sized humanoid robot that is able to detect multi‐contacts in real‐time, we succeeded in performing a long‐term “box‐step” with multi‐contacts pHRI, demonstrating the robustness of our approach.

Design and implementation of computer aided resource management system for dance teaching

Traditional teaching methods are limited to time and place, and the performance of dance teaching resources management is poor. Design a computer-assisted dance teaching resource management system. The functional structure of the system includes core computer-assisted teaching and teaching management applications. The data management module is used to store the processed data in data files, and the dance teaching content release module retrieves requests and multimedia. The remote image resource location request of the management module responds to the feedback. In order to improve the management of computer-aided dance teaching resources, this article takes dance robots as the research object, takes dance video information as input, uses deep learning methods to estimate the human body posture in the video, and obtains the key point position coordinates of the human body; The inverse kinematics calculation of the robot obtains the angle values of each joint of the robot, and the angle values of the lower body joints are adjusted to maintain the balance of the robot. In addition, this paper also proposes a method to automatically generate robot dance sequence. Gated cyclic unit (GRU) network is used to learn the correlation between the global characteristics of music and dance gesture relationship characteristics, the correlation between music local characteristics and dance movement density characteristics, and then combine the dance movement graphs to sample and plan Robot dance moves synchronized with the beat. Experimental results show that whether it is robot dance movement imitation or dance movement generation, it can improve the computer-aided management of dance teaching.

Multiple Visual Feature Integration Based Automatic Aesthetics Evaluation of Robotic Dance Motions

Imitation of human behaviors is one of the effective ways to develop artificial intelligence. Human dancers, standing in front of a mirror, always achieve autonomous aesthetics evaluation on their own dance motions, which are observed from the mirror. Meanwhile, in the visual aesthetics cognition of human brains, space and shape are two important visual elements perceived from motions. Inspired by the above facts, this paper proposes a novel mechanism of automatic aesthetics evaluation of robotic dance motions based on multiple visual feature integration. In the mechanism, a video of robotic dance motion is firstly converted into several kinds of motion history images, and then a spatial feature (ripple space coding) and shape features (Zernike moment and curvature-based Fourier descriptors) are extracted from the optimized motion history images. Based on feature integration, a homogeneous ensemble classifier, which uses three different random forests, is deployed to build a machine aesthetics model, aiming to make the machine possess human aesthetic ability. The feasibility of the proposed mechanism has been verified by simulation experiments, and the experimental results show that our ensemble classifier can achieve a high correct ratio of aesthetics evaluation of 75%. The performance of our mechanism is superior to those of the existing approaches.

The Path Direction Control System for Lanange Jagad Dance Robot Using the MPU6050 Gyroscope Sensor

The ability to walk straight on a dance robot is very important considering that in competitions, dance robots are required to be able to walk through several zones starting from the starting zone and ending with the closed zone. Therefore, a control system is needed in the Lanange Jagad dance robot so that the robot can control the direction of its walking motion and reduce errors in dance motion while walking on the dance robot. This control system uses a reading value based on the orientation of the rotating motion on the yaw angle axis on the MPU6050 gyroscope sensor which will later be used as a corrector for dance robots when performing various dance movements while walking in the competition arena. From the results of the overall test of the Lanange Jagad dance robot after adding the road direction control system, the percentage of the success rate in the battery power supply condition is 12 volts to 12.6 volts by 100% with the greater the battery power supply, the error in the robot's final angle average to The starting angle of the robot is getting smaller and the percentage of the success rate at the slope of the 0o to 4o race arena is 93.3%. With the tilted race arena, the error in the mean error of the robot's final angle to the starting angle of the robot is also greater, so it can be concluded that the robot can be controlled direction of walking and can walk straight to the finish in the closed zone.

Research on Music Adaptive Method Based on Dance Robots

Music adaptive method, one of the important directions of dance robot research, will improve the robot's intelligence and adaptive ability, thereby greatly enriching people's growing spiritual and cultural life. From the current point of view, an overly complex adaptive method used, a large amount of computer resources will be encroached on, and the dance robot system will freeze or even crash. The dance robot is a kind of embedded device. Under the premise of saving system resources, how to make the robot do different actions according to different music and how to make it produce movements in step with the music will become a hot topic in the research of robot adaptive methods. This paper proposes a dance robot adaptive method based on music analysis. After the music signal is preprocessed, a reasonable threshold range is set to obtain music features. Through the feature and coordinate transformation, the action duration is obtained, alternative actions are selected, and action frequency is set to achieve the coordination of amplitude change and action switching. Experiments have shown that with this method used on a dance robot, the system runs smoothly featuring coordinated movement, hence a better adaptive effect.

Quantum-Based Creative Generation Method for a Dancing Robot.

In this paper, we propose a creative generation process model based on the quantum modeling simulation method. This model is mainly aimed at generating the running trajectory of a dancing robot and the execution plan of the dancing action. First, we used digital twin technology to establish data mapping between the robot and the computer simulation environment to realize intelligent controllability of the robot's trajectory and the dance movements described in this paper. Second, we conducted many experiments and carried out a lot of research into information retrieval, information fidelity, and result evaluation. We constructed a multilevel three-dimensional spatial quantum knowledge map (M-3DQKG) based on the coherence and entangled states of quantum modeling and simulation. Combined with dance videos, we used regions with convolutional neural networks (R-CNNs) to extract character bones and movement features to form a movement library. We used M-3DQKG to quickly retrieve information from the knowledge base, action library, and database, and then the system generated action models through a holistically nested edge detection (HED) network. The system then rendered scenes that matched the actions through generative adversarial networks (GANs). Finally, the scene and dance movements were integrated, and the creative generation process was completed. This paper also proposes the creativity generation coefficient as a means of evaluating the results of the creative process, combined with artificial brain electroenchalographic data to assist in evaluating the degree of agreement between creativity and needs. This paper aims to realize the automation and intelligence of the creative generation process and improve the creative generation effect and usability of dance movements. Experiments show that this paper has significantly improved the efficiency of knowledge retrieval and the accuracy of knowledge acquisition, and can generate unique and practical dance moves. The robot's trajectory is novel and changeable, and can meet the needs of dance performances in different scenes. The creative generation process of dancing robots combined with deep learning and quantum technology is a required field for future development, and could provide a considerable boost to the progress of human society.

JAIPONG DANCE ROBOT PROTOTYPE USING CM-530 CONTROLLER

Technological advances in the field of robotics have entered various aspects of human life ranging from automation in the fields of industry, military, entertainment, and in the medical field. Humanoid robot or humanoid robot is a form of technology implementation in the field of robotics that can mimic some human activities to walk and move several body organs like humans. The design of Jaipong Dance Robot based on CM-530 consists of a minimum system that functions as a controller of servo motor movement as a robot actuator and processes the data generated by the gyro sensor as a balance sensor. The design of Jaipong Dance Robot based on CM-530 has been successfully made and can work well, from standing movements, opening 1, opening 2, pencugan 1, pencugan 2, mincid kendor 1, pencugan 3, mincid kendor 2, mincid gancang 1, mincid gancang 2, mincid gancang 3, mincid gancang 4, nibakeun, dan mincid gancang 5, the success rate is 80%. Gyroscope GS-12 sensor can provide the desired balance point, the Dynamixel AX-12A Servo Motor as propulsion which is located at each joint of the jaipong dance robot, the Battery / Power Supply as the power provider.

Design of dancing robot control system based on STM32

Dancing robot control system for children’s entertainment is introduced. In order to achieve better interaction, this robot use the ARM core chip STM32 as the control core, Bluetooth module as the means of wireless communication, and smart phones as the face display screen and motion controller of dance robots. Experimental results show that the robot is stable, it can work given dance actions. The robot is good operable and maintainable.

Creating a Computable Cognitive Model of Visual Aesthetics for Automatic Aesthetics Evaluation of Robotic Dance Poses

Inspired by human dancers who can evaluate the aesthetics of their own dance poses through mirror observation, this paper presents a corresponding mechanism for robots to improve their cognitive and autonomous abilities. Essentially, the proposed mechanism is a brain-like intelligent system that is symmetrical to the visual cognitive nervous system of the human brain. Specifically, a computable cognitive model of visual aesthetics is developed using the two important aesthetic cognitive neural models of the human brain, which is then applied in the automatic aesthetics evaluation of robotic dance poses. Three kinds of features (color, shape and orientation) are extracted in a manner similar to the visual feature elements extracted by human brains. After applying machine learning methods in different feature combinations, machine aesthetics models are built for automatic evaluation of robotic dance poses. The simulation results show that our approach can process visual information effectively by cognitive computation, and achieved a very good evaluation performance of automatic aesthetics.

Cognitive ability and psychological effectiveness of brain training dance robot therapy for elderly people

Introduction: Regarding dementia prevention, as it has been reported that the volume of hippocampus increase with continuous dancing and that dancers’ gray matter increases, dancing and recognition tasks have been combined and developed into brain training. Furthermore, we equipped a robot with dance therapy and a cognitive evaluation scale, the results of which we will exam as the focus of this study. Methods: Comparison of the results of cognitive evaluation tests before and after 7 weeks of continuous cognitive dance therapy once a week. The cognitive evaluation test utilized was the cognitive test used by the National Police Agency for the renewal of elderly person’s licenses. Moreover, mental states before and after dance were compared on a five-level Likert scale, and we surveyed the psychological condition of participants after robot therapy. Results: Of the 91 registrants, with no missing values data of 71 people was analyzed with paired t-test. The mean age was 70.3±5.7 years old, the average score for the cognitive test prior to intervention was 86.8 points. After 7 weeks of intervention, the average score was 94.7 points, significant increase (p<0.01). At the Likert scale of the mental state, there was a significant improvement in mood comfort, satisfaction, and vitality (p<0.01). And there was a correlation between cognitive function and mental state (p <0.05). Next, the average score of the robot therapy satisfaction level was as high as 4.73 out of 5 points. In the comments section, there were in descending order comments such as being happy, healing, clarity of the mind, facilitation of social interactions, et cetera. Conclusion: Cognitive dance therapy improves cognitive ability. Remembering songs and choreography together with music is indeed easy and effective. Additionally, with the combined effects of the results from robot psychological healing, it was shown that both the mind and body of older people are revitalized. Keywords: Brain training, Dance therapy, Robot therapy, Cognitive function, Psychological effectiveness.