Human-robot Interaction Technologies Research Articles

Human–robot interaction: predicting research agenda by long short-term memory

The article addresses the identification and prediction of research topics in human–robot interaction (HRI), fundamental in Industry 4.0 (I4.0) and future Industry 5.0 (I5.0). In the absence of research agendas in the scientific literature, the study proposes a multilayered model to create a precise agenda to guide the scientific community in new developments in collaborative robotics and HRI technologies. The methodology is divided into four stages, which make up the three layers of the model. In the first two stages, scientific articles on HRI for the period 2020–2021 were collected and analyzed using data mining techniques together with VantagePoint and Gephi software to identify keywords and their relationships. These initial stages form layer 1 of the model, where the main scientific themes are recognized. In the third stage, article titles and abstracts are cleaned and processed using natural language processing (NLP) techniques, generating word embeddings models that highlight relevant HRI-related terms, forming layer 2. The fourth and final stage uses Recurrent Neural Networks (RNN) with long short-term memory (LSTM) architecture to predict future topics, consolidating the previously identified terms and forming layer 3 of the model. The results show that in layer 1 HRI has intensive application in various sectors through advanced computational algorithms, with trust as a key feature. In layer 2, terms such as vision, sensors, communication, collaboration and anthropomorphic aspects are fundamental, while layer 3 anticipates future topics such as design, performance, method and controllers, essential to improve robot interaction. The study concludes that the methodology is effective in defining a robust and relevant research agenda. By identifying future trends and needs, this work fills a gap in the scientific literature, providing a valuable tool for the research community in the field of HRI.

Research on shared control of robots based on hybrid brain-computer interface

BackgroundWith the arrival of the new generation of artificial intelligence wave, new human-robot interaction technologies continue to emerge. Brain–computer interface (BCI) offers a pathway for state monitoring and interaction control between human and robot. However, the unstable mental state reduce the accuracy of human brain intent decoding, and consequently affects the precision of BCI control. New methodsThis paper proposes a hybrid BCI-based shared control (HB-SC) method for brain-controlled robot navigation. Hybrid BCI fuses electroencephalogram (EEG) and electromyography (EMG) for mental state monitoring and interactive control to output human perception and decision. The shared control based on multi-sensory fusion integrates the special obstacle information perceived by humans with the regular environmental information perceived by the robot. In this process, valid BCI commands are screened by mental state assessment and output to a layered costmap for fusion. ResultsEight subjects participated in the navigation experiment with dynamically changing mental state levels to validate the effects of a hybrid brain-computer interface through two shared control modes. The results show that the proposed HB-SC reduces collisions by 37.50 %, improves the success rate of traversing obstacles by 25.00 %, and the navigation trajectory is more consistent with expectations. ConclusionsThe HB-SC method can dynamically and intelligently adjust command output according to different brain states, helping to reduce errors made by subjects in a unstable mental state, thereby greatly enhancing the system's safety.

Interactive robots for personalised multimodal comedy experiments

This research proposes a novel system for personalising humour delivery using interactive robots with AI, natural language processing, and human-robot interaction technologies. The robots can adapt the multimodal content of jokes tactically now and strategically over time based on recording and analyzing each user’s subjective emotional reactions. The study evaluated pragmatic aspects of timing and framing jokes by having the robots try different “comeback tactics” and observing audience response. An ANOVA analysis revealed six distinct response categories. 78 % of participants positively rated the personalized robot comedy across the six scenarios. However, when both robot comedians took a negative tone, only 23 % approved (p < 0.001). Interestingly, audiences validated the robot comedian’s performance (89 % approval) more than the human comedian (54 % approval), a statistically significant difference (p < 0.01). These findings shed light on ideal characteristics for effective human-robot comedy interactions, highlighting the importance of topicality, maintaining a generally positive mood, and the unique strengths of robot performers. The study paves the way for further optimization of personalized, adaptive robot humour systems leveraging multimodal AI capabilities to enhance entertainment experiences tailored to individuals.

Gesture Controlled Robot for the Disabled using Arduino

Abstract: The gesture-controlled robot project presents a novel approach to human-robot interaction, utilizing hand gestures as an intuitive and natural means of control. This project aims to develop a system where users can seamlessly communicate with a robot using simple hand movements. The project consists of two main components: a handheld device for gesture input and a robot equipped with sensors and actuators for movement. Through wireless communication, the handheld device transmits signals encoding the user's hand gestures to the robot's receiver module, which decodes the signals, processes them, and translates them into commands for controlling the robot's movements. Advantages of this system include intuitive control, wireless operation, real-time interaction, enhanced safety, adaptability, educational value, and versatility. However, challenges such as gesture recognition complexity and limited gesture vocabulary must be addressed to ensure the system's effectiveness. The project has numerous applications, including entertainment, assistive technology, industrial automation, education, home automation, IoT, security, and interactive exhibits. By leveraging gesture-controlled robotics, this project contributes to advancing human-robot interaction technology and fostering innovation in various domains.

Mechanically Robust and Linearly Sensitive Soft Piezoresistive Pressure Sensor for a Wearable Human-Robot Interaction System.

Soft piezoresistive pressure sensors play an underpinning role in enabling a plethora of future Internet of Things (IoT) applications such as human-robot interaction (HRI) technologies, wearable devices, and metaverse ecosystems. Despite significant attempts to enhance the performance of these sensors, existing sensors still fall short of achieving high strain tolerance and linearity simultaneously. Herein, we present a low-cost, facile, and scalable approach to fabricating a highly strain-tolerant and linearly sensitive soft piezoresistive pressure sensor. Our design utilizes thin nanocracked gold films (NC-GFs) deposited on poly(dimethylsiloxane) (PDMS) as electrodes of the sensor. The large mismatch stress between gold (Au) and PDMS induces the formation of secondary wrinkles along the pyramidal-structured electrode under pressure; these wrinkles function as protuberances on the electrode and enable exceptional linear sensitivity of 4.2 kPa-1 over a wide pressure range. Additionally, our pressure sensor can maintain its performance even after severe mechanical deformations, including repeated stretching up to 30% strain, due to the outstanding strain tolerance of NC-GF. Our sensor's impressive sensing performance and mechanical robustness make it suitable for diverse IoT applications, as demonstrated by its use in wearable pulse monitoring devices and human-robot interaction systems.

Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion.

As human-robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method's effectiveness through a series of experiments, evaluating the system's robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion.

Modelling and optimisation of two-sided disassembly line balancing problem with human–robot interaction constraints

The mature application of human–robot interaction technology provides a new operation method for remanufacturing waste products, especially in the disassembly process, which further exerts the respective advantages of humans and robots. Therefore, this paper proposes an innovative two-sided disassembly line balancing problem with human–robot interaction constraints (TDLBP-HRI). First, a mixed integer programming model is established for TDLBP-HRI to minimise the number of workstations and smoothness index while simultaneously optimising robot energy consumption and disassembly costs. Then, a discrete squirrel search algorithm (DSSA) is developed to efficiently solve large-scale problems caused by the NP-hard property of TDLBP-HRI. Since SSA is an algorithm that can only solve single-objective linear problems, this research attempts to discretise and improve it to solve multi-objective TDLBP-HRI. In addition, according to the characteristics of the problem, a two-layer decoding strategy is constructed to ensure the uniqueness of the solution, and a fast non-dominated sort is introduced to enhance the multi-objective optimisation capability of the DSSA. To verify the performance of the proposed DSSA, the benchmarks of two other DLBPs types are calculated, and the obtained results are found to be significantly better than those reported in the literature. The correctness of the MIP model and DSSA is verified by comparing the results of the GUROBI solver and DSSA in solving two small-scale TDLBP-HRI examples. Following this, DSSA is shown to outperform four other classical algorithms through different scales cases of TDLBP-HRI. Finally, the proposed model and algorithm are applied to a waste power battery disassembly example. The results show that the proposed model and method can improve the performance of the disassembly line and provide new insights into the application of human–robot interaction technology in the disassembly line.

EMG-Based Dynamic Hand Gesture Recognition Using Edge AI for Human–Robot Interaction

Recently, human–robot interaction technology has been considered as a key solution for smart factories. Surface electromyography signals obtained from hand gestures are often used to enable users to control robots through hand gestures. In this paper, we propose a dynamic hand-gesture-based industrial robot control system using the edge AI platform. The proposed system can perform both robot operating-system-based control and edge AI control through an embedded board without requiring an external personal computer. Systems on a mobile edge AI platform must be lightweight, robust, and fast. In the context of a smart factory, classifying a given hand gesture is important for ensuring correct operation. In this study, we collected electromyography signal data from hand gestures and used them to train a convolutional recurrent neural network. The trained classifier model achieved 96% accuracy for 10 gestures in real time. We also verified the universality of the classifier by testing it on 11 different participants.

A Review on Usability and User Experience of Assistive Social Robots for Older Persons

In the advancement of human-robot interaction technology, assistive social robots have been recognized as one of potential technologies that can provide physical and cognitive supports in older person’s care. However, a major challenge faced by the designers is to develop an assistive social robot with prodigious usability and user experience for older persons who were known to have physical and cognitive limitations. A considerable number of published literatures was reporting on the technological design process of assistive social robots. However, only a small amount of attention has been paid to review the usability and user experience of the robots. The objective of this paper is to provide an overview of established researches in the literatures concerning usability and user experience issues faced by the older persons when interacting with assistive social robots. The authors searched relevant articles from the academic databases such as Google Scholar, Scopus and Web of Science as well as Google search engine for the publication period 2000 to 2021. Several search keywords were typed such as ‘older persons’ ‘elderly’, ‘senior citizens’, ‘assistive social robots’, ‘companion robots’, ‘personal robots’, ‘usability’ and ‘user experience’. This online search found a total of 215 articles which are related to assistive social robots in elderly care. Out of which, 54 articles identified as significant references, and they were examined thoroughly to prepare the main content of this paper. This paper reveals usability issues of 28 assistive social robots, and user experience feedback based on 41 units of assistive social robots. Based on the research articles scrutinized, the authors concluded that the key elements in the design and development of assistive social robots to improve acceptance of older persons were determined by three factors: functionality, usability and users’ experience. Functionality refers to ability of robots to serve the older persons. Usability is ease of use of the robots. It is an indicator on how successful of interaction between the robots and the users. To improve usability, robot designers should consider the limitations of older persons such as vision, hearing, and cognition capabilities when interactingwith the robots. User experience reflects to perceptions, preferences and behaviors of users that occur before, during and after use the robots. Combination of superior functionality and usability lead to a good user experiencein using the robots which in the end achieves satisfaction of older persons.

A brain-inspired intention prediction model and its applications to humanoid robot.

With the development of artificial intelligence and robotic technology in recent years, robots are gradually integrated into human daily life. Most of the human-robot interaction technologies currently applied to home service robots are programmed by the manufacturer first, and then instruct the user to trigger the implementation through voice commands or gesture commands. Although these methods are simple and effective, they lack some flexibility, especially when the programming program is contrary to user habits, which will lead to a significant decline in user experience satisfaction. To make that robots can better serve human beings, adaptable, simple, and flexible human-robot interaction technology is essential. Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction model to enable the robot to perform actions according to the user's intention. With the spike-timing-dependent plasticity (STDP) mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by (N 2 - N)/4, where N is the number of intentions.

Active tactile sensing of small insect force by a soft microfinger toward microfinger-insect interactions

Human–robot interaction technology has contributed to improving sociality for humanoid robots. At scales far from human scales, a microrobot can interact with an environment in a small world. Microsensors have been applied to measurement of forces by flying or walking insects. Meanwhile, most previous works focused on the measurement of the behavior of insects. Here, we propose microrobot-insect interactions by soft microfingers integrated with artificial muscle actuators and tactile sensors, which has been developed for a haptic teleoperation robot system. A soft pneumatic balloon actuator acts as the artificial muscle, and a flexible strain sensor using a liquid metal provides tactile sensing. Force interaction between a pill bug and the microfinger could be accomplished. The microfinger (12 mm × 3 mm × 490 μm) can move and touch an insect, and it can detect reaction force from an insect. The measured reaction force from the legs of a pill bug as a representative insect was less than 10 mN. This paper presents a microfinger as an end effector for the active sensing of reaction force from a small insect. We anticipate that our results will lead to further evaluation of small living things as well as technology development for human–environment interaction.

Introduction to the Special Issue on Test Methods for Human-Robot Teaming Performance Evaluations

This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance.

A novel start-up method of sensorless passive lead-through programming for industrial robots

PurposeSensorless passive lead-through programming (LTP) is a promising physical human-robot interaction technology that enables manual trajectory demonstrations based on gravity and friction compensation. The major difficulty lies in static friction compensation during LTP start-up. Instead of static friction compensation, conventional methods only compensate for Coulomb friction after the joint velocity exceeds a threshold. Therefore, conventional start-up external torques must overcome static friction. When the static friction is considerable, it is difficult for conventional LTP to start up and make small movements. This paper aims to decrease the start-up external torque and improve the small movement performance.Design/methodology/approachThis paper reveals a novel usage of a high-gain position-loop in industrial robot applications aimed at sensitively detecting external torque during start-up. Then, the static friction is partly compensated by Coulomb friction to facilitate start-up. In addition, a detailed transition method between the proposed start-up and conventional passive LTP is proposed based on a finite state machine.FindingsExperiments are implemented on the ROKAE XB4 robot to verify the effectiveness of the proposed external torque detection. Compared with the conventional LTP method, the proposed LTP method significantly decreases the start-up external torque and facilitates small movements.Originality/valueThis paper proposes and verifies a novel start-up method of sensorless LTP based on a start-up external torque detection and a transition method between start-up and conventional LTP. This research improves the LTP start-up performance, especially for industrial robots with large static friction.

Virtual reality human–robot interaction technology acceptance model for learning direct current and alternating current

Virtual reality human–robot interaction technology acceptance model for learning direct current and alternating current

Development and usability evaluation of a bedside robot system for inpatients.

Many inpatients become anxious or frightened about scheduled treatment processes, and medical staff do not have sufficient time to provide emotional support. The recent advancement of information and communications technology (ICT) and the use of artificial intelligence (AI), including robots, in the health care field is being put to the test. This study aimed to develop a bedside robot system to deliver information and provide emotional support to inpatients and to evaluate the usability and perceptions of the developed robot. The first stage was accomplished by deriving essential functions from the results of user demand surveys on robots and by implementing a prototype by mapping each essential function to the robot's hardware and software. For the second stage, the robot was assessed for usability and perceptions in a simulation center, a hospital-like environment, by 10 nurses, 10 inpatients, and family caregivers. Usability and perception were evaluated using the think-aloud method, a survey, and individual interviews. Based on the usability evaluation, the perceived usefulness, ease of use, and satisfaction were 5.28 ± 1.27 points, 5.42 ± 1.55 points, and 5.27 ± 1.46 points out of 7, respectively. It was found that overall, the robot was positively perceived by participants. As a result of the qualitative data analysis, the participants perceived the robot as an object that had the positive effect of providing emotional support through communication. The bedside robot in this study, which incorporated human-robot interaction (HRI) technology, is an alternative suited to the new normal era that will contribute to ensuring that patients have more self-directed hospital stays as well as emotional support through information delivery and communication.

Identification of Hand Gestures Using the Inertial Measurement Unit of a Smartphone: A Proof-of-Concept Study

Assessing the hand postures is crucial to implement gesture-based user-computer interfaces for controlling robots and assistive devices. Apart from data gloves and optical tracking, techniques such as surface electromyography and force myography provide a straightforward, non-invasive way to estimate poses and intentions through the forearm muscles assessment. However, most of the myography systems rely on bulky, dedicated hardware with arrays of electrodes or force probes. Therefore, this work introduces the smartphone as an alternative for identifying gestures: with the mobile device attached to the forearm, the embedded inertial measurement unit detects muscular contractions produced during the transitions between postures, yielding signatures in acquired waveforms. After computing the correlation of measured and template patterns, a competitive layer votes the class with greater probability and identifies the gesture. Prior characterization studies evaluated the effect of smartphone placement and forearm orientation in the sensor response, revealing that the IMU signatures are repeatable and robust to positioning deviations. Next, using 10-fold cross-validation, the system discerned four gestures (fist, open hand, wave in, and wave out) performed by six volunteers in ten repetitions, providing 96.6% and 94.1% average accuracies for self-calibration and inter-participant analyses, respectively. The smartphone figures as a ubiquitous and straightforward alternative for assessing gestures, with further applications in human-robot interaction and assistive technologies.

Research Progress and Development Trend of the Safety of Human-robot Interaction Technology

Research Progress and Development Trend of the Safety of Human-robot Interaction Technology

Live human-robot interactive public demonstrations with automatic emotion and personality prediction.

Communication with humans is a multi-faceted phenomenon where the emotions, personality and non-verbal behaviours, as well as the verbal behaviours, play a significant role, and human-robot interaction (HRI) technologies should respect this complexity to achieve efficient and seamless communication. In this paper, we describe the design and execution of five public demonstrations made with two HRI systems that aimed at automatically sensing and analysing human participants' non-verbal behaviour and predicting their facial action units, facial expressions and personality in real time while they interacted with a small humanoid robot. We describe an overview of the challenges faced together with the lessons learned from those demonstrations in order to better inform the science and engineering fields to design and build better robots with more purposeful interaction capabilities. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Multi-floor laboratory transportation technologies based on intelligent mobile robots

AbstractTransportation technologies for mobile robots include indoor navigation, intelligent collision avoidance and target manipulation. This paper discusses the research process and development of these interrelated technologies. An efficient multi-floor laboratory transportation system for mobile robots developed by the group at the Center for Life Science Automation (CELISCA) is then introduced. This system is integrated with the multi-floor navigation and intelligent collision avoidance systems, as well as a labware manipulation system. A multi-floor navigation technology is proposed, comprising sub-systems for mapping and localization, path planning, door control and elevator operation. Based on human–robot interaction technology, a collision avoidance system is proposed that improves the navigation of the robots and ensures the safety of the transportation process. Grasping and placing operation technologies using the dual arms of the robots are investigated and integrated into the multi-floor transportation system. The proposed transportation system is installed on the H20 mobile robots and tested at the CELISCA laboratory. The results show that the proposed system can ensure the mobile robots are successful when performing multi-floor laboratory transportation tasks.

Applying Human-Robot Interaction Technology in Retail Industries

Applying Human-Robot Interaction Technology in Retail Industries