Human-robot Interaction Applications Research Articles

A Quantitative Analysis of Dressing Dynamics for Robotic Dressing Assistance

Assistive robots have a great potential to address issues related to an ageing population and an increased demand for caregiving. Successful deployment of robots working in close proximity with people requires consideration of both safety and human-robot interaction. One of the established activities of daily living where robots could play an assistive role is dressing. Using the correct force profile for robot control will be essential in this application of human-robot interaction requiring careful exploration of factors related to the user’s pose and the type of garments involved. In this paper a Baxter robot was used to dress a jacket onto a mannequin and human participants considering several combinations of user pose and clothing type (base layers), whilst recording dynamic data from the robot, a load cell and an IMU. We also report on suitability of these sensors for identifying dressing errors, e.g. fabric snagging. Data was analyzed by comparing the overlap of confidence intervals to determine sensitivity to dressing. We expand the analysis to include classification techniques such as decision tree and support vector machines using k-fold cross-validation. The 6-axis load cell successfully discriminated between clothing types with predictive model accuracies between 72-97%. Used independently, the IMU and Baxter sensors were insufficient to discriminate garment types with the IMU showing 40-72% accuracy, but when used in combination this pair of sensors achieved an accuracy similar to the more expensive load cell (98%). When observing dressing errors (snagging) Baxter’s sensors and the IMU data demonstrated poor sensitivity but applying machine learning methods resulted in model with high predicative accuracy and low false negative rates (≤5%). The results show that the load cell could be used independently for this application with good accuracy but a combination of the lower cost sensors could also be used without a significant loss in precision which will be a key element in the robot control architecture for safe human-robot interaction.

A Bayesian hierarchy for robust gaze estimation in human–robot interaction

In this text, we present a probabilistic solution for robust gaze estimation in the context of human–robot interaction. Gaze estimation, in the sense of continuously assessing gaze direction of an interlocutor so as to determine his/her focus of visual attention, is important in several important computer vision applications, such as the development of non-intrusive gaze-tracking equipment for psychophysical experiments in neuroscience, specialised telecommunication devices, video surveillance, human–computer interfaces (HCI) and artificial cognitive systems for human–robot interaction (HRI), our application of interest. We have developed a robust solution based on a probabilistic approach that inherently deals with the uncertainty of sensor models, but also and in particular with uncertainty arising from distance, incomplete data and scene dynamics. This solution comprises a hierarchical formulation in the form of a mixture model that loosely follows how geometrical cues provided by facial features are believed to be used by the human perceptual system for gaze estimation. A quantitative analysis of the proposed framework's performance was undertaken through a thorough set of experimental sessions. Results show that the framework performs according to the difficult requirements of HRI applications, namely by exhibiting correctness, robustness and adaptiveness.

On human–robot interaction of a 3-DOF decoupled parallel mechanism based on the design and construction of a novel and low-cost 3-DOF force sensor

This paper addresses the extension of a 3-degrees-of-freedom (3-DOF) decoupled parallel mechanism for human–robot interaction purposes. To this end, a low-cost 3-DOF force sensor for human–robot interaction applications is proposed, designed and constructed. In the latter force sensor, five load cells are placed in order to identify the amount of the applied force along each Cartesian direction. In addition, an experimental identification procedure based on least square method is carried out in order to obtain the first and third degree polynomial models of the sensor output model. From the practical tests it has been reveled that the force sensor has a reasonable precision of 0.1 N in both x and y-axes and 0.2 N in z-axis, within a range of 5 N which is suitable for human–robot interaction applications. Then, using the proposed force sensor, two control methods, namely “position control” and “speed control” are applied for human–robot interaction purposes and their performances are compared.

Soft Pneumatic Bending Actuator with Integrated Carbon Nanotube Displacement Sensor

The excellent compliance and large range of motion of soft actuators controlled by fluid pressure has lead to strong interest in applying devices of this type for biomimetic and human-robot interaction applications. However, in contrast to soft actuators fabricated from stretchable silicone materials, conventional technologies for position sensing are typically rigid or bulky and are not ideal for integration into soft robotic devices. Therefore, in order to facilitate the use of soft pneumatic actuators in applications where position sensing or closed loop control is required, a soft pneumatic bending actuator with an integrated carbon nanotube position sensor has been developed. The integrated carbon nanotube position sensor presented in this work is flexible and well suited to measuring the large displacements frequently encountered in soft robotics. The sensor is produced by a simple soft lithography process during the fabrication of the soft pneumatic actuator, with a greater than 30% resistance change between the relaxed state and the maximum displacement position. It is anticipated that integrated resistive position sensors using a similar design will be useful in a wide range of soft robotic systems.

Passivation of Projection-Based Null Space Compliance Control Via Energy Tanks

A manipulator with kinematic redundancy w.r.t. a main task enables to perform an additional, subordinate task in its null space. The classical hierarchy-based approach is to apply dynamically consistent null space projections such that this lower-priority task does not disturb the main task. The major problem is that the control task hierarchy is established at the expense of the passivity of the system, which causes problems in terms of stability, robustness, and safety. Here, we extend this classical approach and passivate the closed-loop system by means of an energy tank, which is filled by the dissipated power of the active damping on both hierarchy levels. We validate the method in simulations and experiments on a torque-controlled manipulator. The concept is predestined for physical human-robot interaction and safety-relevant applications, where passivity must be guaranteed.

Affective Haptics: Current Research and Future Directions

Touch plays a prominent role in communicating emotions and intensifying interpersonal communication. Affective haptics is an emerging field, which focuses on the analysis, design, and evaluation of systems that can capture, process, or display emotions through the sense of touch. The objective of this paper is to present an overview of the recent achievements in affective haptics and to discuss how the sense of touch can elicit or influence human emotions. We first introduce a definition to the term affective haptics and describe its multidisciplinary nature—as a field that integrates ideas from affective computing, haptic technology, and user experience. Second, we provide a thorough discussion about the effectiveness of using the haptic channel to communicate affective information through direct and mediated means. Third, we present a variety of applications in the area ranging from interhuman social interaction systems to human robot interaction applications. Finally, we discuss some of the key findings discerned from the various surveyed papers, and present some of the challenges and trends in this field. We extract the following conclusions pertaining to affective haptics: 1) haptic stimulation can be successfully used to achieve a higher level of emotional immersion during media consumption or emotional telepresence; 2) existing research has demonstrated that haptics is effective in communicating valence and arousal, and the emotions of happiness, sadness, anger and fear, and less focus have been given to the communication of disgust and surprise; 3) the haptic-based affect detection remains an understudied topic, whereas the haptic-based affect display is a well-established subject; and 4) the interpretation of the haptic stimulation by human beings is highly contextual.

OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks

OpenPTrack is an open source software for multi-camera calibration and people tracking in RGB-D camera networks. It allows to track people in big volumes at sensor frame rate and currently supports a heterogeneous set of 3D sensors.In this work, we describe its user-friendly calibration procedure, which consists of simple steps with real-time feedback that allow to obtain accurate results in estimating the camera poses that are then used for tracking people. On top of a calibration based on moving a checkerboard within the tracking space and on a global optimization of cameras and checkerboards poses, a novel procedure which aligns people detections coming from all sensors in a x-y-time space is used for refining camera poses.While people detection is executed locally, in the machines connected to each sensor, tracking is performed by a single node which takes into account detections from all over the network. Here we detail how a cascade of algorithms working on depth point clouds and color, infrared and disparity images is used to perform people detection from different types of sensors and in any indoor light condition.We present experiments showing that a considerable improvement can be obtained with the proposed calibration refinement procedure that exploits people detections and we compare Kinect v1, Kinect v2 and Mesa SR4500 performance for people tracking applications. OpenPTrack is based on the Robot Operating System and the Point Cloud Library and has already been adopted in networks composed of up to ten imagers for interactive arts, education, culture and human–robot interaction applications.

Probabilistic Social Behavior Analysis by Exploring Body Motion-Based Patterns

Understanding human behavior through nonverbal-based features, is interesting in several applications such as surveillance, ambient assisted living and human-robot interaction. In this article in order to analyze human behaviors in social context, we propose a new approach which explores interrelations between body part motions in scenarios with people doing a conversation. The novelty of this method is that we analyze body motion-based features in frequency domain to estimate different human social patterns: Interpersonal Behaviors (IBs) and a Social Role (SR). To analyze the dynamics and interrelations of people's body motions, a human movement descriptor is used to extract discriminative features, and a multi-layer Dynamic Bayesian Network (DBN) technique is proposed to model the existent dependencies. Laban Movement Analysis (LMA) is a well-known human movement descriptor, which provides efficient mid-level information of human body motions. The mid-level information is useful to extract the complex interdependencies. The DBN technique is tested in different scenarios to model the mentioned complex dependencies. The study is applied for obtaining four IBs (Interest, Indicator, Empathy and Emphasis) to estimate one SR (Leading).The obtained results give a good indication of the capabilities of the proposed approach for people interaction analysis with potential applications in human-robot interaction.

Review of Semantic-Free Utterances in Social Human–Robot Interaction

As a young and emerging field in social human–robot interaction (HRI), semantic-free utterances (SFUs) research has been receiving attention over the last decade. SFUs are an auditory interaction means for machines that allow emotion and intent expression, which are composed of vocalizations and sounds without semantic content or language dependence. Currently, SFUs are most commonly utilized in animation movies (e.g., R2-D2, WALL-E, Despicable Me), cartoons (e.g., “Teletubbies,” “Morph,” “La Linea”), and computer games (e.g., The Sims) and hold significant potential for applications in HRI. SFUs are categorized under four general types: Gibberish Speech (GS), Non-Linguistic Utterances (NLUs), Musical Utterances (MU), and Paralinguistic Utterances (PU). By introducing the concept of SFUs and bringing multiple sets of studies in social HRI that have never been analyzed jointly before, this article addresses the need for a comprehensive study of the existing literature for SFUs. It outlines the current grand challenges, open questions, and provides guidelines for future researchers considering to utilize SFU in social HRI.

Flexible Optical Fiber Bending Transducer for Application in Glove-Based Sensors

The development of a low-cost and flexible optical fiber transducer for measurement of angular displacements is reported. The light intensity attenuation due to fiber microbending losses is correlated to the variations in flexing angle, yielding to a sensitivity of 1.80°. The device was also mounted in a fabric glove to the monitoring of flexion and abduction movements of index and thumb fingers. Once calibrated by a simple procedure, the glove-based system was capable to measure the angular positions with average errors <;5° and 7° for interphalangeal and metacarpophalangeal joints, respectively. Additionally, the repeatability analysis resulted in average range and standard deviations of 8.06° and 3.45°, respectively. The optical fiber sensor provides a low-cost alternative to the real-time monitoring of hand posture, and can be suitable for applications in human-robot and human-computer interactions.

Robust Force Control of Series Elastic Actuators

Force-controlled series elastic actuators (SEA) are widely used in novel human-robot interaction (HRI) applications, such as assistive and rehabilitation robotics. These systems are characterized by the presence of the “human in the loop”, so that control response and stability depend on uncertain human dynamics, including reflexes and voluntary forces. This paper proposes a force control approach that guarantees the stability and robustness of the coupled human-robot system, based on sliding-mode control (SMC), considering the human dynamics as a disturbance to reject. We propose a chattering free solution that employs simple task models to obtain high performance, comparable with second order solutions. Theoretical stability is proven within the sliding mode framework, and predictability is reached by avoiding the reaching phase by design. Furthermore, safety is introduced by a proper design of the sliding surface. The practical feasibility of the approach is shown using an SEA prototype coupled with a human impedance in severe stress tests. To show the quality of the approach, we report a comparison with state-of-the-art second order SMC, passivity-based control and adaptive control solutions.

Human-adaptive control of series elastic actuators

SUMMARYForce-controlled series elastic actuators (SEAs) are the widely used components of novel physical human–robot interaction applications such as assistive and rehabilitation robotics. These systems are characterized by the presence of the “human in the loop” so that control response and stability depend on uncertain human dynamics. A common approach to guarantee stability is to use a passivity-based controller. Unfortunately, existing passivity-based controllers for SEAs do not define the performance of the force/torque loop. We propose a method to obtain predictable force/torque dynamics based on adaptive control and oversimplified human models. We propose a class of stable human-adaptive algorithms and experimentally show advantages of the proposed approach.

Reactive virtual agent learning for NUI-based HRI applications

The natural user interface (NUI) has been investigated in a variety of fields in application software. This paper proposes an approach to generate virtual agents that can support users for NUI-based applications through human---robot interaction (HRI) learning in a virtual environment. Conventional human---robot interaction (HRI) learning is carried out by repeating processes that are time-consuming, complicated and dangerous because of certain features of robots. Therefore, a method is needed to train virtual agents that interact with virtual humans imitating human movements in a virtual environment. Then the result of this virtual agent can be applied to NUI-based interactive applications after the interaction learning is completed. The proposed method was applied to a model of a typical house in virtual environment with virtual human performing daily-life activities such as washing, eating, and watching TV. The results show that the virtual agent can predict a human's intent, identify actions that are helpful to the human, and can provide services 16 % faster than a virtual agent trained using traditional Q-learning.

A Methodology for Recognition of Emotions Based on Speech Analysis, for Applications to Human-Robot Interaction. An Exploratory Study

AbstractA system for recognition of emotions based on speech analysis can have interesting applications in human-robot interaction. In this paper, we carry out an exploratory study on the possibility to use a proposed methodology to recognize basic emotions (sadness, surprise, happiness, anger, fear and disgust) based on phonetic and acoustic properties of emotive speech with the minimal use of signal processing algorithms. We set up an experimental test, consisting of choosing three types of speakers, namely: (i) five adult European speakers, (ii) five Asian (Middle East) adult speakers and (iii) five adult American speakers. The speakers had to repeat 6 sentences in English (with durations typically between 1 s and 3 s) in order to emphasize rising-falling intonation and pitch movement. Intensity, peak and range of pitch and speech rate have been evaluated. The proposed methodology consists of generating and analyzing a graph of formant, pitch and intensity, using the open-source PRAAT program. From the experimental results, it was possible to recognize the basic emotions in most of the cases

Applications of PCA and SVM-PSO Based Real-Time Face Recognition System

This paper incorporates principal component analysis (PCA) with support vector machine-particle swarm optimization (SVM-PSO) for developing real-time face recognition systems. The integrated scheme aims to adopt the SVM-PSO method to improve the validity of PCA based image recognition systems on dynamically visual perception. The face recognition for most human-robot interaction applications is accomplished by PCA based method because of its dimensionality reduction. However, PCA based systems are only suitable for processing the faces with the same face expressions and/or under the same view directions. Since the facial feature selection process can be considered as a problem of global combinatorial optimization in machine learning, the SVM-PSO is usually used as an optimal classifier of the system. In this paper, the PSO is used to implement a feature selection, and the SVMs serve as fitness functions of the PSO for classification problems. Experimental results demonstrate that the proposed method simplifies features effectively and obtains higher classification accuracy.

Automated Proxemic Feature Extraction and Behavior Recognition: Applications in Human-Robot Interaction

In this work, we discuss a set of feature representations for analyzing human spatial behavior (proxemics) motivated by metrics used in the social sciences. Specifically, we consider individual, physical, and psychophysical factors that contribute to social spacing. We demonstrate the feasibility of autonomous real-time annotation of these proxemic features during a social interaction between two people and a humanoid robot in the presence of a visual obstruction (a physical barrier). We then use two different feature representations—physical and psychophysical—to train Hidden Markov Models (HMMs) to recognize spatiotemporal behaviors that signify transitions into (initiation) and out of (termination) a social interaction. We demonstrate that the HMMs trained on psychophysical features, which encode the sensory experience of each interacting agent, outperform those trained on physical features, which only encode spatial relationships. These results suggest a more powerful representation of proxemic behavior with particular implications in autonomous socially interactive and socially assistive robotics.

A Novel Fuzzy Clustering Method with No Outliers Influence

Fuzzy C-Means (FCM) clustering algorithm can be used to classify hand gesture images in human-robot interaction application. However, FCM algorithm does not work well on those images in which noises exist. The noises or outliers make all the cluster centers towards to the center of all points. In this paper, a new FCM algorithm is proposed to detect the outliers and then make the outliers have no influence on centers calculation. The experiment shows that the new FCM algorithm can get more accurate centers than the traditional FCM algorithm.

End-user programming architecture facilitates the uptake of robots in social therapies

This paper proposes an architecture that makes programming of robot behavior of an arbitrary complexity possible for end-users and shows the technical solutions in a way that is easy to understand and generalize to different situations. It aims to facilitate the uptake and actual use of robot technologies in therapies for training social skills to autistic children. However, the framework is easy to generalize for an arbitrary human–robot interaction application, where users with no technical background need to program robots, i.e. in various assistive robotics applications. We identified the main needs of end-user programming of robots as a basic prerequisite for the uptake of robots in assistive applications. These are reusability, modularity, affordances for natural interaction and the ease of use. After reviewing the shortcomings of the existing architectures, we developed an initial architecture according to these principles and embedded it in a robot platform. Further, we used a co-creation process to develop and concretize the architecture to facilitate solutions and create affordances for robot specialists and therapists. Several pilot tests showed that different user groups, including therapists with general computer skills and adolescents with autism could make simple training or general behavioral scenarios within 1 h, by connecting existing behavioral blocks and by typing textual robot commands for fine-tuning the behaviors. In addition, this paper explains the basic concepts behind the TiViPE based robot control platform, and gives guidelines for choosing the robot programming tool and designing end-user platforms for robots.

Human motion behavior while interacting with an industrial robot

Human workers and industrial robots both have specific strengths within industrial production. Advantageously they complement each other perfectly, which leads to the development of human-robot interaction (HRI) applications. Bringing humans and robots together in the same workspace may lead to potential collisions. The avoidance of such is a central safety requirement. It can be realized with sundry sensor systems, all of them decelerating the robot when the distance to the human decreases alarmingly and applying the emergency stop, when the distance becomes too small. As a consequence, the efficiency of the overall systems suffers, because the robot has high idle times. Optimized path planning algorithms have to be developed to avoid that. The following study investigates human motion behavior in the proximity of an industrial robot. Three different kinds of encounters between the two entities under three robot speed levels are prompted. A motion tracking system is used to capture the motions. Results show, that humans keep an average distance of about 0,5m to the robot, when the encounter occurs. Approximation of the workbenches is influenced by the robot in ten of 15 cases. Furthermore, an increase of participants' walking velocity with higher robot velocities is observed.

Safety of Interactive Robotics—Learning from Accidents

Finland is ranked rather high in international robot density statistics. In Finland, robots are typically used in applications where they operate in close proximity to humans. The research described in this paper, sourced from Finnish databases, identified 25 severe accidents which can be attributed to robots. The current accident data can provide an insight into the type of accidents associated with future human-robot interaction (HRI) applications. Accident statistics indicate that most of the severe robot-related accidents involved crushing a person against a rigid object. As crushing hazards currently dominate accident statistics, and with HRI applications becoming increasingly common, humans are expected to be exposed to more crushing hazards in the future. The close proximity of the robots means that there is very little time to escape from crushing hazard. The prevention of collisions between robots and humans is paramount to reducing the amount of accidents. Actions to diminish the effects of any subsequent collision are also important. The control after a collision, however, needs to be very quick in order to minimise the damage caused by an impact. Current practice demands that upon detection of a collision, active movements are typically not allowed without a human supervision. Moving a robot away to a safe position and releasing any pressure against a person may save lives, but would entail some adjustments or new interpretations of the current safety requirements.