Intuitive Human-robot Interaction Research Articles

Digital K–12 STEM Education through Human–Robot Interaction: Investigation on Prerequisites

This article aims to explore, investigate, and determine the prerequisites that learners (students) should possess for participating in and being adequately benefitted from digital (robotics-enabled) K–12 STEM education offered through intuitive human–robot interaction. We selected 23 middle school mathematics and science teachers who received training on how to design, develop, and implement robotics-enabled lessons. The teachers then implemented robotics-enabled lessons in actual classroom settings, and separately responded to a survey based on their training, classroom experiences and observations, and self-brainstorming. We derived a set of prerequisite knowledge, skills, and abilities, including their relative importance for the students by analyzing the survey responses. The results showed that the students should not only possess prerequisite knowledge in the subject matter, but also possess behavioral, social, scientific, cognitive, and intellectual skills and abilities to participate in and receive benefits from robotics-enabled human–robot interactive digital STEM education. Out of the many prerequisites, the computational thinking ability of students was identified as one of the most required prerequisites to participate in robotics-enabled digital STEM education. To validate the derived prerequisites, teachers separately assessed the fulfillment of prerequisites by 38 participating students, and the results showed user acceptance, effectiveness, and suitability of the derived prerequisites set. We also identified a set of limitations of the studies and proposed action plans to enable students to meet the prerequisites. The results presented herein can help determine required instructional efforts and scaffolds before implementing robotics-enabled digital STEM lessons, and thus foster incorporating technology-enhanced (robotics-enabled) digital STEM education into K–12 curricula.

Multimodal Interaction Strategies for Walker-Assisted Gait: A Case Study for Rehabilitation in Post-Stroke Patients

Stroke has been considered the main cause of neuromuscular damages worldwide and one of the most common causes of walking disabilities, with approximately 60% of the individuals suffering from persistent problems in walking. These patients generally use technical aids for walking to achieve independent gait, however, when cognitive impairments are also present, conventional assistive devices such as walkers could be difficult to handle. By leveraging multimodal interfaces, smart walkers can offer natural and intuitive human-robot interaction. In this work, we present two multimodal interaction strategies for smart walkers focusing on guiding post-stroke patients through their environment. These strategies leverage different communication channels and provide distinct levels of guidance: one strategy uses haptic feedback and a visual interface to indicate the desired path to the user, while the other strategy uses haptic feedback and a virtual torque to maintain the user on path. We also present two case studies with post-stroke patients to preliminarily validate these interaction strategies with their target population and to collect valuable insight as to how multimodal strategies for smart walkers can be enhanced to deal with the characteristic asymmetries of post-stroke patients. Our results show that both strategies can guide the volunteers, however, the first one demands more effort from the volunteer and is more suited for patients with increased levels of independence. The second interaction strategy allows for higher linear velocity (Volunteer 1, 0.18\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{0.18}$$\\end{document}±0.026\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\pm 0.026}$$\\end{document}m/s\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m/s}$$\\end{document}; Volunteer 2, 0.22\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{0.22}$$\\end{document}±0.0283\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\pm 0.0283}$$\\end{document}m/s\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m/s}$$\\end{document}) than the first one (Volunteer 1, 0.10\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{0.10}$$\\end{document}±0.031\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\pm 0.031}$$\\end{document}m/s\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m/s}$$\\end{document}; Volunteer 2, 0.20\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{0.20}$$\\end{document}±0.012\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\pm 0.012}$$\\end{document}m/s\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m/s}$$\\end{document}), suggesting improved guidance.

Physical Human–Robot Interaction Using a Macro–Mini Robotic System

In this article, we investigate the characteristics of a collaborative hybrid parallel robot. A comparison between this robot and other collaborative robots regarding the Cartesian inertia is presented to highlight the very low inertia of the hybrid parallel robot. A task space stiffness-damping control is also proposed. In order to improve the performance of macro--mini systems in the interaction context, an active macro-active mini system consisting of the hybrid parallel robot mounted on a three degrees of freedom (3-DOF) gantry translational robot is introduced. A control strategy for physical human–robot interaction is applied to the macro--mini arrangement. The stability of this method is analyzed and it is shown that the macro–mini combination is more stable than the mini alone. An experimental validation is then carried out. The results obtained with the hybrid parallel robot show that the desired interaction force can be tracked at high speed and that prescribed impedance parameters can be precisely rendered to the human operator without using any additional sensor. Also, it is pointed out that the bandwidth of interaction of the hybrid parallel robot, as well as of the macro--mini system is much higher than that of other commercial collaborative robots. In addition, other tests are realized on the macro–mini system in order to verify the performance and to demonstrate potential applications. The macro–mini robot introduced in this work yields a very intuitive human–robot interaction, which makes it ideal for many applications in which direct physical teaching or assistance is needed.

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

A speech-enabled virtual assistant for efficient human–robot interaction in industrial environments

This paper presents a natural language-enabled virtual assistant (VA), named Max, developed to support flexible and scalable human–robot interactions (HRI) with industrial robots. Regardless of the numerous natural language interfaces already proposed for intuitive HRI on the industrial shop floor, most of those interfaces remain tightly bound with a specific robotic system. Besides, the lack of a natural and efficient human–robot communication protocol hinders the user experience. Therefore three key elements characterize the proposed framework. First, a Client–Server style architecture is introduced so Max can provide a centralized solution for managing and controlling various types of robots deployed on the shop floor. Second, inspired by human–human communication, two conversation strategies, lexical-semantic and general diversion strategies, are used to guide Max’s response generation. These conversation strategies were embedded to improve the operator’s engagement with the manufacturing tasks. Third, we fine-tuned the state-of-the-art (SOTA) pre-trained model, Bidirectional Encoder Representations from Transformers (BERT), to support a highly accurate prediction of requested intents from the operator and robot services. Multiple experiments were conducted using the latest iteration of our autonomous industrial mobile manipulator, “Little Helper (LH)”, to validate Max’s performance in a real manufacturing environment.

Towards autonomous ergonomic upper-limb exoskeletons: A computational approach for planning a human-like path

Computational path planning approaches can enable development of autonomous rehabilitation and assistive exoskeletons. Using a human-like reference behavior for such wearable systems can ensure safe, effective, and intuitive human–robot interaction. This is of significant importance since the quality of interaction and ergonomic considerations have a substantial effect on technology usability and acceptance by the users. This paper proposes a novel framework for generating human-like paths for wearable exoskeletons in the shoulder-elbow level. The introduced method is a two-stage process where a human-like reference path is planned in the configuration space of the human arm, followed by an analytical transformation that directly maps the derived path to the configuration space of the exoskeleton. The analytical mapping presented is a function of the kinematic parameters of the system and can be adapted for other upper-limb exoskeletons. As a case study, the proposed method is used for generating human-like reference motions for a six-degree-of-freedom exoskeleton supporting scapulohumeral rhythm, glenohumeral rotations, and elbow flexion/extension. Firstly, it is shown that reaching motions associated with activities of daily living can be predicted with high accuracy in the human joint space. This is demonstrated by analyzing the experimental data collected from healthy subjects. Subsequently, it is verified through kinematic analysis that the transformation of generated paths to the exoskeleton configuration space does not alter their spatial profile in the task space.

A Human-like Upper-limb Motion Planner: Generating naturalistic movements for humanoid robots

As robots are starting to become part of our daily lives, they must be able to cooperate in a natural and efficient manner with humans to be socially accepted. Human-like morphology and motion are often considered key features for intuitive human–robot interactions because they allow human peers to easily predict the final intention of a robotic movement. Here, we present a novel motion planning algorithm, the Human-like Upper-limb Motion Planner, for the upper limb of anthropomorphic robots, that generates collision-free trajectories with human-like characteristics. Mainly inspired from established theories of human motor control, the planning process takes into account a task-dependent hierarchy of spatial and postural constraints modelled as cost functions. For experimental validation, we generate arm-hand trajectories in a series of tasks including simple point-to-point reaching movements and sequential object-manipulation paradigms. Being a major contribution to the current literature, specific focus is on the kinematics of naturalistic arm movements during the avoidance of obstacles. To evaluate human-likeness, we observe kinematic regularities and adopt smoothness measures that are applied in human motor control studies to distinguish between well-coordinated and impaired movements. The results of this study show that the proposed algorithm is capable of planning arm-hand movements with human-like kinematic features at a computational cost that allows fluent and efficient human–robot interactions.

SMILE: A verbal and graphical user interface tool for speech-control of soccer robots in Ghana

SMILE (Smartphone Intuitive Likeness and Engagement) application, a portable Android application that allows a human to control a robot using speech input. SMILE is a novel open source and platform independent tool that will contribute to the robot soccer research by allowing robot handlers to verbally command robots. The application resides on a smartphone embedded in the face of a humanoid robot, using a speech recognition engine to analyze user speech input while using facial expressions and speech generation to express comprehension feedback to the user. With the introduction of intuitive human robot interaction into the arena of robot soccer, we discuss a couple specific scenarios in which SMILE could improve both the pace of the game and autonomous appearance of the robots. The ability of humans to communicate verbally is essential for any cooperative task, especially fast-paced sports. In the game of soccer, players must speak with coaches, referees, and other players on either team. Therefore, if humanoids are expected to compete on the same playing field as elite soccer players in the near future, then we must expect them to be treated like humans, which include the ability to listen and converse. SMILE (Smartphone Intuitive Likeness and Engagement) is the first platform independent smartphone based tool to equip robots with these capabilities. Currently, humanoid soccer research is heavily focused on walking dynamics, computer vision, and intelligent systems; however human-robot interaction (HRI) is overlooked. We delved into this area of robot soccer by implementing SMILE, an Android application that sends data packets to the robot's onboard computer upon verbal interaction with a user.

Robots and Wizards: An Investigation Into Natural Human–Robot Interaction

The goal of the study was to research different communication modalities needed for intuitive Human-Robot Interaction. This study utilizes a Wizard of Oz prototyping method to enable a restriction-free, intuitive interaction with an industrial robot. The data from 36 test subjects suggests a high preference for speech input, automatic path planning and pointing gestures. The catalogue developed during this experiment contains intrinsic gestures suggesting that the two most popular gestures per action can be sufficient to cover the majority of users. The system scored an average of 74% in different user interface experience questionnaires, while containing forced flaws. These findings allow a future development of an intuitive Human-Robot interaction system with high user acceptance.

Online Recognition of Incomplete Gesture Data to Interface Collaborative Robots

Online recognition of gestures is critical for intuitive human-robot interaction (HRI) and further push collaborative robotics into the market, making robots accessible to more people. The problem is that it is difficult to achieve accurate gesture recognition in real unstructured environments, often using distorted and incomplete multisensory data. This paper introduces an HRI framework to classify large vocabularies of interwoven static gestures (SGs) and dynamic gestures (DGs) captured with wearable sensors. DG features are obtained by applying data dimensionality reduction to raw data from sensors (resampling with cubic interpolation and principal component analysis). Experimental tests were conducted using the UC2017 hand gesture dataset with samples from eight different subjects. The classification models show an accuracy of 95.6% for a library of 24 SGs with a random forest and 99.3% for 10 DGs using artificial neural networks. These results compare equally or favorably with different commonly used classifiers. Long short-term memory deep networks achieved similar performance in online frame-by-frame classification using raw incomplete data, performing better in terms of accuracy than static models with specially crafted features, but worse in training and inference time. The recognized gestures are used to teleoperate a robot in a collaborative process that consists in preparing a breakfast meal.

A Virtual Reality Interface for an Autonomous Spray Painting UAV

PaintCopter is an autonomous unmanned aerial vehicle (UAV) capable of spray painting on complex three-dimensional (3D) surfaces. This letter aims to make PaintCopter more user-friendly and to enable more intuitive human–robot interaction. We propose a virtual reality interface that allows the user to immerse in a virtual environment, navigate around the target surface, and paint at desired locations using a virtual spray gun. A realistic paint simulator provides a real-time previsualization of the painting activity that can either be processed right away or stored to a disk for later execution. An efficient optimization based planner uses this information to plan the painting task and execute it. The proposed planner maximizes the paint quality while respecting the spray nozzle constraints and platform dynamics. Our experiments show that the interface allows the user to make precise modifications to the target surface. Finally, we demonstrate the use of virtual reality interface to define a painting mission, and then the PaintCopter carrying out the mission to paint a desired multicolored pattern on a 3D surface.

Variable Admittance Control of Robot Manipulators Based on Human Intention

This paper presents a variable admittance control method to achieve intuitive human–robot interactions that consider human intentions. Human intention is classified into two categories—direct and indirect. With respect to direct intention, the concept of standard force is introduced to adjust the interacting force. The proposed variable admittance control method improves intuitiveness when velocity is used as an estimate of direct intention. In the estimation of indirect intention, a force guidance method is suggested to make a robot follow and guide a human. The proposed control methodology is adapted to a six-DOF manipulator based on a one-dimensional analysis. The experiments are conducted with a manipulator (Universal Robots, UR10) and a force/torque sensor (Robotus, RFT60-HA) to evaluate the performance. The experiments validate that variable admittance control enhances the execution time, accuracy, and comfort of the operator.

Discovery and recognition of motion primitives in human activities.

We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the ‘motion flux’, a quantity which captures the motion variation of a group of skeletal joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis.

Development of Flexible Robot Skin for Safe and Natural Human⁻Robot Collaboration.

For industrial manufacturing, industrial robots are required to work together with human counterparts on certain special occasions, where human workers share their skills with robots. Intuitive human–robot interaction brings increasing safety challenges, which can be properly addressed by using sensor-based active control technology. In this article, we designed and fabricated a three-dimensional flexible robot skin made by the piezoresistive nanocomposite based on the need for enhancement of the security performance of the collaborative robot. The robot skin endowed the YuMi robot with a tactile perception like human skin. The developed sensing unit in the robot skin showed the one-to-one correspondence between force input and resistance output (percentage change in impedance) in the range of 0–6.5 N. Furthermore, the calibration result indicated that the developed sensing unit is capable of offering a maximum force sensitivity (percentage change in impedance per Newton force) of 18.83% N−1 when loaded with an external force of 6.5 N. The fabricated sensing unit showed good reproducibility after loading with cyclic force (0–5.5 N) under a frequency of 0.65 Hz for 3500 cycles. In addition, to suppress the bypass crosstalk in robot skin, we designed a readout circuit for sampling tactile data. Moreover, experiments were conducted to estimate the contact/collision force between the object and the robot in a real-time manner. The experiment results showed that the implemented robot skin can provide an efficient approach for natural and secure human–robot interaction.

Gesture-based human-robot interaction for human assistance in manufacturing

The paradigm for robot usage has changed in the last few years, from a scenario in which robots work isolated to a scenario where robots collaborate with human beings, exploiting and combining the best abilities of robots and humans. The development and acceptance of collaborative robots is highly dependent on reliable and intuitive human-robot interaction (HRI) in the factory floor. This paper proposes a gesture-based HRI framework in which a robot assists a human co-worker delivering tools and parts, and holding objects to/for an assembly operation. Wearable sensors, inertial measurement units (IMUs), are used to capture the human upper body gestures. Captured data are segmented in static and dynamic blocks recurring to an unsupervised sliding window approach. Static and dynamic data blocks feed an artificial neural network (ANN) for static, dynamic, and composed gesture classification. For the HRI interface, we propose a parameterization robotic task manager (PRTM), in which according to the system speech and visual feedback, the co-worker selects/validates robot options using gestures. Experiments in an assembly operation demonstrated the efficiency of the proposed solution.

Intelligent Control for Human-Robot Cooperation in Orthopedics Surgery.

Cooperation between surgeon and robot is one of the key technologies that limit the robot to be widely used in orthopedic clinics. In this study, the evolution of human-robot cooperation methods and the control strategies for typical human-robot cooperation in robot-assisted orthopedics surgery were reviewed at first. Then an intelligent admittance control method, which combines the fuzzy model reference learning control with the virtual constraint control, is proposed to solve the requirements of intuitive human-robot interaction during orthopedics surgery. That is, a variable damping parameter model of the admittance control based on fuzzy model learning control algorithm is introduced to make the robot move freely by using the reference model of surgeon's motion equation with the minimum jerk trajectory. And the virtual constraint control method based on the principle of virtual fixture is adopted to make the robot move within the pre-defined area so as to perform more safe surgery. The basic principle and its realization of this intelligent control method are described in details. At last, a test platform is built based on our designed 6 DOF articulated robot. Experiments of safety and precision on acrylic model with this method show that the robot has the ability of better intuitive interaction and the high precision. And the pilot experiment of bone tumor resection on sawbone model shows the effectiveness of this method.

Implicit Intention Communication in Human–Robot Interaction Through Visual Behavior Studies

The emergence of assistive robots presents the possibility of restoring vital degrees of independence to the elderly and impaired in activities of daily living (ADL). However, one of the main challenges is the lack of a means for effective and intuitive human–robot interaction (HRI). While humans can express their intentions in different ways (e.g., physical gestures or motions, or speech or language patterns), gaze-based implicit intention communication is still underdeveloped. In this study, a novel nonverbal implicit communication framework based on eye gaze is introduced for HRI. In this framework, a user's eye-gaze movements are proactively tracked and analyzed to infer the user's intention in ADL. Then, the inferred intention can be used to command assistive robots for proper service. The advantage of this framework is that gaze-based communication can be handled by most of the people, as it requires very little effort, and most of the elderly and impaired retain visual capability. This framework is expected to simplify HRI, consequently enhancing the adoption of assistive technologies and improving users’ independence in daily living. The testing results of this framework confirmed that a human's subtle gaze cues on visualized objects could be effectively used for human-intention communication. Results also demonstrated that the gaze-based intention communication is easy to learn and use. In this study, the relationship of visual behaviors with the mental process during human intention expression was studied for the first time to build a fundamental understanding of this process. These findings are expected to guide further design of accurate intention inference algorithms and intuitive HRI.

Mobile Mixed-Reality Interfaces That Enhance Human–Robot Interaction in Shared Spaces

Although user interfaces with gesture-based input and augmented graphics have promoted intuitive human-robot interactions (HRI), they are often implemented in remote applications on research-grade platforms requiring significant training and limiting operator mobility. This paper proposes a mobile mixed-reality interface approach to enhance HRI in shared spaces. As a user points a mobile device at the robot's workspace, a mixed-reality environment is rendered providing a common frame of reference for the user and robot to effectively communicate spatial information for performing object manipulation tasks, improving the user's situational awareness while interacting with augmented graphics to intuitively command the robot. An evaluation with participants is conducted to examine task performance and user experience associated with the proposed interface strategy in comparison to conventional approaches that utilize egocentric or exocentric views from cameras mounted on the robot or in the environment, respectively. Results indicate that, despite the suitability of the conventional approaches in remote applications, the proposed interface approach provides comparable task performance and user experiences in shared spaces without the need to install operator stations or vision systems on or around the robot. Moreover, the proposed interface approach provides users the flexibility to direct robots from their own visual perspective (at the expense of some physical workload) and leverages the sensing capabilities of the tablet to expand the robot's perceptual range.

Human Behavior and Hand Gesture Classification for Smart Human-robot Interaction

This paper presents an intuitive human-robot interaction (HRI) framework for gesture and human behavior recognition. It relies on a vision-based system as interaction technology to classify gestures and a 3-axis accelerometer for behavior classification (stand, walking, etc.). An intelligent system integrates static gesture recognition recurring to artificial neural networks (ANNs) and dynamic gesture recognition using hidden Markov models (HMM). Results show a recognition rate of 95% for a library of 22 gestures and 97% for a library of 6 behaviors. Experiments show a robot controlled using gestures in a HRI process.

A Conformable Force/Tactile Skin for Physical Human–Robot Interaction

In this letter, a new sensorized flexible skin has been used to enhance safety and intuitiveness of physical human–robot interaction (HRI) in applications where both intentional and unintentional contacts may occur. The new technological contribution with respect to other skin sensors consists of the capability of measuring both the position of the contact point and the three components of the applied force with high repeatability and accuracy. To show how this innovative technology enables the exploitation of control laws for intuitive HRI, two standard control strategies have been implemented to perform both manual guidance with multiple contact points and safe reaction in case of unintentional collision detection, at the same time. In both cases, an admittance control scheme with a second order kinematic control is adopted. A multipriority redundancy resolution strategy is implemented in the case of manual guidance. The experimental verification of the sensor capabilities is made using a patch of the skin installed on a link of a KUKA LWR4 robot.