Safe Interaction Research Articles

A bionic soft tongue driven by shape memory alloy and pneumatics

Soft grippers have exhibited considerable advantages owing to their flexible deformation, compliant operation, and safe interaction with objects. The ability to grip solid and liquid objects can greatly expand the application range of the soft grippers. Dogs stick out their tongues and then curl them backward to form a ladle shape for eating food and drinking water. The large extension ratio and the ladling motion of the tongues endow dogs with flexible operations for both solids and liquids. In this work, inspired by both the extending and ladling motions of dog tongues, a bionic soft tongue with the capability of handling solid and liquid objects was designed. The bionic soft tongue was composed of a tongue base and a tongue tip, which were driven by pneumatics and shape memory alloy wires, respectively. The tongue base was capable of linearly elongating and contracting with a considerable scale, while the tongue tip could be curled into a ladle shape to grasp objects. The dynamic model of the tongue base was developed and then extended to build the model-based feedback controller for the motion control. A phase dynamic model of the tongue tip was simulated for structural optimization. With the model-based feedback controller, the bionic soft tongue could achieve a fast step response and precise position control. Experimental results showed that the bionic soft tongue could grasp different kinds of objects, including of water, rice, and gel balls. This work is expected to expand the application scope of soft grippers.

Visual Sensor Fusion Based Autonomous Robotic System for Assistive Drinking.

People with severe motor impairments like tetraplegia are restricted in activities of daily living (ADL) and are dependent on continuous human assistance. Assistive robots perform physical tasks in the context of ADLs to support people in need of assistance. In this work a sensor fusion algorithm and a robot control algorithm for localizing the user’s mouth and autonomously navigating a robot arm are proposed for the assistive drinking task. The sensor fusion algorithm is implemented in a visual tracking system which consists of a 2-D camera and a single point time-of-flight distance sensor. The sensor fusion algorithm utilizes computer vision to combine camera images and distance measurements to achieve reliable localization of the user’s mouth. The robot control algorithm uses visual servoing to navigate a robot-handled drinking cup to the mouth and establish physical contact with the lips. This system features an abort command that is triggered by turning the head and unambiguous tracking of multiple faces which enable safe human robot interaction. A study with nine able-bodied test subjects shows that the proposed system reliably localizes the mouth and is able to autonomously navigate the cup to establish physical contact with the mouth.

VIRTUAL SATISFACTION AS A SYMPTOM OF INTERPASSIVITY IN MODERN SOCIETY

Research into how new media are transforming our daily lives is the frontier of modern philosophical and scientific thought. It is difficult to reflect on what is happening to society and to the person at the moment; however, for all the pace of social transformations today, we can still talk about potential risks and opportunities offered by new technologies. The article introduces the concept of virtual satisfaction and examines its connection with the phenomenon of interpassivity which is well known in philosophical discourse, as well as with manifestations of visual normalization in society. The aim of the study is to trace the mechanism of involvement of humans in interpassive practices of a virtualized society through consumption of media images. This environment acts as a virtual link between social reality and the needs of the individual which can only be satisfied with the mediation of the Other. On the one hand, this is the world of safe interactions and quasi-interactions where satisfaction is obtained in screen non-contact forms. On the other hand, this satisfaction turns out to be interpassive, depriving us not only of realization of desire, but also of desire itself, so that people voluntarily lose part of themselves, and therefore part of their humanity.

Optical Force/Tactile Sensors for Robotic Applications

Nowadays, robotic systems use tactile sensing as a key enabling technology to implement complex tasks. For example, manipulation and grasping problems strongly depend on the physical and geometrical characteristics of the objects; in fact, objects may be deformable or change their shape when in contact with the robot or the environment. For this reason, often, robots end effectors are equipped with sensorized fingers which can estimate the objects' features, forces, and contact locations. This is useful in a safe and efficient physical Human-Robot Interaction (pHRI) to perform cooperation and co-manipulation tasks while limiting damage from accidental impacts.

Human-Robot Cooperation for Autonomous Vehicles and Human Drivers: Challenges and Solutions

Connected and automated vehicles (CAVs) are becoming increasingly feasible, and these vehicles are expected to enhance road safety, traffic throughput, energy efficiency, and human comfort, especially at road intersections and for lane changing, merging, and unprotected turning maneuvers. However, there is likely to be a long transition period before human-driven vehicles are completely replaced by such automated vehicles. To leverage the autonomous driving technologies on public roads, we have to design and develop safe interaction and cooperation for mixed traffic environments, where human-driven and autonomous vehicles cooperate with one another in order to avoid vehicle collisions and possible deadlocks. In this article, we study the requirements and challenges for such safe cooperation of autonomous vehicles and human drivers, and review three prospective components: road infrastructure, vehicular communications, and online recognition and prediction. First, road infrastructure contributes to improving CAV capabilities for safe cooperation as a centralized coordinator. In addition, vehicle-to-vehicle communications might be key to safe cooperation in a decentralized manner that can guarantee road safety for multiple connected vehicles. Online motion prediction and real-time gesture recognition are also essential to develop a safe cooperation system in a decentralized manner with non-connected vehicles. These three approaches are not exclusive, and CAVs may complementarily use them in practice. Finally, we discuss the feasibility and limitations of each approach and conclude with future research directions.

The Pransky interview: Dr Nabil Simaan, Vanderbilt University Professor of Mechanical Engineering, Computer Science and Otolaryngology, Thought Leader in Medical Robotics

Purpose The following article is a “Q&A interview” conducted by Joanne Pransky of Industrial Robot Journal as a method to impart the combined technological, business and personal experience of a prominent, robotic industry PhD and innovator regarding his pioneering efforts. The paper aims to discuss these issues. Design/methodology/approach The interviewee is Dr Nabil Simaan, Professor of Mechanical Engineering, Computer Science and Otolaryngology at Vanderbilt University. He is also director of Vanderbilt’s Advanced Robotics and Mechanism Applications Research Laboratory. In this interview, Simaan shares his unique perspective and approaches on his journey of trying to solve real-world problems in the medical robotics area. Findings Simaan received his BSc, MSc and PhD in mechanical engineering from the Technion – Israel Institute of Technology. He served as Postdoctoral Research Scientist in Computer Science at Johns Hopkins University. In 2005, he joined Columbia University, New York, NY, as an Assistant Professor of Mechanical Engineering until 2010, when he joined Vanderbilt. His current applied research interests include synthesis of novel robotic systems for surgical assistance in confined spaces with applications to minimally invasive surgery of the throat, natural orifice surgery, cochlear implant surgery and dexterous bimanual microsurgery. Theoretical aspects of his research include robot design and kinematics. Originality/value Dr Simaan is a leading pioneer on designing robotic systems and mechanisms for medical applications. Examples include technologies for snake robots licensed to Intuitive Surgical; technologies for micro-surgery of the retina, which led to the formation of AURIS Surgical Robotics; the insertable robotic effector platform (IREP) single-port surgery robot that served as the research prototype behind the Titan Medical Inc. Sport (Single Port Orifice Robotic Technology). Simaan received the NSF Career award for young investigators to design new algorithms and robots for safe interaction with the anatomy. He has served as the Editor for IEEE International Conference on Robotics and Automation, Associate Editor for IEEE Transactions on Robotics, Editorial Board Member of Robotica, Area Chair for Robotics Science and Systems and corresponding Co-chair for the IEEE Technical Committee on Surgical Robotics. In January 2020, he was bestowed the award of Institute of Electrical and Electronics Engineers (IEEE) Fellow for Robotics Advancements. At the end of 2020, he was named a top voice in health-care robotics by technology discovery platform InsightMonk and market intelligence firm BIS Research. Simaan holds 15 patents. A producer of human capital, his education goal is to achieve the best possible outcome with every student he works with.

Impact of External Human–Machine Interface Communication Strategies of Automated Vehicles on Pedestrians’ Crossing Decisions and Behaviors in an Urban Environment

The development of automated vehicles (AVs) and their integration into traffic are seen by many vehicle manufacturers and stakeholders such as cities or transportation companies as a revolution in mobility. In future urban traffic, it is more likely that AVs will operate not in separated traffic spaces but in so-called mixed traffic environments where different types of traffic participants interact. Therefore, AVs must be able to communicate with other traffic participants, e.g., pedestrians as vulnerable road users (VRUs), to solve ambiguous traffic situations. To achieve well-working communication and thereby safe interaction between AVs and other traffic participants, the latest research discusses external human–machine interfaces (eHMIs) as promising communication tools. Therefore, this study examines the potential positive and negative effects of AVs equipped with static (only displaying the current vehicle automation status (VAS)) and dynamic (communicating an AV’s perception and intention) eHMIs on the interaction with pedestrians by taking subjective and objective measurements into account. In a Virtual Reality (VR) simulator study, 62 participants were instructed to cross a street while interacting with non-automated (without eHMI) and automated vehicles (equipped with static eHMI or dynamic eHMI). The results reveal that a static eHMI had no effect on pedestrians’ crossing decisions and behaviors compared to a non-automated vehicle without any eHMI. However, participants benefit from the additional information of a dynamic eHMI by making earlier decisions to cross the street and higher certainties regarding their decisions when interacting with an AV with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. Implications for a holistic evaluation of eHMIs as AV communication tools and their safe introduction into traffic are discussed based on the results.

Introducing the Microbes and Social Equity Working Group: Considering the Microbial Components of Social, Environmental, and Health Justice.

ABSTRACTHumans are inextricably linked to each other and our natural world, and microorganisms lie at the nexus of those interactions. Microorganisms form genetically flexible, taxonomically diverse, and biochemically rich communities, i.e., microbiomes that are integral to the health and development of macroorganisms, societies, and ecosystems. Yet engagement with beneficial microbiomes is dictated by access to public resources, such as nutritious food, clean water and air, safe shelter, social interactions, and effective medicine. In this way, microbiomes have sociopolitical contexts that must be considered. The Microbes and Social Equity (MSE) Working Group connects microbiology with social equity research, education, policy, and practice to understand the interplay of microorganisms, individuals, societies, and ecosystems. Here, we outline opportunities for integrating microbiology and social equity work through broadening education and training; diversifying research topics, methods, and perspectives; and advocating for evidence-based public policy that supports sustainable, equitable, and microbial wealth for all.

Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function

Aiming at human-robot collaboration in manufacturing, the operator's safety is the primary issue during the manufacturing operations. This paper presents a deep reinforcement learning approach to realize the real-time collision-free motion planning of an industrial robot for human-robot collaboration. Firstly, the safe human-robot collaboration manufacturing problem is formulated into a Markov decision process, and the mathematical expression of the reward function design problem is given. The goal is that the robot can autonomously learn a policy to reduce the accumulated risk and assure the task completion time during human-robot collaboration. To transform our optimization object into a reward function to guide the robot to learn the expected behaviour, a reward function optimizing approach based on the deterministic policy gradient is proposed to learn a parameterized intrinsic reward function. The reward function for the agent to learn the policy is the sum of the intrinsic reward function and the extrinsic reward function. Then, a deep reinforcement learning algorithm intrinsic reward-deep deterministic policy gradient (IRDDPG), which is the combination of the DDPG algorithm and the reward function optimizing approach, is proposed to learn the expected collision avoidance policy. Finally, the proposed algorithm is tested in a simulation environment, and the results show that the industrial robot can learn the expected policy to achieve the safety assurance for industrial human-robot collaboration without missing the original target. Moreover, the reward function optimizing approach can help make up for the designed reward function and improve policy performance.

Surveillance and privacy – Beyond the panopticon. An exploration of 720-degree observation in level 3 and 4 vehicle automation

On the path to high-level vehicle automation, the degree of surveillance both inside and outside the car increases significantly. Consequently, ethical considerations are becoming central to questions around surveillance regimes and data privacy implicit in level 3 and 4 vehicle automation. In this paper, we focus on outputs from the EU Horizon 2020 project Vision Inspired Driver Assistance Systems (VI-DAS). In particular, we assess the VI-DAS 720-degree observation technology, critical to ensuring a safe Human Machine Interaction (HMI), from multiple theoretical perspectives to contribute to a better understanding of the phenomena of privacy. As a synonym for surveillance, we started our evaluation with Bentham's ideation of the panopticon. From there, it is a relatively short step to radical Foucauldian critiques that offered more dystopian technologies of power. However, both theorems demonstrate a limited understanding of the issue of data privacy in the context of safe transportation along the evolution of highly automated vehicles. Thus, to allow the debate to move beyond more binary discussions on privacy versus control/power and to a certain degree escape the shadow of the panopticon, we applied the Nissenbaum four theses framework of Contextual Integrity (CI). Her decision heuristic allowed us to introduce structure and a degree of precision in our thinking on the matter of privacy that represents a step forward to phenomena of privacy in a specific context. Our approach concludes that the VI-DAS 720-degree observation technology can respect the user's privacy through an appropriate flow of personal information. However, the flows of personal data must be strongly regulated to ensure that data is seen as a value in terms of a commodity to protect human life and not seen as an asset that needs to be turned into value in terms of capital or the facilitation of asymmetric power-relations.

A bilateral teleoperation with interaction force constraint in unknown environment using non linear model predictive control

In critical scenarios, the interaction forces between a robot and the environment could lead to damages and dangerous situations. Complex tasks like grasping fragile objects or physical human-robot interaction in collaborative robotics require the capability of controlling forces. In bilateral teleoperation, force feedback is used to provide telepresence to the operator. In such situation, the force is commonly measured by a force/torque sensor at the end effector of the remote robot. Even if force feedback allows the operator to feel the interaction with the environment this does not prevent unsafe motion. In this paper, we propose a model predictive control (MPC) based bilateral teleoperation able to guarantee safe interaction with the environment by constraining the forces. The method does not assume any prior knowledge of the environment.

Micrometer Positioning Accuracy With a Planar Parallel Continuum Robot.

Parallel Continuum Robots (PCR) have several advantages over classical articulated robots, notably a large workspace, miniaturization capabilities and safe human-robot interactions. However, their low accuracy is still a serious drawback. Indeed, several conditions have to be met for PCR to reach a high accuracy, namely: a repeatable mechanical structure, a correct kinematic model, and a proper estimation of the model’s parameters. In this article, we propose a methodology that allows reaching a micrometer accuracy with a PCR. This approach emphasizes the importance of using a repeatable continuum mechanism, identifying the most influential parameters of an accurate kinematic model of the robot and precisely measuring them. The experimental results show that the proposed approach allows to reach an accuracy of 3.3 µm in position and 0.5 mrad in orientation over a 10 mm long circular path. These results push the current limits of PCR accuracy and make them good potential candidates for high accuracy automatic positioning tasks.

Scaling up urban infrastructure for physical activity in the COVID-19 pandemic and beyond

Scaling up urban infrastructure for physical activity in the COVID-19 pandemic and beyond

Introducing GARMI - A Service Robotics Platform to Support the Elderly at Home: Design Philosophy, System Overview and First Results

This letter introduces GARMI, a service robotics platform conceptualized with a focus on assisting elderly at home. GARMI is designed to provide support with household tasks, as an avatar for tactile-enabled communication and as an interface for telemedicine and emergency assistance. Its unique humanoid design features a sensor-equipped multi-modal head designed for natural human-machine communication as well as a whole-body torque-control interface for safe physical human-machine interaction. GARMI's modular software architecture and distinctive whole-body control scheme allows multimodal dynamic coupling. Additionally, every system component can actively produce as well as sense forces and can thus serve as a haptic feedback interface when interacting with its environment and users. Furthermore, GARMI is the first mobile humanoid designed with specialized use-inspired avatar stations: PARTI for dual-arm-based exoskeleton-like remote-control with force-feedback and MUCKI for transparent remote doctor-patient interaction with both audiovisual and safe haptic feedback. The specialized software architecture allows for rapid prototyping and field-testing of new behaviors for telemedicine, multi-modal interaction and autonomous physical and service assistance. Our first results reveal the potential of our use-driven force-based whole-body control mobile humanoid for daily living and telemedicine scenarios in an elderly care research facility.

Virtual Radar: Real-Time Millimeter-Wave Radar Sensor Simulation for Perception-Driven Robotics

This article presents ViRa <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Available online: https://vira.aau.at/., a real-time open-source millimeter-wave radar simulation framework for perception-driven robotic applications. ViRa provides (i) raw data of radar sensors in real-time, simulation of (ii) multi-antenna configurations for spatial estimation of objects, (iii) wave penetration of non-conductive objects to infer information in occluded situations, (iv) different radar beam patterns, and, (v) configurations of radar sensors as given by real-world radars. By using ViRa, researchers can simulate radar sensors in different robotic scenarios and investigate radars prior to the installation. This allows an acceleration in the development of radar sensors for robotic applications without the need of real hardware. Contrary to simple model abstractions, which only output loose features, ViRa generates raw radar data using computer graphics techniques on graphics processing unit (GPU) level embedded inside a game engine environment. ViRa allows to feed data directly into machine learning frameworks, which enables further improvement in novel research directions, such as safe human-robot interaction or agile drone flights in obstacle-rich environments. The proposed simulation framework is validated with data from different scenarios in robotics such as human tracking for human-robot interaction. The obtained results are compared with a reference simulation framework and show significantly higher correlation when compared to real-world measurement data.

Visual Servoing of a Cable-Driven Soft Robot Manipulator With Shape Feature

Soft continuum robot is of superiority in the applications in unstructured environments due to its high flexibility and safe interaction ability. Accurate shape control is regarded as one of the prerequisites to improve its practicability. Considering the situation where the 3D position signals are not available, this letter investigates the vision-based shape control scheme of a soft robot manipulator. The curvature features used to depict the reference image shape of the robot backbone curve are solved through nonlinear optimization with given constraints of two endpoints in pixel coordinates. Thereafter, an adaptive image-based visual servoing controller is designed to track the reference image shape features with an uncalibrated monocular camera. The proposed algorithm was tested on an eight-cable driving soft robot prototype and proved its validity in convergence to the desired image shape.

Considering Human Variability in the Design of Safe Interaction with Agricultural Machinery: The Case of Foldable Roll-Over Protective Structure (FROPS) Manual Handling

The foldable roll-over protective structure (FROPS) protects the operators against fatal injuries in tractor roll-over accidents. However, a rear-mounted FROPS is often folded down or removed. In the present study, the accessible zones and grasping areas in a rear-mounted FROPS were redesigned and adapted to the 5th, 50th, and 95th European human anthropometric percentiles to enhance its correct and comfortable use. Then, a rod was proposed as a design solution to make the roll-bar grasping areas fall within the new accessible zones. The rod prototype increased roll-bar reachability and facilitated the raising handling, especially for shorter users. The present study results and the accessible zones redesigned, taking into account the human percentiles, will be helpful in rethinking reachability issues in manual handling of machinery components, to support the correct behaviours, and make human-machine interaction more comfortable and safer for all.

Flying robot path planning techniques and its trends

A Flying robot is a system capable of vertical take-off and landing to perform some specific task with no direct human intervention. Flying robots are distinct from other robots by their ability to fly with no direct human control and are capable of making a decision based on the situation. Drones, Aerial robots, Unmanned Aerial Vehicles (UAV) are few examples of flying robots. Path determination of flying robots is one of the most critical aspects of robot Routing design. Path planning is a technique which tells flying robot how to fly, where to fly and to find a collision-free optimal path. Numerous techniques have been put forward over the past few years to find an optimal path for flying robots. This work discusses on new trends led by artificial intelligence and safe human-robot interaction for optimal path planning of aerial robots. Furthermore, a comprehensive study is carried out on various 2D and 3D path planning algorithms done to date and different algorithms are classified based on important criteria for optimal path planning for flying robots.

A 4-DOF Upper Limb Exoskeleton for Physical Assistance: Design, Modeling, Control and Performance Evaluation

Wheelchair mounted upper limb exoskeletons offer an alternative way to support disabled individuals in their activities of daily living (ADL). Key challenges in exoskeleton technology include innovative mechanical design and implementation of a control method that can assure a safe and comfortable interaction between the human upper limb and exoskeleton. In this article, we present a mechanical design of a four degrees of freedom (DOF) wheelchair mounted upper limb exoskeleton. The design takes advantage of non-backdrivable mechanism that can hold the output position without energy consumption and provide assistance to the completely paralyzed users. Moreover, a PD-based trajectory tracking control is implemented to enhance the performance of human exoskeleton system for two different tasks. Preliminary results are provided to show the effectiveness and reliability of using the proposed design for physically disabled people.

Modeling and motion control of an octopus-like flexible manipulator actuated by shape memory alloy wires

Soft robots are concerned by researchers due to several characteristics, such as high adaptability in complex unstructured environments, safe interaction with environments, and a high degree of dexterity. The deformability and dexterity of the octopus arm are remarkable and interesting for the development of bioinspiration soft robots. In this study, we proposed a biomimetic flexible manipulator actuated by shape memory alloy (SMA) wires and its PI controller, and evaluated its performances with focused experiments. This study aims to find out the advantages of the developed manipulator actuated by SMA wires. We designed the bionic structure to achieve flexible bending in 3D space based on the analysis of the octopus muscle structure. Then we built the mathematical models that described the relationship between the bending angle and the driving parameters. Surprisingly, the soft movement capacity of the bionic manipulator was investigated at different heating voltages and PWM duty cycles by the PI controller based on the self-sensing resistance feedback. The soft manipulator was able to bend flexibly with the maximum bending angle of 60°. Compared to the traditional soft arm, the soft manipulator has higher accuracy, with the error of the deflection angle less than 5° and the errors of bending angles less than 2°.