State Of Robot Research Articles

Deep reinforcement learning for shared control of mobile robots

AbstractShared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance. However, prior work that seeks to find the optimal shared control ratio needs an accurate human model, which is usually challenging to obtain. In this study, the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient (DDPG) (TD3X)‐based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally. The robot's states, shared control ratio in the previous time step, and human's control input is used as inputs to the reinforcement learning (RL) agent, which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model. Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio. Furthermore, to accelerate the training process and protect the robot, a navigation demonstration policy and a safety guard are developed. A neural network (NN) structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed. In addition, an extended DAGGER (DAGGERX) human agent is developed for training the RL agent to reduce human workload. Robot simulations and experiments with humans in the loop are conducted. The results show that the DAGGERX human agent can simulate real human inputs in the worst‐case scenarios with a mean square error of 0.0039. Compared to the original TD3 agent, the TD3X‐based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment. The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment, while the performance is equally good in the complex environment because of the use of an advanced human agent. In the human subject tests, participants' average perceived workload was significantly lower in shared control than that in exclusively manual control (26.90 vs. 40.07, p = 0.013).

EnviRobots: How Human–Robot Interaction Can Facilitate Sustainable Behavior

Environmental psychology aims to study human behavior with regard to the environment and how psychological techniques can be used to motivate behavior change. We argue that these concepts can be applied to interactive robots designed for other tasks, which then enables them to encourage sustainability behaviors in humans. We first present a literature review on the current state of social robots that are used to encourage sustainable behaviors. We next present eight hypothetical scenarios which are informed by the progress that has already been made in social robots in sustainability, as well as notable gaps where further environmental psychological concepts could be utilized. These scenarios encompass possible robots that range from limited sensing and no manipulation capabilities, to more sophisticated sensing and no manipulation, to sophisticated sensing and manipulation capabilities. We present these scenarios in which human–robot interaction could potentially result in pro-environmental behavioral changes in humans as recommendations for robot designers interested in helping design social robots for sustainability.

Flexible IoT Gas Sensor Node for Automated Life Science Environments Using Stationary and Mobile Robots

In recent years the degree of automation in life science laboratories increased considerably by introducing stationary and mobile robots. This trend requires intensified considerations of the occupational safety for cooperating humans, since the robots operate with low volatile compounds that partially emit hazardous vapors, which especially do arise if accidents or leakages occur. For the fast detection of such or similar situations a modular IoT-sensor node was developed. The sensor node consists of four hardware layers, which can be configured individually regarding basic functionality and measured parameters for varying application focuses. In this paper the sensor node is equipped with two gas sensors (BME688, SGP30) for a continuous TVOC measurement. In investigations under controlled laboratory conditions the general sensors’ behavior regarding different VOCs and varying installation conditions are performed. In practical investigations the sensor node’s integration into simple laboratory applications using stationary and mobile robots is shown and examined. The investigation results show that the selected sensors are suitable for the early detection of solvent vapors in life science laboratories. The sensor response and thus the system’s applicability depends on the used compounds, the distance between sensor node and vapor source as well as the speed of the automation systems.

Applied energy optimization of multi-robot systems through motion parameter tuning

In this paper, an optimization method for energy reduction of robot stations is presented, including an evaluation on an industrial robot station. The problem is formulated as a convex mixed integer nonlinear optimization problem, where the objective is to reduce the energy use by finding the optimal execution time and execution order of the robot motions. A simulation model of the station is used to find simplified energy models of the robot motions, that is used to solve the optimization problem. The optimal execution times of the robot motions are realized by tuning parameters in the robot control system. Different types of parameter settings are tested, such as reduced acceleration and velocity. The optimal parameter settings are then implemented in robot code in a real four robot welding station. The result shows a 12% reduction in energy use, without extending the cycle time of the station. A validation of the energy models used to solve the optimization problem is also made, by comparing them with real energy measurements.

Patching interpretable And‐Or‐Graph knowledge representation using augmented reality

AbstractWe present a novel augmented reality (AR) interface to provide effective means to diagnose a robot's erroneous behaviors, endow it with new skills, and patch its knowledge structure represented by an And‐Or‐Graph (AOG). Specifically, an AOG representation of opening medicine bottles is learned from human demonstration and yields a hierarchical structure that captures the spatiotemporal compositional nature of the given task, which is highly interpretable for the users. Through a series of psychological experiments, we demonstrate that the explanations of a robotic system, inherited from and produced by the AOG, can better foster human trust compared to other forms of explanations. Moreover, by visualizing the knowledge structure and robot states, the AR interface allows human users to intuitively understand what the robot knows, supervise the robot's task planner, and interactively teach the robot with new actions. Together, users can quickly identify the reasons for failures and conveniently patch the current knowledge structure to prevent future errors. This capability demonstrates the interpretability of our knowledge representation and the new forms of interactions afforded by the proposed AR interface.

Probabilistic Modeling and Bayesian Filtering for Improved State Estimation for Soft Robots

State estimation is one of the key requirements in robot control, which has been achieved by kinematic and dynamic models combined with motion sensors in traditional robotics. However, it is challenging to acquire accurate proprioceptive information in soft robots due to relatively high noise levels and hysteretic responses of soft actuators and sensors. In this article, we propose a method of estimating real-time states of soft robots by filtering noisy output signals and including hysteresis in the models using a Bayesian network. This approach is useful in constructing a state observer for soft robot control when both the kinematic model of the actuator and the model of the sensor are used. In our method, we regard a hysteresis function as a conditional random process model. We then introduce a dynamic Bayesian network composed of the actuator and the sensor models of the target system using distribution hysteresis mapping. Finally, we show that solving a Bayesian filtering problem is equivalent to suboptimal state estimation of the soft system. This article describes two ways for defining modeling and filtering; one is by Gaussian process regression combined with an extended Kalman filter, and the other is based on variational inference with a particle filter. While the first approach relaxes the uncertainty level in modeling to Gaussian, the second approach illustrates a general probability distribution. We experimentally validate the proposed methods through real-time state estimation of a sensor-integrated soft robotic gripper. The result shows significant improvement in state estimation compared to conventional estimation methods.

Informing Real-Time Corrections in Corrective Shared Autonomy Through Expert Demonstrations

Corrective Shared Autonomy is a method where human corrections are layered on top of an otherwise autonomous robot behavior. Specifically, a Corrective Shared Autonomy system leverages an external controller to allow corrections across a range of task variables (e.g., spinning speed of a tool, applied force, path) to address the specific needs of a task. However, this inherent flexibility makes the choice of what corrections to allow at any given instant difficult to determine. This choice of corrections includes determining appropriate robot state variables, scaling for these variables, and a way to allow a user to specify the corrections in an intuitive manner. This letter enables efficient Corrective Shared Autonomy by providing an automated solution based on Learning from Demonstration to both extract the nominal behavior and address these core problems. Our evaluation shows that this solution enables users to successfully complete a surface cleaning task, identifies different strategies users employed in applying corrections, and points to future improvements for our solution.

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning

Modular robots have the potential for an unmatched ability to perform versatile and robust locomotion. However, designing effective and adaptive locomotion controllers for modular robots is challenging, resulting in a number of model-based methods that typically require various forms of prior knowledge. Deep reinforcement learning (DRL) provides a promising model-free approach for locomotion control by trial-and-error. However, current DRL methods often require extensive interaction data, hindering many possible applications. In this letter, a novel two-level hierarchical locomotion framework for modular quadrupedal robots is proposed. The approach combines a low-level central pattern generator (CPG)-based controller with a high-level neural network to learn a variety of locomotion tasks using DRL. The low-level CPG controller is pre-optimized to generate stable rhythmic walking gaits, while the high-level network is trained to modulate the CPG parameters for achieving task goals based on high-dimensional inputs, including the robot states and user commands. The proposed approach is employed on a simulated modular quadruped. With a limited amount of prior knowledge, the proposed method is demonstrated to be capable of learning a variety of locomotion skills such as velocity tracking, path following, and navigating to a target. Simulation results show that the proposed method can achieve higher sample efficiency than the model-free DRL method and are substantially more robust than the baseline methods to external disturbances and irregular terrain.

Distributed Consensus Algorithm for Nonholonomic Wheeled Mobile Robot

This paper focuses on consensus of the nonholonomic wheeled mobile robotic systems whose geometric center and centroid do not coincide. A consensus control algorithm for mobile robots based on the nonstandard chain systems is proposed. Firstly, coordinate transformation is used to transform the nonholonomic robotic systems into the nonstandard chain model. Then, a distributed cooperative control algorithm is designed, and the Lyapunov stability theorem and LaSalle invariance principle are used to prove that each state of the mobile robot is consensus. Finally, the effectiveness of the algorithm is proved through numerical simulation.

SA-SVM-Based Locomotion Pattern Recognition for Exoskeleton Robot

An exoskeleton robot is a kind of wearable mechanical instrument designed according to the shape and function of the human body. The main purpose of its design and manufacture is to enhance human strength, assist human walking and to help patients recover. The walking state of the exoskeleton robot should be highly consistent with the state of the human, so the accurate locomotion pattern recognition is the premise of the flexible control of the exoskeleton robot. In this paper, a simulated annealing (SA) algorithm-based support vector machine model is proposed for the recognition of different locomotion patterns. In order to improve the overall performance of the support vector machine (SVM), the simulated annealing algorithm is adopted to obtain the optimal parameters of support vector machine. The pressure signal measured by the force sensing resistors integrated on the sole of the shoe is fused with the position and pose information measured by the inertial measurement units attached to the thigh, shank and foot, which are used as the input information of the support vector machine. The max-relevance and min-redundancy algorithm was selected for feature extraction based on the window size of 300 ms and the sampling frequency of 100 Hz. Since the signals come from different types of sensors, normalization is required to scale the input signals to the interval (0,1). In order to prevent the classifier from overfitting, five layers of cross validation are used to train the support vector machine classifier. The support vector machine model was obtained offline in MATLAB. The finite state machine is used to limit the state transition and improve the recognition accuracy. Experiments on different locomotion patterns show that the accuracy of the algorithm is 97.47% ± 1.16%. The SA-SVM method can be extended to industrial robots and rehabilitation robots.

Vision-Assisted Autonomous Lower-Limb Exoskeleton Robot

Lower-limb exoskeleton robots can effectively help patients with lower-limb disabilities caused by stroke or spinal cord injury to walk again. However, when faced with complex ground surfaces, patients find it difficult to quickly and accurately transmit the motion intention to the robot, resulting in falls or errors. In this article, we develop the vision-assisted autonomous lower-limb exoskeleton robot (VALOR). Based on the principles of human visual feedback and motion decision-making, a vision-assisted autonomous gait pattern planning method is proposed to improve the adaptability of the robot to the environment. The robot obtains environmental information via an RGB-D camera and extracts the ground object features that might affect gait. Then, the robot makes an autonomous decision according to the environmental features, robot state, and safety constraints. Lastly, a suitable step length and height are given to the parameterized gait pattern planning model of robot to assist with walking. The feasibility of the proposed method is verified on VALOR in a controlled indoor environment with limited obstacles, and our results demonstrate that the method can significantly improve robot adaptability to complex walking environments.

A Review on Potential of Robotic Rehabilitation in Health Care System

Robotic rehabilitation states as the restorative therapy for the which act as the augmented tool for the health care workers. The methodology was to collect the articles from various scholar sites and where the scrutinization for 180 articles were done. Where the need of the study was to create review upon the robotic technology in the health care. They act as the rehabilitative tool for the upper and lower limb which is further classified into endoskeletons and exoskeletons which is having which is having different mechanism of play and different outcome. The need of rehabilitation technology with robots is inclusive of accurate monitoring, surgery, medicine and rehabilitation. There is a history of robotics in health care as begin in 1960’s for surgery and in 1990’s for rehabilitation and now have expanded in various accessible new technologies that are affordable, having multiple control system with various degrees of freedom and different approach to work. Some of them are highly versatile even can work without any supervision. The need of robotics will increase so as to assist not to remove and gain satisfaction at every end of patient and health worker.
 The past to present to the future have been very challenging but are present with great outcomes.

Improved Online Adjustment of Step Timing and Location for Legged Locomotion

Online step adaptation is critical for legged locomotion to resist disturbances, while current approaches mainly focus on step location regardless of step timing. This paper proposes an online adaptation algorithm that deals with both step timing and step location. A model predictive control framework is developed based on centroidal dynamics to predict robot state trajectories, which are utilized to simultaneously optimize step timing and location with full considerations on capturability, feasibility and reachability. Then, a swing foot trajectory generation method is presented to accommodate mutiple constraints on kinematics and actuators. Moreover, a heuristic compensation is enhanced to the step location for quadruped trotting locomotion, increasing the anti-disturbance ability of legged locomotion. Various push-recovery simulation and experiment on an 18 degree-of-freedom legged robot are implemented to verify the proposed algorithm.

PLC-VIO: Visual–Inertial Odometry Based on Point-Line Constraints

Visual–inertial odometry (VIO) is widely studied and used in autonomous robots. This article proposes a novel tightly coupled monocular VIO system based on point-line constraints (PLC-VIO). In the front end, PLC-VIO presents a line segment extraction and merging algorithm based on the EDLines method and achieves real-time feature tracking based on the geometric constraints between feature points and lines. In the back end, PLC-VIO reconstructs new 3-D landmarks of feature lines through points on the line and optimizes the states by minimizing a cost function that combines the preintegrated inertial measurement unit (IMU) error term together with the point and line reprojection error terms in a sliding window optimization framework. A loop closure module is also integrated, which enables relocalization and drift elimination. The corresponding experimental evaluations are conducted using public datasets to validate the effectiveness and robustness of the proposed system, and the results show that PLC-VIO can achieve good performance when compared with other state-of-the-art systems and, at the same time, with no compromise to real-time performance. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Visual–inertial odometry (VIO) can estimate the states of the rigid body (including position, attitude, and velocity) that is widely used in robotic navigation, autonomous driving, virtual reality (VR), and augmented reality (AR). Aiming at the problem of estimating the states of autonomous robots in the GPS-denied environment, this article proposes a novel VIO system based on the point-line constraints (PLC-VIO). PLC-VIO can not only achieve accurate pose estimation for robots due to the introduction of the line features but also make no concession to real-time performance. Furthermore, PLC-VIO can also enrich the texture features of the environment during the 3-D mapping construction. The corresponding experiments are implemented in public datasets to evaluate the effectiveness, efficiency, and robustness of the proposed system. We believe that PLC-VIO can be widely used in robotic navigation and AR/VR fields to provide accurate position and environment information in real time.

Challenges in improving the minimal invasiveness of the surgical treatment for gastric cancer using robotic technology.

The number of operations performed using the da Vinci Surgical System® (DVSS) has been increasing worldwide in the past decade. We introduced robotic gastrectomy for gastric cancer (GC) in January 2009 to overcome the disadvantage of conventional laparoscopic gastrectomy. Initially, we experienced some troubles in the technical aspect and cost of robotic surgery. After extensive trial and error, we were able to develop the “double bipolar method” and the “da Vinci's plane theory” to use DVSS effectively. We then conducted “Senshiniryo B,” which was a multi‐institutional prospective single‐arm study to determine the safety, feasibility, and effectiveness of robotic gastrectomy for GC in 2014. In that study, we demonstrated that the morbidity rate in the robotic group (2.45%) was significantly lower than that in the historical control group (6.4%). As a consequence of that clinical trial, 12 procedures, including robotic gastrectomy for GC, have been covered under the Japanese national insurance in 2018. An additional seven procedures were newly covered in April 2020. In the first half of this article, we describe the history of robotic surgery in the world and Japan and demonstrate the “double bipolar method” and “da Vinci's plane theory.” In the latter half, we explain the Japanese systems for the safe dissemination of robotic surgery and state our efforts to solve some problems in robotic surgery.

Starfish Inspired Milli Soft Robot With Omnidirectional Adaptive Locomotion Ability

Bionic robot inspired by creatures in nature attracts attention from researchers due to its effective locomotion mode and high environment adaptability. Despite the recent achievements in bionic research and robotics, the bionic robots are almost simple morphology imitation. While the imitation of functional texture and microstructures are also important, which always makes a significant difference to the performance of robot. In this work, we proposed a starfish inspired milli soft robot with excellent locomotive performance and great environmental adaptability. Different from the simple imitation of locomotion mode in the previous literature, we introduce the micro tube feet structures of starfish into the robot design. The magnetization measurement and friction test result indicate that the microstructures of tube feet can strengthen the driving performance of robot, reduce the motion resistance from ground, and enhance the obstacle-climbing ability. Benefit from the radial symmetry shape of the starfish, the milli soft robot can achieve omnidirectional movement under the actuation of magnetic field. Moreover, the large deflection modeling of the variable cross section antenna is built by analyzing the infinitesimal element, and two motion states of milli soft robot are compared. The successful locomotion demonstrations, including moving with “S” trajectory, adapting to wet surface and gravel surface, overcoming obstacles, indicate the potential application of the starfish inspired milli soft robot in harsh environment.

The current state of robotics in total knee arthroplasty.

Robotic total knee arthroplasty (TKA) has demonstrated improved component positioning and a reduction of alignment outliers with regard to pre-operative planning.Early robotic TKA technologies were mainly active systems associated with significant technical and surgical complications.Current robotic TKA systems are predominantly semi-active with additional haptic feedback which minimizes iatrogenic soft tissue injury compared to conventional arthroplasty and older systems.Semi-active systems demonstrate advantages in terms of early functional recovery and hospital discharge compared to conventional arthroplasty.Limitations with current robotic technology include high upfront costs, learning curves and lack of long-term outcomes.The short-term gains and greater technical reliability associated with current systems may justify the ongoing investment in robotic technology.Further long-term data are required to fully ascertain the cost-effectiveness of newer robotic systems.Cite this article: EFORT Open Rev 2021;6:270-279. DOI: 10.1302/2058-5241.6.200052

Zoomorphic Gestures for Communicating Cobot States

Communicating the robot state is vital to creating an efficient and trustworthy collaboration between humans and collaborative robots (cobots). Standard approaches for Robot-tohuman communication face difficulties in industry settings, e.g., because of high noise levels or certain visibility requirements. Therefore, this letter presents zoomorphic gestures based on dog body language as a possible alternative for communicating the state of appearance-constrained cobots. For this purpose, we conduct a visual communication benchmark comparing zoomorphic gestures, abstract gestures, and light displays. We investigate the modalities regarding intuitive understanding, user experience, and user preference. In a first user study (n = 93), we evaluate our proposed design guidelines for all visual modalities. A second user study (n = 214) constituting the benchmark indicates that intuitive understanding and user experience are highest for both gesture-based modalities. Furthermore, zoomorphic gestures are considerably preferred over other modalities. These findings indicate that zoomorphic gestures with their playful nature are especially suitable for novel users and may decrease initial inhibitions.

Corrective Shared Autonomy for Addressing Task Variability.

Many tasks, particularly those involving interaction with the environment, are characterized by high variability, making robotic autonomy difficult. One flexible solution is to introduce the input of a human with superior experience and cognitive abilities as part of a shared autonomy policy. However, current methods for shared autonomy are not designed to address the wide range of necessary corrections (e.g., positions, forces, execution rate, etc.) that the user may need to provide to address task variability. In this paper, we present corrective shared autonomy, where users provide corrections to key robot state variables on top of an otherwise autonomous task model. We provide an instantiation of this shared autonomy paradigm and demonstrate its viability and benefits such as low user effort and physical demand via a system-level user study on three tasks involving variability situated in aircraft manufacturing.

Robust Footstep Planning and LQR Control for Dynamic Quadrupedal Locomotion

In this paper, we aim to improve the robustness of dynamic quadrupedal locomotion through two aspects: 1) fast model predictive foothold planning, and 2) applying LQR to projected inverse dynamic control for robust motion tracking. In our proposed planning and control framework, foothold plans are updated at 400 Hz considering the current robot state and an LQR controller generates optimal feedback gains for motion tracking. The LQR optimal gain matrix with non-zero off-diagonal elements leverages the coupling of dynamics to compensate for system underactuation. Meanwhile, the projected inverse dynamic control complements the LQR to satisfy inequality constraints. In addition to these contributions, we show robustness of our control framework to unmodeled adaptive feet. Experiments on the quadruped ANYmal demonstrate the effectiveness of the proposed method for robust dynamic locomotion given external disturbances and environmental uncertainties.