Dynamic Movement Primitives Research Articles

Hierarchical critic learning optimal control for lower limb exoskeleton robots with prescribed constraints

AbstractThis paper presents a hierarchical critic learning optimal control (H‐CLOC) strategy for a lower limb exoskeleton system with prescribed state constraints. The prescribed state constraints of the exoskeleton are mathematically modeled by means of a state error transition function, which transforms the constrained state into an equivalent new unconstrained state. A hierarchical framework including a high‐level motion trajectory planner and a low‐level critic learning (CL) controller is proposed for optimal control of the exoskeleton. The estimation of human motion intentions is achieved using dynamic movement primitives (DMPs), which are capable of accurately replicating the original training trajectories during motion. The CL algorithm is implemented on the transformed unconstrained system while enhancing the learning capability of the proposed algorithm for the exoskeleton. The optimal control with prescribed constraints is analyzed for stability using the Lyapunov theorem. Finally, simulation results validate the effectiveness of the proposed H‐CLOC strategy for lower limb exoskeletons.

Coordinated Motion Planning for Heterogeneous Autonomous Vehicles Based on Driving Behavior Primitives

Heterogeneous autonomous vehicle (HAV) coordinated motion planning must guide each vehicle out of the conflict zone based on the differences in vehicle platform characteristics. Decomposing complex driving tasks into primitives is an effective way to improve algorithm efficiency. Hence, the purpose of this paper is to complete the coordinated motion planning tasks through offline driving behavior primitive (DBP) library generation, online extension and selection of DBPs. The proposed algorithm applies dynamic movement primitives and singular value decomposition to learn driving behavior patterns from driving data, integrates them into a model-based optimization generation method as constraints, and builds a DBP library by fusing driving data and vehicle model. Based on the generated DBP library and primitive association probabilities learned from labeled driving segments via stochastic context-free grammar, the planning method completes the independent DBP extension of each vehicle in the conflict zone, generates an interaction DBP tree, and uses the mixed-integer linear programming algorithm to optimally select the primitives to be executed. This study demonstrates that the generated DBP library not only expands the types of primitives, but also distinguishes the characteristics of HAVs. We also present how to utilize the DBP libraries to obtain coordinated motion planning results with spatiotemporal information in the form of DBP extension and selection. The results obtained by real vehicle platforms and simulation show that the proposed method can accomplish coordinated motion planning tasks without relying on specific scene elements and highlight the unique motion characteristics of HAVs.

Robotic peg-in-hole assembly based on reversible dynamic movement primitives and trajectory optimization

Robotic peg-in-hole assembly based on reversible dynamic movement primitives and trajectory optimization

A Motion Planning Method for Robots Based on DMPs and Modified Obstacle-Avoiding Algorithm

This paper addresses the motion planning of the manipulator in task space. To improve the overall trajectory performance, a special motion planning method based on DMPs (Dynamic Movement Primitives) and the modified obstacle-avoiding algorithm is proposed. The proposed method solves the problems of trajectory jitter and inability to avoid obstacles in some scenarios, which are faced by the scheme of steering angle. Besides, it improves the retention of teaching intentions, reduces the loss of free space, and helps the system adapt to the dynamic environment. At the theoretical level, the convergence of the target state has been proved using Lyapunov stability theory-based analysis. The availability of the proposed method is validated and analyzed by performing a series of numerical simulations and Baxter robot experiments. The results indicate that the proposed method can provide reliable solutions for motion planning. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —From the perspective of demonstration learning, this paper aims to elevate the motion planning of the manipulator in task space, especially in improving the obstacle avoidance performance. The existing DMPs-based motion planning algorithm uses the scheme of steering angle to achieve obstacle avoidance, and the obstacle is regarded as a mesh of points on the boundary. The obstacle-avoiding performance is limited. How to combine the obstacle-avoiding algorithm with the DMPs-based motion planning algorithm more effectively, so as to simultaneously achieve retaining teaching intentions as much as possible, still faces challenges. This paper designs a modified obstacle-avoiding algorithm, which solves the problems of trajectory jitter and inability to avoid obstacles in some circumstances, and improves the retention of teaching intentions. The modified obstacle-avoiding algorithm is also applied to the dynamic environment. The proposed method satisfies Lyapunov stability theory-based analysis, and the experiments on Baxter robot verify the feasibility.

Decoding Upper-Limb Movement Intention Through Adaptive Dynamic Movement Primitives: A Proof-of-Concept Study with a Shoulder-Elbow Exoskeleton.

This work presents an intention decoding algorithm that can be used to control a 4 degrees-of-freedom shoulder-elbow exoskeleton in reaching tasks. The algorithm was designed to assist the movement of users with upper-limb impairments who can initiate the movement by themselves. It relies on the observation of the initial part of the user's movement through joint angle measures and aims to estimate in real-time the phase of the movement and predict the goal position of the hand in the reaching task. The algorithm is based on adaptive Dynamic Movement Primitives and Gaussian Mixture Models. The performance of the algorithm was verified in robot-assisted planar reaching movements performed by one healthy subject wearing the exoskeleton. Tests included movements of different amplitudes and orientations. Results showed that the algorithm could predict the hand's final position with an error lower than 5 cm after 0.25 s from the movement onset, and that the final position reached during the tests was on average less than 4 cm far from the target position. Finally, the effects of the assistance were observed in a reduction of the activation of the Biceps Brachii and of the time to execute the reaching tasks.

Continuous Phase Estimation in a Variety of Locomotion Modes Using Adaptive Dynamic Movement Primitives.

Accurate gait phase estimation algorithms can be used to synchronize the action of wearable robots to the volitional user movements in real time. Current-day gait phase estimation methods are designed mostly for rhythmic tasks and evaluated in highly controlled walking environments (namely, steady-state walking). Here, we implemented adaptive Dynamic Movement Primitives (aDMP) for continuous real-time phase estimation in the most common locomotion activities of daily living, which are level-ground walking, stair negotiation, and ramp negotiation. The proposed method uses the thigh roll angle and foot-contact information and was tested in real time with five subjects. The estimated phase resulted in an average root-mean-square error of 3.98% ± 1.33% and a final estimation error of 0.60% ± 0.55% with respect to the linear phase. The results of this study constitute a viable groundwork for future phase-based control strategies for lower-limb wearable robots, such as robotic prostheses or exoskeletons.

Dynamic movement primitives in robotics: A tutorial survey

Biological systems, including human beings, have the innate ability to perform complex tasks in a versatile and agile manner. Researchers in sensorimotor control have aimed to comprehend and formally define this innate characteristic. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally leads to the formulation of the motor primitives’ theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems and are well suited to generate motor commands for artificial systems like robots. In the last decades, DMPs have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control, physical interaction, and human–robot co-working, resulting in a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMP formulations in rigorous mathematical terms and discuss the advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state-of-the-art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.

Turn-Taking Prediction for Human–Robot Collaborative Assembly Considering Human Uncertainty

Abstract Human–robot collaboration (HRC) combines the repeatability and strength of robots and human’s ability of cognition and planning to enable a flexible and efficient production mode. The ideal HRC process is that robots can smoothly assist workers in complex environments. This means that robots need to know the process’s turn-taking earlier, adapt to the operating habits of different workers, and make reasonable plans in advance to improve the fluency of HRC. However, many of the current HRC systems ignore the fluent turn-taking between robots and humans, which results in unsatisfactory HRC and affects productivity. Moreover, there are uncertainties in humans as different humans have different operating proficiency, resulting in different operating speeds. This requires the robots to be able to make early predictions of turn-taking even when human is uncertain. Therefore, in this paper, an early turn-taking prediction method in HRC assembly tasks with Izhi neuron model-based spiking neural networks (SNNs) is proposed. On this basis, dynamic motion primitives (DMP) are used to establish trajectory templates at different operating speeds. The length of the sequence sent to the SNN network is judged by the matching degree between the observed data and the template, so as to adjust to human uncertainty. The proposed method is verified by the gear assembly case. The results show that our method can shorten the human–robot turn-taking recognition time under human uncertainty.

Safe and Human‐Like Trajectory Planning of Self‐Driving Cars: A Constraint Imitative Method

Safe and human‐like trajectory planning is crucial for self‐driving cars. While model‐based planning has demonstrated reliability, it is beneficial to incorporate human demonstrations and align the results with human behaviors. This work aims at bridging the gap between model‐based planning and driver imitation by proposing a constraint imitative trajectory planning method (CITP). CITP integrates artificial potential field and dynamic movement primitives, which have achieved both the ability to imitate human demonstrations as well as ensure safety constraints. During the planning process, CITP first encodes human demonstrations, local driving target, and traffic obstacles as attractive or repulsive effects, and then the trajectory planning problem is solved through model predictive optimization. To address the dynamics of traffic scenarios, a hierarchical planning strategy is proposed based on the division of planning process. CITP is designed with five modules, including LSTM‐based target generation, encoding attractive and repulsive effects with target, demonstrations and obstacles, and trajectory planning with model predictive optimization. Data collection and experiments are carried out based on CARLA driving simulator, and the effectiveness in terms of both safety and consistency with human behavior are reported.

High Precision Trajectory Learning Method Based Improved Dynamic Movement Primitives for Robot Skill Learning

Abstract In robot skill learning, the higher accuracy of trajectory reproduction, the smaller difference between the trajectory reproduced by the robot and the expected trajectory, and the higher accuracy and reliability of the robot’s task execution. In robot skill learning methods based on dynamic movement primitives, the problems are found that the tail exponential decay of Gaussian kernel function and the number of shape parameters have a great influence on the accuracy of trajectory reproduction, a high precision trajectory learning method is proposed. A novel truncation processing method is proposed to eliminate the impact of tail exponential decay, and the optimization on the number of shape parameters is used to improve the approximation of the local gradient of the target forcing term, which are combined to improve trajectory reproduction accuracy. The principle of the proposed method is described in detail. The simulation and comparison experiments are performed to verify the effectiveness of the proposed method in improving trajectory reproduction accuracy. This paper makes contributions to the field of robot skill trajectory learning and provides a promising method for improving trajectory reproduction accuracy.

Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving

As learning-based methods make their way from perception systems to planning/control stacks, robot control systems have started to enjoy the benefits that data-driven methods provide. Because control systems directly affect the motion of the robot, data-driven methods, especially black box approaches, need to be used with caution considering aspects such as stability and interpretability. In this paper, we describe a differentiable and hierarchical control architecture. The proposed representation, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-abstractive neural controller</i> , uses the input image to control the transitions within a novel discrete behavior planner (referred to as the visual automaton generative network, or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vAGN</i> ). The output of a vAGN controls the parameters of a set of dynamic movement primitives which provides the system controls. We train this neural controller with real-world driving data via behavior cloning and show improved explainability, sample efficiency, and similarity to human driving.

Dynamic Multiaction Recognition and Expert Movement Mapping for Closed Pelvic Reduction

Pelvic fractures are one of the most serious traumas in orthopedic care, and reduction during routine surgery is a significant challenge. Because there are so many vital organs, blood vessels, and nerves around the pelvis, and the reduction force is large, the operational requirements for the surgeon are extremely strict and require extensive experience and surgical skills. This paper proposes a method for collecting and digitizing doctors' reduction movements, which aims to help intelligent devices recognize surgeons' reduction actions and provides a means to learn from expert experience to improve the accuracy of surgery. First, the Conv-BiLSTM algorithm with multilayer cross-fused features is proposed. It extracts time and spatial correlations between multimodal data in a hierarchical manner. Second, discrete dynamic motion primitives (DMPs) are adopted for mapping the surgeon's palm movement trajectory. Finally, this paper constructs a data acquisition platform and collects data from surgeons with varying proficiency in closed reduction. Experiment results show that the closed reduction action recognition accuracy is 99% and posture recognition accuracy is 95.5%. The recognition algorithm proposed by this paper is significantly higher than the commonly used algorithms in terms of Accuracy, Precision, Recall, and F1-Score. This work provides methods and means for the digitization of surgical expertise and transfers learning for robot-assisted surgery.

A Planning Framework for Robotic Insertion Tasks via Hydroelastic Contact Model

Robotic contact-rich insertion tasks present a significant challenge for motion planning due to the complex force interaction between robots and objects. Although many learning-based methods have shown success in contact tasks, most methods need sampling or exploring to gather sufficient experimental data. However, it is both time-consuming and expensive to conduct real-world experiments repeatedly. On the other hand, while the virtual world enables low cost and fast computations by simulators, there still exists a huge sim-to-real gap due to the inaccurate point contact model. Although finite element analysis might generate accurate results for contact tasks, it is computationally expensive. As such, this study proposes a motion planning framework with bilevel optimization to leverage relatively accurate force information with fast computation time. This framework consists of Dynamic Movement Primitives (DMPs) used to parameterize motion trajectories, Black-Box Optimization (BBO), a derivative-free approach, integrated to improve contact-rich insertion policy with hydroelastic contact model, and simulated variability to account for visual uncertainty in the real world. The accuracy of the simulated model is then validated by comparing our contact results with a benchmark Peg-in-Hole task. Using these integrated DMPs and BBO with hydroelastic contact model, the motion trajectory generated in planning is capable of guiding the robot towards successful insertion with iterative refinement.

Modified Dynamic Movement Primitives: Robot Trajectory Planning and Force Control Under Curved Surface Constraints.

Dynamic movement primitives (DMPs) have been widely applied in robot motion planning and control. However, in some special cases, original discrete DMP fails to generalize proper trajectories. Moreover, it is difficult to produce trajectories on the curved surface. To solve the above problems, a modified DMP method is proposed for robot control by adding the scaling factor and force coupling term. First, the adjusted cosine similarity is defined to assess the similarity of the generalized trajectory with respect to the demonstrated trajectory. By optimizing the similarity, the trajectories can be generated in all situations. Next, by adding the force coupling term derived from adaptive admittance control to the transformation system of the original DMP, the controller achieves the force control ability. Then, the modified DMP-based robot control system is developed. The stability and convergence of the system are proved. Finally, the high precisions of the proposed method are verified by simulations and experiments. The method is significant for trajectory learning and generalization on the curved surface.

Haptic Exploration of Unknown Objects for Robust In-Hand Manipulation.

Human-like robot hands provide the flexibility to manipulate a variety of objects that are found in unstructured environments. Knowledge of object properties and motion trajectory is required, but often not available in real-world manipulation tasks. Although it is possible to grasp and manipulate unknown objects, an uninformed grasp leads to inferior stability, accuracy, and repeatability of the manipulation. Therefore, a central challenge of in-hand manipulation in unstructured environments is to acquire this information safely and efficiently. We propose an in-hand manipulation framework that does not assume any prior information about the object and the motion, but instead extracts the object properties through a novel haptic exploration procedure and learns the motion from demonstration using dynamical movement primitives. We evaluate our approach by unknown object manipulation experiments using a human-like robot hand. The results show that haptic exploration improves the manipulation robustness and accuracy significantly, compared to the virtual spring framework baseline method that is widely used for grasping unknown objects.

Imitation Learning-Based System for the Execution of Self-Paced Robotic-Assisted Passive Rehabilitation Exercises

The development of robotic-assisted rehabilitation exercises involving physical human-robot interaction requires extreme care since an injured limb may be in physical contact with the robot, so compliant behavior is imperative for these tasks. Typical approaches involve force control schemes like admittance controllers that allow humans to adapt the motion. However, when the patient's limb has limited mobility or is potentially injured, unintentional forces may occur during the robot's trajectory that could be incompatible with these controllers. This paper addresses a new way of generating compliant trajectories for passive rehabilitation exercises, considering that previous positions of the trajectory are attainable for the patient, so reversing the trajectory is a safe operation. Since there is no clear way to optimize such a goal due to the physiological variability among patients, the condition of reversal is based on imitation learning by taking the analogous healthy limb of the patient as a reference and encoding the forces using Gaussian Mixture Regression, and reversibility is accomplished by means of Reversible Dynamic Movement Primitives. The system allows for self-paced rehabilitation exercises by back-and-forth movements along the trajectory according to the patient's reaction, and it has been successfully applied to a 4-DOF parallel robot for lower-limb rehabilitation.

Combined Admittance Control With Type II Singularity Evasion for Parallel Robots Using Dynamic Movement Primitives

This article addresses a new way of generating compliant trajectories for control using movement primitives to allow physical human–robot interaction where parallel robots (PRs) are involved. PRs are suitable for tasks requiring precision and performance because of their robust behavior. However, two fundamental issues must be resolved to ensure safe operation: first, the force exerted on the human must be controlled and limited, and second, Type II singularities should be avoided to keep complete control of the robot. We offer a unified solution under the dynamic movement primitives (DMP) framework to tackle both tasks simultaneously. DMPs are used to get an abstract representation for movement generation and are involved in broad areas, such as imitation learning and movement recognition. For force control, we design an admittance controller intrinsically defined within the DMP structure, and subsequently, the Type II singularity evasion layer is added to the system. Both the admittance controller and the evader exploit the dynamic behavior of the DMP and its properties related to invariance and temporal coupling, and the whole system is deployed in a real PR meant for knee rehabilitation. The results show the capability of the system to perform safe rehabilitation exercises.

Object Affordance-Based Implicit Interaction for Wheelchair-Mounted Robotic Arm Using a Laser Pointer.

With the growth of the world's population, limited healthcare resources cannot provide adequate nursing services for all people in need. The wheelchair-mounted robotic arm (WMRA) with interactive technology could help to improve users' self-care ability and relieve nursing stress. However, the users struggle to control the WMRA due to complex operations. To use the WMRA with less burden, this paper proposes an object affordance-based implicit interaction technology using a laser pointer. Firstly, a laser semantic identification algorithm combined with the YOLOv4 and the support vector machine (SVM) is designed to identify laser semantics. Then, an implicit action intention reasoning algorithm, based on the concept of object affordance, is explored to infer users' intentions and learn their preferences. For the purpose of performing the actions about task intention in the scene, the dynamic movement primitives (DMP) and the finite state mechanism (FSM) are respectively used to generalize the trajectories of actions and reorder the sequence of actions in the template library. In the end, we verified the feasibility of the proposed technology on a WMRA platform. Compared with the previous method, the proposed technology can output the desired intention faster and significantly reduce the user's limb involvement time (about 85%) in operating the WMRA under the same task.

Stitching Dynamic Movement Primitives and Image-Based Visual Servo Control

Utilizing perception for feedback control in combination with dynamic movement primitive (DMP)-based motion generation for a robot’s end-effector control is a useful solution for many robotic manufacturing tasks. For instance, while performing an insertion task when the hole or the recipient part is not visible in the eye-in-hand camera, a learning-based movement primitive method can be used to generate the end-effector path. Once the recipient part is in the field of view (FOV), image-based visual servo (IBVS) can be used to control the motion of the robot. Inspired by such applications, this article presents a generalized control scheme that switches between motion generation using DMP and IBVS control. To facilitate the design, a common state-space representation for the DMP and the IBVS systems is first established. The stability analysis of the switched system using multiple Lyapunov functions shows that the state trajectories converge to a bound asymptotically. The developed method is validated by three real-world experiments using the eye-in-hand configuration of a Baxter research robot.

A Step Towards Conditional Autonomy - Robotic Appendectomy

In recent years, Robot-Assisted Minimally Invasive Surgery (RAMIS) has been widely adopted worldwide due to its high precision, improved ergonomics and intuitive control. With the advances in artificial intelligence and surgical robot technologies, it is anticipated that the cognitive load on the surgeons could be relieved with a higher level of autonomy on surgical robots as well as simultaneously improving the precision of surgical robots' operations, minimising the risk of human errors and leading to safer operations. However, much research is still focused on task autonomy due to the complexity of the operations, and the limited clinical data on robotic surgery in some specific scenarios. Therefore, recent research has widely used simulated surgery scenarios to collect data to train artificial intelligence (AI) models. In this paper, we propose a method as a step towards conditional autonomy for robotic appendectomy. To achieve conditional autonomy, demonstration data was collected from human operators performing the appendectomy in a simulated robotic scene. The robotic instrument movements and trajectories for performing the appendectomy were learned from the demonstrated data. Once the surgeon specifies the location of the appendix, the proposed framework can carry out the appendectomy autonomously by using the dynamic motion primitives (DMP) generated from the learned surgical movements. Through the extensive validations in a simulated environment and on the da Vinci Research Kit (dVRK), we have shown that the proposed method can carry out the appendectomy procedure semi-automatically. Based on five metrics, it shows that the framework can reduce the total task path length, completion time and appendix stump length while maintaining a high degree of similarity to the demonstrated trajectory. This work demonstrated the feasibility of conditional autonomy in robotic surgeries.