Motion Primitives Research Articles

Symmetry and motion primitives in model predictive control

Symmetries, e.g. rotational and translational invariances for the class of mechanical systems, allow to characterize solution trajectories of nonlinear dynamical systems. Thus, the restriction to symmetry-induced dynamics, e.g. by using the concept of motion primitives, may be considered as a quantization of the system. Symmetry exploitation is well established in both motion planning and control. However, the linkage between the respective techniques to optimal control is not yet fully explored. In this manuscript, we want to lay the foundation for the usage of symmetries in Model Predictive Control (MPC). To this end, we investigate a mobile robot example in detail where our contribution is twofold: Firstly, we establish asymptotic stability of a desired set point w.r.t. the MPC closed loop, which is also demonstrated numerically by using motion primitives applied to the parallel parking scenario. Secondly, if the optimization criterion is not consistent with the symmetry action, we provide guidelines to rigorously derive stability guarantees based on symmetry exploitation.

Reinforcement learning based on movement primitives for contact tasks

Abstract Recently, robot learning through deep reinforcement learning has incorporated various robot tasks through deep neural networks, without using specific control or recognition algorithms. However, this learning method is difficult to apply to the contact tasks of a robot, due to the exertion of excessive force from the random search process of reinforcement learning. Therefore, when applying reinforcement learning to contact tasks, solving the contact problem using an existing force controller is necessary. A neural-network-based movement primitive (NNMP) that generates a continuous trajectory which can be transmitted to the force controller and learned through a deep deterministic policy gradient (DDPG) algorithm is proposed for this study. In addition, an imitation learning algorithm suitable for NNMP is proposed such that the trajectories similar to the demonstration trajectory are stably generated. The performance of the proposed algorithms was verified using a square peg-in-hole assembly task with a tolerance of 0.1 mm. The results confirm that the complicated assembly trajectory can be learned stably through NNMP by the proposed imitation learning algorithm, and that the assembly trajectory is improved by learning the proposed NNMP through the DDPG algorithm.

Generating Pointing Motions for a Humanoid Robot by Combining Motor Primitives.

The human motor system is robust, adaptive and very flexible. The underlying principles of human motion provide inspiration for robotics. Pointing at different targets is a common robotics task, where insights about human motion can be applied. Traditionally in robotics, when a motion is generated it has to be validated so that the robot configurations involved are appropriate. The human brain, in contrast, uses the motor cortex to generate new motions reusing and combining existing knowledge before executing the motion. We propose a method to generate and control pointing motions for a robot using a biological inspired architecture implemented with spiking neural networks. We outline a simplified model of the human motor cortex that generates motions using motor primitives. The network learns a base motor primitive for pointing at a target in the center, and four correction primitives to point at targets up, down, left and right from the base primitive, respectively. The primitives are combined to reach different targets. We evaluate the performance of the network with a humanoid robot pointing at different targets marked on a plane. The network was able to combine one, two or three motor primitives at the same time to control the robot in real-time to reach a specific target. We work on extending this work from pointing to a given target to performing a grasping or tool manipulation task. This has many applications for engineering and industry involving real robots.

The Pragmatic Classification of Upper Extremity Motion in Neurological Patients: A Primer.

Recent advances in wearable sensor technology and machine learning (ML) have allowed for the seamless and objective study of human motion in clinical applications, including Parkinson's disease, and stroke. Using ML to identify salient patterns in sensor data has the potential for widespread application in neurological disorders, so understanding how to develop this approach for one's area of inquiry is vital. We previously proposed an approach that combined wearable inertial measurement units (IMUs) and ML to classify motions made by stroke patients. However, our approach had computational and practical limitations. We address these limitations here in the form of a primer, presenting how to optimize a sensor-ML approach for clinical implementation. First, we demonstrate how to identify the ML algorithm that maximizes classification performance and pragmatic implementation. Second, we demonstrate how to identify the motion capture approach that maximizes classification performance but reduces cost. We used previously collected motion data from chronic stroke patients wearing off-the-shelf IMUs during a rehabilitation-like activity. To identify the optimal ML algorithm, we compared the classification performance, computational complexity, and tuning requirements of four off-the-shelf algorithms. To identify the optimal motion capture approach, we compared the classification performance of various sensor configurations (number and location on the body) and sensor type (IMUs vs. accelerometers). Of the algorithms tested, linear discriminant analysis had the highest classification performance, low computational complexity, and modest tuning requirements. Of the sensor configurations tested, seven sensors on the paretic arm and trunk led to the highest classification performance, and IMUs outperformed accelerometers. Overall, we present a refined sensor-ML approach that maximizes both classification performance and pragmatic implementation. In addition, with this primer, we showcase important considerations for appraising off-the-shelf algorithms and sensors for quantitative motion assessment.

Learning Generalizable Coupling Terms for Obstacle Avoidance via Low-Dimensional Geometric Descriptors

Unforeseen events are frequent in the real-world environments where robots are expected to assist, raising the need for fast replanning of the on-going policy to guarantee operational safety. Inspired by human behavioral studies of obstacle avoidance and route selection, this letter presents a hierarchical framework that generates reactive yet bounded obstacle avoidance behaviors through a multi-layered analysis. The framework leverages the strengths of learning techniques and the versatility of dynamic movement primitives to efficiently unify perception, decision, and action levels via environmental low-dimensional geometric descriptors. Experimental evaluation on synthetic environments and a real anthropomorphic manipulator proves the robustness and generalization capabilities of the proposed approach regardless of the obstacle avoidance scenario.

Reinforcement learning of motor skills using Policy Search and human corrective advice

Robot learning problems are limited by physical constraints, which make learning successful policies for complex motor skills on real systems unfeasible. Some reinforcement learning methods, like Policy Search, offer stable convergence toward locally optimal solutions, whereas interactive machine learning or learning-from-demonstration methods allow fast transfer of human knowledge to the agents. However, most methods require expert demonstrations. In this work, we propose the use of human corrective advice in the actions domain for learning motor trajectories. Additionally, we combine this human feedback with reward functions in a Policy Search learning scheme. The use of both sources of information speeds up the learning process, since the intuitive knowledge of the human teacher can be easily transferred to the agent, while the Policy Search method with the cost/reward function take over for supervising the process and reducing the influence of occasional wrong human corrections. This interactive approach has been validated for learning movement primitives with simulated arms with several degrees of freedom in reaching via-point movements, and also using real robots in such tasks as “writing characters” and the ball-in-a-cup game. Compared with standard reinforcement learning without human advice, the results show that the proposed method not only converges to higher rewards when learning movement primitives, but also that the learning is sped up by a factor of 4–40 times, depending on the task.

Optimal Constant Acceleration Motion Primitives

This paper proposes new motion primitives that are time optimal and feasible for a vehicle (wheeled-mobile robot). They are parameterized by a constant acceleration and a constant deceleration in an obstacle-free environment between an initial and a final configuration with given poses and velocities. The derived compact parametric solution has an implicit optimal velocity profile, is continuous in C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , considers driving constraints, and is computationally efficient. The path is derived analytically considering constraints on maximal allowable driving velocity and accelerations. The obtained motion primitives have continuous transition of curvature and are, thus, easily drivable by the vehicle. The proposed motion primitives are evaluated on several path-planning examples, and the obtained solutions are compared to a numerically expensive planner using Bernstein-Bézier curve with path shape and velocity profile optimization. Numerous experiments have been conducted not only in the simulation environment, but also during testing and comparisons on the actual mobile robot platforms. It is shown that the proposed solution is computationally efficient, time optimal under given constraints and trapezoidal velocity profile assumptions, and is close to globally optimal solution with an arbitrary velocity profile.

Switched Systems With Multiple Equilibria Under Disturbances: Boundedness and Practical Stability

This paper addresses robustness to external disturbances of switched discrete and continuous systems with multiple equilibria. First, we prove that if each subsystem of the switched system is input-to-state stable (ISS), then under switching signals that satisfy an average dwell-time bound, the solutions are ultimately bounded within a compact set. The size of this set varies monotonically with the supremum norm of the disturbance signal. These results generalize existing ones in the common equilibrium case to accommodate multiple equilibria. Then, we relax the (global) ISS conditions to consider equilibria that are locally exponentially stable (LES), and we establish practical stability for such switched systems under disturbances. Our motivation for studying this class of switched systems arises from certain motion planning problems in robotics, where primitive movements, each corresponding to an equilibrium point of a dynamical system, must be composed to obtain more complex motions. As a concrete example, we consider the problem of realizing safe adaptive locomotion of a three-dimensional biped under persistent external force by switching among motion primitives characterized by LES limit cycles. The results of this paper, however, are relevant to a much broader class of applications, in which composition of different modes of behavior is required.

Robot complex motion learning based on unsupervised trajectory segmentation and movement primitives

This paper presents a robot skill acquisition framework for learning and reproducing humanoid trajectories with complex forms. A new unsupervised segmentation method is proposed to detect motion units in the demonstrated kinematic data using the concept of key points. To find the consistent features of trajectories, a Hidden Semi-Markov Model (HSMM) is used to identify key points common to all the demonstrations. Generalizing the motion units is achieved via a Probability-based Movement Primitive (PbMP), which encapsulates multiple trajectories into one model. Such a framework can generate trajectories suitable for robot execution with arbitrary shape and complexity from a small number of demonstrations, which greatly expands the application scenarios of robot programming by demonstration. The automatic segmentation process does not rely on a priori knowledge or models for specific tasks, and the generalized trajectory retains more consistent features than those produced by other algorithms. We demonstrate the effectiveness of the proposed framework through simulations and experiments.

Predicting Perceived Naturalness of Human Animations Based on Generative Movement Primitive Models

We compared the perceptual validity of human avatar walking animations driven by six different representations of human movement using a graphics Turing test. All six representations are based on movement primitives (MPs), which are predictive models of full-body movement that differ in their complexity and prediction mechanism. Assuming that humans are experts at perceiving biological movement from noisy sensory signals, it follows that these percepts should be describable by a suitably constructed Bayesian ideal observer model. We build such models from MPs and investigate if the perceived naturalness of human animations are predictable from approximate Bayesian model scores of the MPs. We found that certain MP-based representations are capable of producing movements that are perceptually indistinguishable from natural movements. Furthermore, approximate Bayesian model scores of these representations can be used to predict perceived naturalness. In particular, we could show that movement dynamics are more important for perceived naturalness of human animations than single frame poses. This indicates that perception of human animations is highly sensitive to their temporal coherence. More generally, our results add evidence for a shared MP-representation of action and perception. Even though the motivation of our work is primarily drawn from neuroscience, we expect that our results will be applicable in virtual and augmented reality settings, when perceptually plausible human avatar movements are required.

Adaptation and Transfer of Robot Motion Policies for Close Proximity Human-Robot Interaction.

In the context of human-robot collaboration in close proximity, safety and comfort are the two important aspects to achieve joint tasks efficiently. For safety, the robot must be able to avoid dynamic obstacles such as a human arm with high reliability. For comfort, the trajectories and avoidance behavior of the robot need to be predictable to the humans. Moreover, these two aspects might be different from person to person or from one task to another. This work presents a framework to generate predictable motions with dynamic obstacle avoidance for the robot interacting with the human by using policy improvement method. The trajectories are generated using Dynamic Motion Primitives with an additional potential field term that penalizes trajectories that may lead to collisions with obstacles. Furthermore, human movements are predicted using a data-driven approach for proactive avoidance. A cost function is defined which measures different aspects that affect the comfort and predictability of human co-workers (e.g., human response time, joint jerk). This cost function is then minimized during human-robot interaction by the means of policy improvement through black-box optimization to generate robot trajectories that adapt to human preferences and avoid obstacles. User studies are performed to evaluate the trust and comfort of human co-workers when working with the robot. In addition, the studies are also extended to various scenarios and different users to analyze the task transferability. This improves the learning performance when switching to a new task or the robot has to adapt to a different co-worker.

A framework for robotic clothing assistance by imitation learning

ABSTRACTThe recent demographic trend across developed nations shows a dramatic increase in the aging population, fallen fertility rates and a shortage of caregivers. Hence, the demand for service robots to assist with dressing which is an essential Activity of Daily Living (ADL) is increasing rapidly. Robotic Clothing Assistance is a challenging task since the robot has to deal with two demanding tasks simultaneously, (a) non-rigid and highly flexible cloth manipulation and (b) safe human–robot interaction while assisting humans whose posture may vary during the task. On the other hand, humans can deal with these tasks rather easily. In this paper, we propose a framework for robotic clothing assistance by imitation learning from a human demonstration to a compliant dual-arm robot. In this framework, we divide the dressing task into three phases, i.e. reaching phase, arm dressing phase, and body dressing phase. We model the arm dressing phase as a global trajectory modification using Dynamic Movement Primitives (DMP), while we model the body dressing phase toward a local trajectory modification applying Bayesian Gaussian Process Latent Variable Model (BGPLVM). We show that the proposed framework developed towards assisting the elderly is generalizable to various people and successfully performs a sleeveless shirt dressing task. We also present participants feedback on public demonstration at the International Robot Exhibition (iREX) 2017. To our knowledge, this is the first work performing a full dressing of a sleeveless shirt on a human subject with a humanoid robot.

Rehabilitation Exercise Segmentation for Autonomous Biofeedback Systems with ConvFSM.

Segmenting physical movements is a key step for any accelerometry-based autonomous biofeedback system oriented to rehabilitation and physiotherapy activities. Fundamentally, this can be reduced to the detection of recurrent patterns, also called motion primitives, in longer inertial signals. Most of the solutions developed in the literature require extensive domain knowledge, or are incapable of scaling to complex motion patterns and new exercises. In this paper, we explore the capabilities of inertial measurement units for the segmentation of upper limb rehabilitation exercises. To do so, we introduce a novel segmentation technique based on Convolutional Neural Networks and Finite State Machines, called ConvFSM. ConvFSM is able to isolate motion primitives from raw streaming data, using very little domain knowledge. We also investigate different combinations of sensors, in order to identify the most effective and flexible setup that could fit a home-based rehabilitation feedback system. Experimental results are presented, based on a dataset obtained from a combination of common upper limb and lower limb exercises.

Incorporating Safety Into Parametric Dynamic Movement Primitives

Parametric dynamic movement primitives (PDMPs) are powerful motion representation algorithms, which encode multiple demonstrations and generalize them. As an online trajectory from PDMPs emulates the provided demonstrations, managing the safety guarantee of the demonstrations for a given scenario is an important issue. This letter presents a process to manage the demonstration set in PDMPs when some demonstrations are poor in terms of safety. Our proposed process distinguishes safe motion primitives from unsafe ones. In order to establish a criterion for determining whether a motion is safe or not, we calculate the safe region of the PDMPs parameters called style parameters using an optimization technique. In the optimization formulation, we calculate the unsafe style parameters that produce the closest motion to the unsafe region of the state space. By eliminating unsafe demonstrations with the parameters based on the safety criterion, and replacing them with new safe ones, we incorporate safety in the PDMPs framework. Simulation and experimental results validate that the proposed process can expand the motion primitives in the PDMPs framework to the new environmental settings by efficiently utilizing the previous demonstrations.

Combining Analytical Modeling and Learning to Simplify Dexterous Manipulation With Adaptive Robot Hands

In this paper, we focus on the formulation of a hybrid methodology that combines analytical models, constrained optimization schemes, and machine learning techniques to simplify the execution of dexterous, in-hand manipulation tasks with adaptive robot hands. More precisely, the constrained optimization scheme is used to describe the kinematics of adaptive hands during the grasping and manipulation processes, unsupervised learning (clustering) is used to group together similar manipulation strategies, dimensionality reduction is used to either extract a set of representative motion primitives (for the identified groups of manipulation strategies) or to solve the manipulation problem in a low-d space and finally an automated experimental setup is used for unsupervised, automated collection of large data sets. We also assess the capabilities of the derived manipulation models and primitives for both model and everyday life objects, and we analyze the resulting manipulation ranges of motion (e.g., object perturbations achieved during the dexterous, in-hand manipulation). We show that the proposed methods facilitate the execution of fingertip-based, within-hand manipulation tasks while requiring minimal sensory information and control effort, and we demonstrate this experimentally on a range of adaptive hands. Finally, we introduce DexRep, an online repository for dexterous manipulation models that facilitate the execution of complex tasks with adaptive robot hands. Note to Practitioners —Robot grasping and dexterous, in-hand manipulations are typically executed with fully actuated robot hands that rely on analytical methods, computation of the hand object system Jacobians, and extensive numerical simulations for deriving optimal strategies. However, these hands require sophisticated sensing elements, complicated control laws, and are not robust to external disturbances or perception uncertainties. Recently, a new class of adaptive hands was proposed which uses structural compliance and underactuation (less motors than the available degrees of freedom) to offer increased robustness and simplicity. In this paper, we propose hybrid methodologies that blend analytical models with constrained optimization schemes and learning techniques to simplify the execution of dexterous, in-hand manipulation tasks with adaptive robot hands.

Embodied robots driven by self-organized environmental feedback

Which kind of complex behavior may arise from self-organizing principles? We investigate this question for the case of snake-like robots composed of passively coupled segments, with every segment containing two wheels actuated separately by a single neuron. The robot is self-organized both on the level of the individual wheels and with respect to inter-wheel coordination, which arises exclusively from the mechanical coupling of the individual wheels and segments. For the individual wheel, the generating principle proposed results in locomotive states that correspond to self-organized limit cycles of the sensorimotor loop. Our robot interacts with the environment by monitoring the state of its actuators, that is, via propriosensation. External sensors are absent. In a structured environment the robot shows complex emergent behavior that includes pushing movable blocks around, reversing direction when hitting a wall, and turning when climbing a slope. On flat grounds the robot wiggles in a snake-like manner, when moving at higher velocities. We also investigate the emergence of motor primitives, namely, the route to locomotion, which is characterized by a series of local and global bifurcations in terms of dynamical system theory.

An Admissible Heuristic to Improve Convergence in Kinodynamic Planners Using Motion Primitives

This letter introduces a new heuristic function that can be incorporated in any kinodynamic planner using motion primitives, to the purpose of increasing its convergence rate. The heuristic function is proven to be admissible and, hence, the optimality properties of the planning algorithm are preserved. Notably, it can be applied to planning problems with generic agent motion models and cost criteria, since it depends only on the database of motion primitives. The proposed heuristic has been integrated into a randomized sampling-based and a deterministic kinodynamic planner, and its effectiveness has been shown in numerical examples with different agent motion models and cost criteria.

A passive robot controller aiding human coaching for kinematic behavior modifications

Abstract Precise programming of robots for industrial tasks is inflexible to variations and time-consuming. Teaching a kinematic behavior by demonstration and encoding it with dynamical systems that are robust with respect to perturbations, is proposed in order to address this issue. Given a kinematic behavior encoded by Dynamic Movement Primitives (DMP), this work proposes a passive control scheme for assisting kinesthetic modifications of the learned behavior in task variations. It employs the utilization of penetrable spherical Virtual Fixtures (VFs) around the DMP’s virtual evolution that follows the teacher’s motion. The controller enables the user to haptically ‘inspect’ the spatial properties of the learned behavior in SE(3) and significantly modify it at any required segment, while facilitating the following of already learned segments. A demonstration within the VFs could signify that the kinematic behavior is taught correctly and could lead to autonomous execution, with the DMP generating the newly learned reference commands. The proposed control scheme is theoretically proved to be passive and experimentally validated with a KUKA LWR4+ robot. Results are compared with the case of using a gravity compensated robot agnostic of the previously learned task. It is shown that the time duration of teaching and the user’s cognitive load are reduced.

Human-Inspired Online Path Planning and Biped Walking Realization in Unknown Environment.

The focus of research in biped locomotion has moved toward real-life scenario applications, like walking on uneven terrain, passing through doors, climbing stairs and ladders. As a result, we are witnessing significant advances in the locomotion of biped robots, enabling them to move in hazardous environments while simultaneously accomplishing complex manipulation tasks. Yet, considering walking in an unknown environment, the efficiency of humanoid robots is still far from being comparable with the human. Currently, bipeds are very sensitive to external changes and they have severe constraints for adaptation of walk to conditions from such a complex environment. Promising approaches for efficient generation and realization of walking in a complex environment are based on biological solutions that have been developed for many years of evolution. This work presents one such human-inspired methodology for path planning and realization of biped walk appropriate for motion in a complex unfamiliar environment. Path planning results in calculating clothoid curves that represent well the human-like walking path. The robot walk is realized by the composition of parametric motion primitives. Such an approach enables on-line modification of planned path and walk parameters at any moment, instantly. To establish the relationship between high-level path planner and the low-level joint motion realization, we had to find a way to extract the parameters of the clothoid paths that can be linked with the parameters of the walk and consequently to motion primitive parameters. This enabled the robot to adopt its walking for avoiding the obstacles and for a smooth transition between different paths. In this paper we provide a complete framework that integrates the following components: (i) bio-inspired online path planning, (ii) path-dependent automatic calculation of high-level gait parameters (step length, walking speed, direction, and the height of the foot sole), and (iii) automatic calculation of low-level joint movements and corresponding control terms (driving motor voltage) through the adaptation of motion primitives which realize walking pattern and preserves the dynamic balance of the robot.

A Study on a Power Transmission Line Mobile Robot for Bundled Conductor Navigation

We introduces a mobile robot that can navigate on a power transmission line arranged in bundled conductors. The designs of the proposed robot are performed for navigation on bundled conductors, and the navigation method for bundled conductors and obstacle avoidance are presented. The robot consists of 13 degrees of freedom (DOF) with a symmetrical structure for the left and right parts, including the four wheel joints. The navigation method is designed using a combination of three motion primitives such as linear motion of counterbalancing box, linear motion of robot arm, and rotational motion of wheel part. To examine the performance of the proposed robot, navigation simulations are conducted using ADAMSTM. The robot navigations were simulated on obstacle environments that consisted of two- and four-conductor bundles. Based on the simulation results, the performance of the proposed robot was reviewed through the analysis of the trajectories of end-effectors. We confirmed that the proposed robot was capable of achieving optimal navigation on bundled conductors that included obstacles.