Motion Generation Research Articles

Reactive motion generation on learned Riemannian manifolds

In recent decades, advancements in motion learning have enabled robots to acquire new skills and adapt to unseen conditions in both structured and unstructured environments. In practice, motion learning methods capture relevant patterns and adjust them to new conditions such as dynamic obstacle avoidance or variable targets. In this paper, we investigate the robot motion learning paradigm from a Riemannian-manifold perspective. We argue that Riemannian manifolds may be learned via human demonstrations in which geodesics are natural motion skills. The geodesics are generated using a learned Riemannian metric produced by our novel variational autoencoder (VAE), which is intended to recover full-pose end-effector states and joint-space configurations. In addition, we propose a technique for facilitating on-the-fly end-effector/multiple-limb obstacle avoidance by reshaping the learned manifold using an obstacle-aware ambient metric. The motion generated using these geodesics may naturally result in multiple-solution tasks that have not been explicitly demonstrated previously. We extensively tested our approach in task-space and joint-space scenarios using a 7-DoF robotic manipulator. We demonstrate that our method is capable of learning and generating motion skills based on complicated motion patterns demonstrated by a human operator. Additionally, we assess several obstacle-avoidance strategies and generate trajectories in multiple-mode settings.

Increasing visitors attention with introductory portal technology to complex cultural sites

We investigate the effectiveness of visually impacting interfaces which were located next to the entrance of the San Martino Charterhouse in Naples (Italy), by using high quality 3D reconstructions annotated with semantic information. Semantic data were used to develop a gesture based orienting interface, for the generation of camera movements and pointing gestures for a 3D avatar. Using the Visitor Employed Photography protocol, we collected data about visitors noticing details and discovered that those exposed to the informative systems were able to detect more details than visitors who did not experience it.

Retracted: A Hybrid Density Network-Based Dance Movement Generation Algorithm and Internet of Things for Young Children

Retracted: A Hybrid Density Network-Based Dance Movement Generation Algorithm and Internet of Things for Young Children

Incorporating structural plasticity into self-organization recurrent networks for sequence learning.

Spiking neural networks (SNNs), inspired by biological neural networks, have received a surge of interest due to its temporal encoding. Biological neural networks are driven by multiple plasticities, including spike timing-dependent plasticity (STDP), structural plasticity, and homeostatic plasticity, making network connection patterns and weights to change continuously during the lifecycle. However, it is unclear how these plasticities interact to shape neural networks and affect neural signal processing. Here, we propose a reward-modulated self-organization recurrent network with structural plasticity (RSRN-SP) to investigate this issue. Specifically, RSRN-SP uses spikes to encode information, and incorporate multiple plasticities including reward-modulated spike timing-dependent plasticity (R-STDP), homeostatic plasticity, and structural plasticity. On the one hand, combined with homeostatic plasticity, R-STDP is presented to guide the updating of synaptic weights. On the other hand, structural plasticity is utilized to simulate the growth and pruning of synaptic connections. Extensive experiments for sequential learning tasks are conducted to demonstrate the representational ability of the RSRN-SP, including counting task, motion prediction, and motion generation. Furthermore, the simulations also indicate that the characteristics arose from the RSRN-SP are consistent with biological observations.

Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure.

Synthesizing human motion with a global structure, such as a choreography, is a challenging task. Existing methods tend to concentrate on local smooth pose transitions and neglect the global context or the theme of the motion. In this work, we present a music-driven motion synthesis framework that generates long-term sequences of human motions which are synchronized with the input beats, and jointly form a global structure that respects a specific dance genre. In addition, our framework enables generation of diverse motions that are controlled by the content of the music, and not only by the beat. Our music-driven dance synthesis framework is a hierarchical system that consists of three levels: pose, motif, and choreography. The pose level consists of an LSTM component that generates temporally coherent sequences of poses. The motif level guides sets of consecutive poses to form a movement that belongs to a specific distribution using a novel motion perceptual-loss. And the choreography level selects the order of the performed movements and drives the system to follow the global structure of a dance genre. Our results demonstrate the effectiveness of our music-driven framework to generate natural and consistent movements on various dance types, having control over the content of the synthesized motions, and respecting the overall structure of the dance.

ACTIVITY IN BARREL CORTEX RELATED TO TRACE EYEBLINK CONDITIONING.

In mammals several memory systems are responsible for learning and storage of associative memory. Even apparently simple behavioral tasks, like Pavlovian conditioning, have been suggested to engage, for instance, implicit and explicit memory processes. Here we used single whisker tactile trace eyeblink conditioning (TTEBC) to investigate learning and its neuronal bases in the mouse barrel column, the primary neocortical tactile representation of one whisker. Behavioral analysis showed that conditioned responses (CR) are spatially highly restricted, they generalize from the principal whisker only to its direct neighbors. Within the respective neural representation, the principal column and its direct neighbors, spike activity showed a learning-related spike rate suppression starting during the late phase of conditioning stimulus (CS) presentation that was sustained throughout the stimulus-free trace period (Trace). Trial-by-trial analysis showed that learning-related activity was independent from the generation of eyelid movements within a trial, and set in around the steepest part of the learning curve. Optogenetic silencing of responses and their learning-related changes during CS and Trace epochs blocked CR acquisition, but not its recall after learning. Silencing during the Trace alone, which carried major parts of the learning-related changes, had no effect. In summary, we demonstrate specific barrel column spike rate plasticity during TTEBC that can be partially decoupled from the CR, the learned eye closure, a hallmark of implicit learning. Our results, thus, point to a possible role of the barrel column in contributing to other kinds of memory as well.Significance StatementAssociation learning relies on different memory systems. The Pavlovian eyeblink reflex conditioning paradigm already involves the generation of a new movement ('move the eyelid', implicit memory), and to learn the task contingencies ('know the rule of the game', explicit memory). Using this paradigm, we study the involvement of whisker-related mouse primary somatosensory cortex (S1) in different memory systems. With recording and causal stimulation techniques, we found spike plasticity in S1 during presentation of the conditioned stimulus that is causal for implicit learning. Notably, however, blocking learning-related changes during stimulus-free trace period did not impair implicit memory. We conclude that S1 activity is related to the implicit and at least one other memory system, presumably the explicit one.

Heat Generation and Thermal Radiation Effects on Magneto Hydrodynamics Non Newtonian Casson Nanofluid with Gyro Tactic Microorganisms Over a Plate, Stagnation and Wedge Through Porous Media

The numerical interventions of two dimensional steady flow of MHD Non-Newtonian nanofluids containing the gyro-tactic microorganisms through porous media over a plate, wedge, and stagnation point are highlighted in this paper. Mainly the Peclet number, bioconvection, Brownian motion, thermophoresis, and heat generation impacts are addressed to consolidate thermal and nanofluid concentration conservative equations with passively controlled boundary conditions for three different geometrical conditions of flow over a plate, wedge, and stagnation point. By considering the impacts of the varying pertinent parameters, namely thermophoresis, Brownian motion, Prandtl number, heat generation, chemical reaction, bio convectional and magnetic parameters, results are analysed graphically for the momentum, temperature, nanoparticle volume fractions, and the density of motile microorganisms profile, as well as the local Nusselt and motile microorganism numbers. Relevant similarity transformations are used to obtain the system of ordinary differential equations and the equations are solved numerically by using Bvp4c via MATLAB based on the shooting technique.

Hydrodynamics of an odd active surfer in a chiral fluid

We theoretically and computationally study the low-Reynolds-number hydrodynamics of a linear active microswimmer surfing on a compressible thin fluid layer characterized by an odd viscosity. Since the underlying three-dimensional fluid is assumed to be very thin compared to any lateral size of the fluid layer, the model is effectively two-dimensional. In the limit of small odd viscosity compared to the even viscosities of the fluid layer, we obtain analytical expressions for the self-induced flow field, which includes non-reciprocal components due to the odd viscosity. On this basis, we fully analyze the behavior of a single linear swimmer, finding that it follows a circular path, the radius of which is, to leading order, inversely proportional to the magnitude of the odd viscosity. In addition, we show that a pair of swimmers exhibits a wealth of two-body dynamics that depends on the initial relative orientation angles as well as on the propulsion mechanism adopted by each swimmer. In particular, the pusher–pusher and pusher–puller-type swimmer pairs exhibit a generic spiral motion, while the puller–puller pair is found to either co-rotate in the steady state along a circular trajectory or exhibit a more complex chaotic behavior resulting from the interplay between hydrodynamic and steric interactions. Our theoretical predictions may pave the way toward a better understanding of active transport in active chiral fluids with odd viscosity, and may find potential applications in the quantitative microrheological characterization of odd-viscous fluids.

Dynamic Neural Network for Bicriteria Weighted Control of Robot Manipulators.

In recent years, bicriteria optimization schemes for manipulator control have become preferred by researchers, given their satisfactory performance. In this article, a bicriteria weighted (BCW) scheme to remedy joint drift and minimize the infinity norm of joint velocity is proposed. The scheme adopts a novel repetitive motion index that can theoretically decouple the joint error and the position error, which many conventional cyclic motion generation schemes cannot achieve. Subsequently, through transformation, the BCW scheme is converted into a time-varying quadratic programming (QP) problem. Then, a dynamic neural network (DNN) system with a new Fisher-Burmeister function is proposed to address the resulting QP problem. It is proven that the proposed DNN system is free of residual errors, which means that the actual solution is able to converge to the theoretical solution. Another essential feature of the DNN system is that it has a suppression effect on noise. To demonstrate the convergence and robustness of the proposed DNN system, comparative simulations are carried out in nominal and noisy environments. Finally, experiments on Franka Emika Panda are conducted to elucidate the availability of the BCW scheme addressed by the DNN system.

Dynamic Optimization Fabrics for Motion Generation

Optimization fabrics are a geometric approach to real-time local motion generation, where motions are designed by the composition of several differential equations that exhibit a desired motion behavior. We generalize this framework to dynamic scenarios and nonholonomic robots and prove that fundamental properties can be conserved. We show that convergence to desired trajectories and avoidance of moving obstacles can be guaranteed using simple construction rules of the components. In addition, we present the first quantitative comparisons between optimization fabrics and model predictive control and show that optimization fabrics can generate similar trajectories with better scalability, and thus, much higher replanning frequency (up to 500 Hz with a 7 degrees of freedom robotic arm). Finally, we present empirical results on several robots, including a nonholonomic mobile manipulator with 10 degrees of freedom and avoidance of a moving human, supporting the theoretical findings.

Example-based Motion Synthesis via Generative Motion Matching

We present GenMM, a generative model that "mines" as many diverse motions as possible from a single or few example sequences. In stark contrast to existing data-driven methods, which typically require long offline training time, are prone to visual artifacts, and tend to fail on large and complex skeletons, GenMM inherits the training-free nature and the superior quality of the well-known Motion Matching method. GenMM can synthesize a high-quality motion within a fraction of a second, even with highly complex and large skeletal structures. At the heart of our generative framework lies the generative motion matching module, which utilizes the bidirectional visual similarity as a generative cost function to motion matching, and operates in a multi-stage framework to progressively refine a random guess using exemplar motion matches. In addition to diverse motion generation, we show the versatility of our generative framework by extending it to a number of scenarios that are not possible with motion matching alone, including motion completion, key frame-guided generation, infinite looping, and motion reassembly.

Humanoid motion generation in a world of stairs

Consider the problem of generating humanoid motions in an environment consisting of horizontal patches located at different heights (world of stairs). To this end, the paper proposes an integrated scheme which combines footstep planning and gait generation. In particular, footsteps are produced by a randomized algorithm that guarantees both feasibility and quality of the plan according to a chosen criterion; whereas for 3D gait generation we devise an ad hoc extension of the Intrinsically Stable MPC scheme. In its basic form, the proposed scheme addresses the off-line case (known environments), but a sensor-based adaptation is developed for the on-line case (unknown environments) based on an anytime version of the footstep planner. In order to validate the proposed approach, we present simulations in CoppeliaSim for the HRP-4 humanoid robot navigating scenarios of different complexity, both in the on-line and off-line case.

Ageing reduces persistent inward current contribution to motor neurone firing: Potential mechanisms and the role of exercise.

Nervous system deterioration is a primary driver of age-related motor impairment. The motor neurones, which act as the interface between the central nervous system and the muscles, play a crucial role in amplifying excitatory synaptic input to produce the desired motor neuronal firing output. For this, they utilise their ability to generate persistent (long-lasting) depolarising currents that increase cell excitability, and both amplify and prolong the output activity of motor neurones for a given synaptic input. Modulation of these persistent inward currents (PICs) contributes to the motor neurones' capacities to attain the required firing frequencies and rapidly modulate them to competently complete most tasks. Thus, PICs are crucial for adequate movement generation. Impairments in intrinsic motor neurone properties can impact motor unit firing capacity, with convincing evidence indicating that the PIC contribution to motor neurone firing is reduced in older adults. Indeed, this could be an important mechanism underpinning the age-related reductions in strength and physical function. Furthermore, resistance training has emerged as a promising intervention to counteract age-associated PIC impairments, with changes in PICs being correlated with improvements in muscular strength and physical function after training. In this review, we present the current knowledge of the PIC magnitude decline during ageing and discuss whether reduced serotonergic and noradrenergic input onto the motor neurones, voltage-gated calcium channel dysfunction or inhibitory input impairments are candidates that: (i) explain age-related reductions in the PIC contribution to motor neurone firing and (ii) underpin the enhanced PIC contribution to motor neurone firing following resistance training in older adults.

Multimodal attention for lip synthesis using conditional generative adversarial networks

The synthesis of lip movements is an important problem for a socially interactive agent (SIA). It is important to generate lip movements that are synchronized with speech and have realistic co-articulation. We hypothesize that combining lexical information (i.e., sequence of phonemes) and acoustic features can lead not only to models that generate the correct lip movements matching the articulatory movements, but also to trajectories that are well synchronized with the speech emphasis and emotional content. This work presents attention-based frameworks that use acoustic and lexical information to enhance the synthesis of lip movements. The lexical information is obtained from automatic speech recognition (ASR) transcriptions, broadening the range of applications of the proposed solution. We propose models based on conditional generative adversarial networks (CGAN) with self-modality attention and cross-modalities attention mechanisms. These models allow us to understand which frames are considered more in the generation of lip movements. We animate the synthesized lip movements using blendshapes. These animations are used to compare our proposed multimodal models with alternative methods, including unimodal models implemented with either text or acoustic features. We rely on subjective metrics using perceptual evaluations and an objective metric based on the LipSync model. The results show that our proposed models with attention mechanisms are preferred over the baselines on the perception of naturalness. The addition of cross-modality attentions and self-modality attentions has a significant positive impact on the performance of the generated sequences. We observe that lexical information provides valuable information even when the transcriptions are not perfect. The improved performance observed by the multimodal system confirms the complementary information provided by the speech and text modalities.

Simulation of sports training recognition system based on internet of things video behavior analysis

The video behavior analysis of the Internet of Things is the apparent characteristics and continuous state of the two-dimensional human body in the time dimension. The patterns representing human behavior can be divided into states based on human body structure, forms based on common features and forms based on temporal features, local spaces, and behaviors based on deep learning. Applying IoT video behavior analysis to sports training will have a huge impact. Physical training is the process of training people to improve their physical fitness, increase courage and endurance, and acquire practical skills. It is the basis of various training courses and an important way to improve human physical fitness. The motion recognition system is a DTW pattern matching method, which uses a finite state to describe the spatial characteristics of the angle between the joint points. This method can accurately meet the low-delay requirements of the motion recognition of each frame and the standard motion. It can match and eliminate the influence of individual differences on motion recognition. At the same time, it can also effectively expand the pattern matching method to recognize newly filled motions, and it is a powerful composite general motion recognition system.

Connectivity-based Seismic Response Analysis of Water Transmission Networks

Recent frequent earthquakes have caused serious damage to water supply structures, significantly affecting the drinking water supply of local residents. In particular, because water supply structures are extensively buried underground, prediction of their deteriorated performance against external disturbances is crucial for maintenance purpose. Therefore, this study aims to evaluate the degraded seismic performance of the water transmission network based on connectivity-based network analysis against seismic conditions. The proposed methodology consists of i) network map reconstruction using a graph theory, ii) ground motion generation considering the uncertainty of seismic motion, and iii) connectivity verification between source and sink nodes. To demonstrate the proposed methodology, the network topology was reconstructed by adopting the water transmission network located in A-city, and the seismic performance of the target network was evaluated considering the epicenter of the historical earthquakes. As a result of numerical analysis, the seismic response of the water transmission network showed the highest network connectivity when the water was supplied from node 62, and water was stably secured at node 107 under different earthquake magnitudes.

Composable energy policies for reactive motion generation and reinforcement learning

In this work, we introduce composable energy policies (CEP), a novel framework for multi-objective motion generation. We frame the problem of composing multiple policy components from a probabilistic view. We consider a set of stochastic policies represented in arbitrary task spaces, where each policy represents a distribution of the actions to solve a particular task. Then, we aim to find the action in the configuration space that optimally satisfies all the policy components. The presented framework allows the fusion of motion generators from different sources: optimal control, data-driven policies, motion planning, and handcrafted policies. Classically, the problem of multi-objective motion generation is solved by the composition of a set of deterministic policies, rather than stochastic policies. However, there are common situations where different policy components have conflicting behaviors, leading to oscillations or the robot getting stuck in an undesirable state. While our approach is not directly able to solve the conflicting policies problem, we claim that modeling each policy as a stochastic policy allows more expressive representations for each component in contrast with the classical reactive motion generation approaches. In some tasks, such as reaching a target in a cluttered environment, we show experimentally that CEP additional expressivity allows us to model policies that reduce these conflicting behaviors. A field that benefits from these reactive motion generators is the one of robot reinforcement learning. Integrating these policy architectures with reinforcement learning allows us to include a set of inductive biases in the learning problem. These inductive biases guide the reinforcement learning agent towards informative regions or improve collision safety while exploring. In our work, we show how to integrate our proposed reactive motion generator as a structured policy for reinforcement learning. Combining the reinforcement learning agent exploration with the prior-based CEP, we can improve the learning performance and explore safer.

Point Pair-Based Expression of Cutter Swept Envelopes in Five-Axis Milling

This paper proposes an efficient method of formulating cutter swept envelopes in five-axis milling based on conformal geometric algebra. The surface of the rotary cutter is represented as a canal surface, as the envelope of a one-parameter family of spheres. The swept envelopes generated by the general rigid motion of the cutter can be regarded as the envelopes of a two-parameter family of spheres. With the help of the dual representations of spheres and planes in conformal space, a point pair-based expression of the swept envelopes is formulated analytically, which represents two surfaces uniformly. By using the present method, the swept surfaces of a conical cutter, a drum-shaped cutter, and a toroidal cutter are illustrated. As an application, two types of collision detection methods for 5-axis milling are presented based on the point pair-based envelope expression. The benchmark examples demonstrate that the use of the proposed envelope expression significantly improves computational efficiency.

FLAME: Free-Form Language-Based Motion Synthesis & Editing

Text-based motion generation models are drawing a surge of interest for their potential for automating the motion-making process in the game, animation, or robot industries. In this paper, we propose a diffusion-based motion synthesis and editing model named FLAME. Inspired by the recent successes in diffusion models, we integrate diffusion-based generative models into the motion domain. FLAME can generate high-fidelity motions well aligned with the given text. Also, it can edit the parts of the motion, both frame-wise and joint-wise, without any fine-tuning. FLAME involves a new transformer-based architecture we devise to better handle motion data, which is found to be crucial to manage variable-length motions and well attend to free-form text. In experiments, we show that FLAME achieves state-of-the-art generation performances on three text-motion datasets: HumanML3D, BABEL, and KIT. We also demonstrate that FLAME’s editing capability can be extended to other tasks such as motion prediction or motion in-betweening, which have been previously covered by dedicated models.

Subcritical superprocesses conditioned on non-extinction

We consider a class of subcritical superprocesses (Xt)t≥0 with general spatial motions and general branching mechanisms. We study the asymptotic behaviors of Qt,r, the distribution of Xt conditioned on Xt+r not being a null measure. We first give the existence of limt→∞Qt,r and limr→∞Qt,r, and then show that an LlogL-type condition is equivalent to the existence of the iterated limits: limr→∞limt→∞Qt,r and limt→∞limr→∞Qt,r. Finally, when the LlogL-type condition holds, we show that those iterated limits, and the double limit limr,t→∞Qt,r, are the same.