Motion Generation Research Articles

MusicFace: Music-driven expressive singing face synthesis

It remains an interesting and challenging problem to synthesize a vivid and realistic singing face driven by music. In this paper, we present a method for this task with natural motions for the lips, facial expression, head pose, and eyes. Due to the coupling of mixed information for the human voice and backing music in common music audio signals, we design a decouple-and-fuse strategy to tackle the challenge. We first decompose the input music audio into a human voice stream and a backing music stream. Due to the implicit and complicated correlation between the two-stream input signals and the dynamics of the facial expressions, head motions, and eye states, we model their relationship with an attention scheme, where the effects of the two streams are fused seamlessly. Furthermore, to improve the expressivenes of the generated results, we decompose head movement generation in terms of speed and direction, and decompose eye state generation into short-term blinking and long-term eye closing, modeling them separately. We have also built a novel dataset, SingingFace, to support training and evaluation of models for this task, including future work on this topic. Extensive experiments and a user study show that our proposed method is capable of synthesizing vivid singing faces, qualitatively and quantitatively better than the prior state-of-the-art.

Sign Language Dataset for Automatic Motion Generation.

Several sign language datasets are available in the literature. Most of them are designed for sign language recognition and translation. This paper presents a new sign language dataset for automatic motion generation. This dataset includes phonemes for each sign (specified in HamNoSys, a transcription system developed at the University of Hamburg, Hamburg, Germany) and the corresponding motion information. The motion information includes sign videos and the sequence of extracted landmarks associated with relevant points of the skeleton (including face, arms, hands, and fingers). The dataset includes signs from three different subjects in three different positions, performing 754 signs including the entire alphabet, numbers from 0 to 100, numbers for hour specification, months, and weekdays, and the most frequent signs used in Spanish Sign Language (LSE). In total, there are 6786 videos and their corresponding phonemes (HamNoSys annotations). From each video, a sequence of landmarks was extracted using MediaPipe. The dataset allows training an automatic system for motion generation from sign language phonemes. This paper also presents preliminary results in motion generation from sign phonemes obtaining a Dynamic Time Warping distance per frame of 0.37.

On the application of spatial four-bar motion and axode generation for the design of a prosthetic canine knee joint

The novelty of this work is that for the first time, spatial four-bar linkage motion generation and axode generation models have been applied for the design of a joint to replicate natural canine knee motion. This canine knee design method incorporates spatial four-bar Revolute-Revolute-Spherical-Spherical linkage motion generation and axode generation models to ultimately produce the rolling cam surfaces needed to produce natural spatial canine knee motion. This work presents the latest findings from an ongoing study where a transfemoral prosthetic knee design method is being applied to produce concept prosthetic knee joints for canines that replicate natural spatial knee motion during treadmill gait. To date, this study has considered three canine subjects, ranging from ages 1.1 to 10 years and treadmill speeds ranging from 1.3 to 2.9 km/h.

Autonomous nanorobots with powerful thrust under dry solid-contact conditions by photothermal shock

Nanorobotic motion on solid substrates is greatly hindered by strong nanofriction, and powerful nanomotors‒the core components for nanorobotic motion‒are still lacking. Optical actuation addresses power and motion control issues simultaneously, while conventional technologies with small thrust usually apply to fluid environments. Here, we demonstrate micronewton-thrust nanomotors that enable the autonomous nanorobots working like conventional robots with precise motion control on dry surfaces by a photothermal-shock technique. We build a pulsed laser-based actuation and trapping platform, termed photothermal-shock tweezers, for general motion control of metallic nanomaterials and assembled nanorobots with nanoscale precision. The thrust-to-weight ratios up to 107 enable nanomotors output forces to interact with external micro/nano-objects. Leveraging machine vision and deep learning technologies, we assemble the nanomotors into autonomous nanorobots with complex structures, and demonstrate multi-degree-of-freedom motion and sophisticated functions. Our photothermal shock-actuation concept fundamentally addresses the nanotribology challenges and expands the nanorobotic horizon from fluids to dry solid surfaces.

Sign Language Motion Generation from Sign Characteristics.

This paper proposes, analyzes, and evaluates a deep learning architecture based on transformers for generating sign language motion from sign phonemes (represented using HamNoSys: a notation system developed at the University of Hamburg). The sign phonemes provide information about sign characteristics like hand configuration, localization, or movements. The use of sign phonemes is crucial for generating sign motion with a high level of details (including finger extensions and flexions). The transformer-based approach also includes a stop detection module for predicting the end of the generation process. Both aspects, motion generation and stop detection, are evaluated in detail. For motion generation, the dynamic time warping distance is used to compute the similarity between two landmarks sequences (ground truth and generated). The stop detection module is evaluated considering detection accuracy and ROC (receiver operating characteristic) curves. The paper proposes and evaluates several strategies to obtain the system configuration with the best performance. These strategies include different padding strategies, interpolation approaches, and data augmentation techniques. The best configuration of a fully automatic system obtains an average DTW distance per frame of 0.1057 and an area under the ROC curve (AUC) higher than 0.94.

Sampling-Based Motion Planning: A Comparative Review

Sampling-based motion planning is one of the fundamental paradigms to generate robot motions, and a cornerstone of robotics research. This comparative review provides an up-to-date guide and reference manual for the use of sampling-based motion planning algorithms. It includes a history of motion planning, an overview of the most successful planners, and a discussion of their properties. It also shows how planners can handle special cases and how extensions of motion planning can be accommodated. To put sampling-based motion planning into a larger context, a discussion of alternative motion generation frameworks highlights their respective differences from sampling-based motion planning. Finally, a set of sampling-based motion planners are compared on 24 challenging planning problems in order to provide insights into which planners perform well in which situations and where future research would be required. This comparative review thereby provides not only a useful reference manual for researchers in the field but also a guide for practitioners to make informed algorithmic decisions.

A novel finite element model in nonlocal transient analysis of viscoelastic functionally graded porous nanoplates resting on viscoelastic medium

The transient analysis of viscoelastic functionally graded porous (FGP) nanoplate resting on the viscoelastic medium using the finite element method (FEM) is studied in this work. The nanoplate is made from a material that varies in thickness and has mechanical qualities that change depending on the porosity. The whole of the mechanical system rests on a Pasternak medium, which is modeled in accordance with a Kelvin–Voigt viscoelastic model. The general plate motion balance equations are developed by the application of the higher‐order shear plate theory (HSDT) with a new inverse hyperbolic function, the nonlocal hypothesis, and Hamilton's principle. In order to develop a four‐node quadrilateral element, Lagrangian and Hermitian interpolation functions are used. These functions are employed to define the main variables that correspond to the in‐plane displacements and the transverse displacements, respectively. The direct integration method developed by Newmark is used to calculate the dynamic responses of the plate. The outcomes that were anticipated by the essay have been confirmed by reputable sources. The dynamic response characteristics of a viscoelastic FGP nanoplates that is placed on a viscoelastic medium are explored, and a number of factors that influence these features are discovered and explained.

The Influence of Supply Channel Design on the Gas-Dynamic Structure of Air Flow in a Vertical Conical Diffuser

Vertical conical diffusers are used in power engineering, chemical industry, technological processes, and other industries. The efficiency of many machines and pieces of equipment is determined by the gas-dynamic and heat-exchange perfection of processes in diffusers. This study assesses the influence of the air supply method on flow structure in a diffuser. The studies were carried out on a test bench with thermal imaging for air flow rates ranging from 0.018 to 0.057 m3/s (42,500 < Re < 150,000). Two designs were examined: (1) a conventional air supply through one channel from below and (2) a nozzle air supply through four tubes at an angle of 45° to the vertical axis. In addition, the influence of the cross-sectional shape of the supply channels of both designs was studied. It is established that the use of a conventional air supply through one channel leads to the generation of a pronounced central flow along the vertical axis (all configurations of the supply channel) and the creation of stagnant zones in the corners of the diffuser (round and triangular channels; the use of a square supply channel causes the most uniform air distribution throughout the entire volume of the diffuser (while maintaining the central flow). It is found that with nozzle air supply, there are no stagnant zones in the corners and intense air movement generation in the centre of the diffuser (round and triangular tubes) can be observed; the use of square nozzle tubes causes intense flow movement at the base of the diffuser, which quickly collapses upstream, uniformly filling the entire volume of the diffuser’s cylindrical part. The presented data can be useful for designing various machines and pieces of equipment with vertical conical diffusers.

An Integrated Kinematic Mapping and Fourier Method to Design Spherical Coupled Serial Chain Mechanisms for Single-Joint Rehabilitation

Abstract Robotic devices are capable of reducing the physical burden on rehabilitation therapists and providing training programs of good repeatability, high efficiency, and high precision. When designing the kinematic structure for rehabilitation robots, there has been a growing interest toward one-degree-of-freedom (DOF) end-effector mechanisms due to their simpler structure and less complicated control algorithms. Compared with current one-DOF mechanism designs that are mainly customized for multi-joint robotic training, spherical coupled serial chain (SCSC) mechanisms are proposed in this paper to specifically deliver the single-joint robotic training, which is of equal importance to the effective physical recovery. Using kinematic mapping theory, the end-effector motion of SCSC mechanisms can be naturally transformed to two trigonometric curves composed of finite Fourier series in two separate planes of the image space. This novel formulation helps to establish an analytical and direct relationship between the design parameters of the SCSC mechanism and the harmonic parameters of the image-space representation of the task rehabilitation motion. The result is a simple and effective method for kinematic synthesis of SCSC mechanisms for generation of single-joint motion with an arbitrary number of spherical poses. An example of designing SCSC mechanisms for shoulder-joint rehabilitation is presented at the end of this paper to illustrate the feasibility of the proposed method.

Dimensional synthesis of motion generation in a spherical four-bar mechanism

A new objective function for optimizing spherical, four-bar mechanisms for motion generation is presented. Rigid-body poses of a spherical four-bar mechanism in a standard installation position are investigated, and a normalization processing method is proposed. After processing the coupler points of the mechanism, the feature points generated are located on the circles. The formation principle of the feature coupler circles is analysed, and the internal relationship between the centre angle of the adjacent feature points on the feature coupler circles and the coupler angle of the spherical four-bar mechanism is determined. An input angle determination method is proposed for the spherical four-bar mechanism in a general installation position. An objective function is then established to optimize the basic dimensional types, relative input angles, and installation position parameters of the desired spherical four-bar mechanism. The proposed method is applicable to both prescribed and unprescribed timing problems. Notably, the dimension of the optimization variables is only eight, which is exceptionally low for multiple position motion generation without prescribed timing; therefore, the global optimal solution is more easily obtained. The optimization process is carried out by the genetic algorithm. The feasibility and effectiveness of our proposed method are demonstrated by examples.

Tension Between Theory and Practice of Replication

A core problem that has been addressed in the scientific reform movement so far is the low rates of reproducibility of research results. Mainstream reform literature has aimed at increasing reproducibility rates by implementing procedural changes in research practice and scientific policy. At the sidelines of reform, theoreticians have worked on understanding the underlying causes of irreproducibility from the ground up. Each approach faces its own challenges. While the mainstream focus on swift practical changes has not been buttressed by sound theoretical arguments, theoretical work is slow and initially is only capable of answering questions in idealized setups, removed from real life constraints. In this article, we continue to develop theoretical foundations in understanding non-exact replications and meta-hypothesis tests in multi-site replication studies, juxtapose these theoretical intuitions with practical reform examples, and expose challenges we face. In our estimation, a major challenge in the next generation of the reform movement is to bridge the gap between theoretical knowledge and practical advancements.

Tracking a moving visual target in the rhesus monkey: influence of the occurrence frequency of the target path.

Following previous studies documenting the ability to generate anticipatory responses, we tested whether the repeated motion of a visual target along the same path affected its oculomotor tracking. In six rhesus monkeys, we evaluated how the frequency of a target path influenced the onset, accuracy, and velocity of eye movements. Three hundred milliseconds after its extinction, a central target reappeared and immediately moved toward the periphery in four possible (oblique) directions and at a constant speed (20°/s or 40°/s). During each daily session, the frequency of one motion direction was either uncertain (25% of trials) or certain (100% of trials). Our results show no reduction of saccade latency between the two sessions. No express saccades were observed in either session. A slow eye movement started after target onset (presaccadic glissade) and its velocity was larger during the "certain" sessions only with the 40°/s target. No anticipatory eye movement was observed. Longer intersaccadic intervals were found during the "certain" sessions but the postsaccadic pursuit velocity exhibited no change. No correlation was found between the accuracy and precision of saccades (interceptive or catch-up) and the postsaccadic pursuit velocity. Repeatedly tracking a target that moves always along the same path does not favor the generation of anticipatory eye movements, saccadic or slow. Their occurrence is not spontaneous but seems to require preliminary training. Finally, for both sessions, the lack of correlation between the saccade-related and pursuit-related kinematic parameters is consistent with separate control of saccadic and slow eye movements.NEW & NOTEWORTHY Following previous studies documenting anticipatory movements, we investigated how the frequency of occurrence of a target path influenced the generation of tracking eye movements. When present, the effects were small. The limited performance that we found suggests that anticipatory responses require preliminary training, in which case, they should not be considered as a behavioral marker of the primates' ability to extrapolate but the outcome of learning and remembering past experience.

Collision-Free Motion Generation Based on Stochastic Optimization and Composite Signed Distance Field Networks of Articulated Robot

Collision-Free Motion Generation Based on Stochastic Optimization and Composite Signed Distance Field Networks of Articulated Robot

Decomposition and Reconstruction of Human Palm Movements.

The human hand is known to have excellent manipulation ability compared to other primate hands. Without the palm movements, the human hand would lose more than 40% of its functions. However, uncovering the constitution of palm movements is still a challenging problem involving kinesiology, physiology, and engineering science. By recording the palm joint angles during common grasping, gesturing, and manipulation tasks, we built a palm kinematic dataset. Then, a method for extracting the eigen-movements to characterize the common motion correlation relationships of palm joints was proposed to explore the palm movement constitution. This study revealed a palm kinematic characteristic that we named the joint motion grouping coupling characteristic. During natural palm movements, there are several joint groups with a high degree of motor independence, while the movements of joints within each joint group are interdependent. Based on these characteristics, the palm movements can be decomposed into seven eigen-movements. The linear combinations of these eigen-movements can reconstruct more than 90% of palm movement ability. Moreover, combined with the palm musculoskeletal structures, we found that the revealed eigen-movements are associated with joint groups that are defined by muscular functions, which provided a meaningful context for palm movement decomposition. This paper suggests that some invariable characteristics underlie the variable palm motor behaviors and can be used to simplify palm movement generation. This paper provides important insights into palm kinematics, and helps facilitate motor function assessment and the development of better artificial hands.

The effects of selected object features on a pick-and-place task: A human multimodal dataset

We propose a dataset to study the influence of object-specific characteristics on human pick-and-place movements and compare the quality of the motion kinematics extracted by various sensors. This dataset is also suitable for promoting a broader discussion on general learning problems in the hand-object interaction domain, such as intention recognition or motion generation with applications in the Robotics field. The dataset consists of the recordings of 15 subjects performing 80 repetitions of a pick-and-place action under various experimental conditions, for a total of 1200 pick-and-places. The data has been collected thanks to a multimodal setup composed of multiple cameras, observing the actions from different perspectives, a motion capture system, and a wrist-worn inertial measurement unit. All the objects manipulated in the experiments are identical in shape, size, and appearance but differ in weight and liquid filling, which influences the carefulness required for their handling.

Cholinergic centro-cingulate network in Parkinson disease and normal aging.

Decreased cholinergic binding within the recently identified centro-cingulate brain network robustly has been shown to robustly correlate with the severity of cognitive impairment in Parkinson disease (PD). This network with key hubs within the cingulum, operculum and peri-central cortical regions also correlates with elements of parkinsonian motor impairments, including postural instability and gait difficulties, such as falls or freezing. MRI neuroimaging studies have shown that the anterior midcingulate cortex is a key node for cognitive aspects of movement generation, i.e., intentional motor control. Recent evidence also suggests a novel aspect of organization of primary motor cortex, describing "effector" regions for fine movement control intercalated with interlinked "inter-effector" regions devoted to whole-body control. A distinguishing feature of inter-effector regions is tight linkage to the cingular and opercular regions. Such inter-effector regions have been proposed to be part of a greater somato-cognitive action network necessary for integration of goals and movement. Recent evidence also points to vulnerabilities of cholinergic nerve terminals in the centro-cingulate network in older non-PD adults. These features of normal aging underscore that cortical cholinergic terminal losses in age-associated neurodegenerative disorders are likely not exclusively the result of disease-specific etiologies but also related to otherwise normal aging. Practical implications of this overlap are that addressing disease-specific and general aging etiologies involved in neurodegeneration, may be of benefit in age-associated neurodegenerative disorders where significant cholinergic systems degeneration is present.

Eyeball translations affect saccadic eye movements beyond brainstem control.

Vision requires that we rotate our eyes frequently to look at informative structures in the scene. Eye movements are planned by the brain but their execution depends on the mechanical properties of the oculomotor plant, that is, the arrangement of eyeball position, muscle insertions, and pulley locations. Therefore, the biomechanics of rotations is sensitive to eyeball translation because it changes muscle levers. Eyeball translations are little researched as they are difficult to measure with conventional techniques. Here, we investigated the effects of eyeball translation on the coordination of eyeball rotation by high-speed MRI recordings of saccadic eye movements during blinks, which are known to produce strong translations. We found that saccades during blinks massively overshoot their targets and that these overshoots occur in a transient fashion such that the gaze is back on target at the time the blink ends. These dynamic overshoots were tightly coupled to the eyeball translation, both in time and in size. Saccades made without blinks were also accompanied by small amounts of transient eyeball retraction, the size of which scaled with saccade amplitude. These findings demonstrate a complex combination of rotation and translation of the eye. The mechanical consequences of eyeball translation on oculomotor control should be considered along with the neural implementation in the brainstem to understand the generation of eye movements and their disorders.NEW & NOTEWORTHY We found that saccades during blinks can massively overshoot their target when the eyeball is retracted. Our data imply that the overshoots are not part of the saccade plan prepared in the brainstem, but instead a consequence of the altered biomechanics resulting from concurrent eyeball translation and rotation. To our best knowledge, this is the first direct observation of dynamic properties of the oculomotor plant altering the execution of rotational eye movements.

Structural equation modeling of female gait attractiveness using gait kinematics

In our social lives, movement’s attractiveness greatly affects interpersonal cognition, and gait kinematics mediates walkers’ attractiveness. However, no model using gait kinematics has so far predicted gait attractiveness. Thus, this study constructed models of female gait attractiveness with gait kinematics and physique factors as explanatory variables for both barefoot and high-heel walking. First, using motion capture data from 17 women walking, including seven professional runway models, we created gait animations. We also calculated the following gait kinematics as candidate variables to explain walking’s attractiveness: four body-silhouette-related variables and six health-related variables. Then, 60 observers evaluated each gait animation’s attractiveness and femininity. We performed correlation analysis between these variables and evaluation scores to obtain explanatory variables. Structural equation modeling suggested two models for gait attractiveness, one composed of trunk and head silhouette factors and the other of physique, trunk silhouette, and health-related gait factors. The study’s results deepened our understanding of mechanisms behind nonverbal interpersonal cognition through physical movement and brought us closer to realization of artificial generation of attractive gait motions.

Simulation of non-stationary ground motions and its applications in high concrete faced rockfill dams via direct probability integral method

Many high concrete face rockfill dams (CFRDs) have been constructed in earthquake-prone areas in recent years, and it is essential to evaluate the seismic safety performance of these dams considering the randomness of seismic excitation. In this paper, the relationship between the stochastic ground motion generation method and the direct probability integral method (DPIM) is established by using the number theory point selection method as the bridge. The DPIM is introduced into the field of stochastic dynamic analysis and multi-index reliability evaluation of high CFRD for the first time. First, a non-stationary stochastic excitation model at five sites is established based on the latest standard for the seismic design of hydraulic structures. Then, taking a practical project as an example, stochastic dynamic response analysis and reliability assessment based on dam crest settlement deformation and dam slope cumulative slippage are conducted by using DIPM. Finally, the influence of these two indexes on reliability is explored by drawing the equal failure probability curve. The results show that the stochastic dynamic analysis and reliability assessment framework established in this paper can provide probabilistic seismic safety assessments for high CFRDs. Sliding damage of the dam slope is more likely than the deformation damage of the rockfill from the perspective of probability.

The E16_Geometric Redesign of Road Parongpong-Ngamprah STA. 0+750-1+500 Based on Atraction-Generation Interzone Movement

The increase in movement between zones between sub-districts in the Regency increases in line with economic development in the region. Parongpon sub-district and Ngamprah sub-district are West Bandung regency areas that have increased economic growth which is quite significant in West Bandung Regency. The geometric redesign research between Parongpong and Ngamprah zones has the aim of adjusting the road geometry based on increasing the volume generation and attraction of traffic movement between these zones. The generation and attraction of traffic movement between these zones is dominated by the volume of vehicles with larger dimensions and heavier loads, so that geometric alignment of the road is required based on the standard slope of the vertical alignment and the super-elevation curve of the horizontal alignment. The research method used is quantitative descriptive in geometric design parameters which are developed in accordance with the development of traffic volume and vehicle types. The research results show the Rise and Attract model of movement between Parongpong-Ngamprah zones with the mathematical model shown by the equation Y = 28.064 +0.0003612X1+0.001X2+0.005X3 with R2 = 0.97. With the influence of the generation of traffic volume movement on these roads, of requires adjustment of the super evaluation curve with a larger radius and smaller vertical alignment slope so that the increase in traffic volume from the influence of the generation and attraction of the movement of the Parongpong-Ngamprah zone can be anticipated with fine.