Motion Sequences Research Articles

A Distributed Deadlock-Free Task Offloading Algorithm for Integrated Communication–Sensing–Computing Satellites with Data-Dependent Constraints

Integrated communication–sensing–computing (ICSC) satellites, which integrate edge computing servers on Earth observation satellites to process collected data directly in orbit, are attracting growing attention. Nevertheless, some monitoring tasks involve sequential sub-tasks like target observation and movement prediction, leading to data dependencies. Moreover, the limited energy supply on satellites requires the sequential execution of sub-tasks. Therefore, inappropriate assignments can cause circular waiting among satellites, resulting in deadlocks. This paper formulates task offloading in ICSC satellites with data-dependent constraints as a mixed-integer linear programming (MILP) problem, aiming to minimize service latency and energy consumption simultaneously. Given the non-centrality of ICSC satellites, we propose a distributed deadlock-free task offloading (DDFTO) algorithm. DDFTO operates in parallel on each satellite, alternating between sub-task inclusion and consensus and sub-task removal until a common offloading assignment is reached. To avoid deadlocks arising from sub-task inclusion, we introduce the deadlock-free insertion mechanism (DFIM), which strategically restricts the insertion positions of sub-tasks based on interval relationships, ensuring deadlock-free assignments. Extensive experiments demonstrate the effectiveness of DFIM in avoiding deadlocks and show that the DDFTO algorithm outperforms benchmark algorithms in achieving deadlock-free offloading assignments.

Visual experience and sensitivity to motion sequences, from human movement patterns to animated shapes

Visual experience and sensitivity to motion sequences, from human movement patterns to animated shapes

The Hippocampus Represents Information about Movements in Their Temporal Position in a Learned Motor Sequence.

Our repertoire of motor skills is filled with sequential movements that need to be performed in a specific order. Here, we used functional magnetic resonance imaging to investigate whether the human hippocampus, a region known to support temporal order in non-motor memory, represents information about the order of sequential motor actions in human participants (both sexes). We also examined such representations in other regions of the motor network (i.e., the premotor cortex, supplementary motor area, anterior superior parietal lobule, and striatum) already known for their critical role in motor sequence learning. Results showed that the hippocampus represents information about movements in their learned temporal position in the sequence, but not about movements or temporal positions in random movement patterns. Other regions of the motor network coded for movements in their learned temporal position, as well as movements and positions in random movement patterns. Importantly, movement coding contributed to sequence learning patterns in primary, supplementary, and premotor cortices but not in striatal and parietal regions. Our findings deepen our understanding of how striatal and cortical regions contribute to motor sequence learning and point to the capacity of the hippocampus to represent movements in their temporal context, an ability possibly explaining its contribution to motor learning.

Transfer Learning and Deep Neural Networks for Robust Intersubject Hand Movement Detection from EEG Signals

In this research, five systems were developed to classify four distinct motor functions—forward hand movement (FW), grasp (GP), release (RL), and reverse hand movement (RV)—from EEG signals, using the WAY-EEG-GAL dataset where participants performed a sequence of hand movements. During preprocessing, band-pass filtering was applied to remove artifacts and focus on the mu and beta frequency bands. The initial system, a preliminary study model, explored the overall framework of EEG signal processing and classification, utilizing time-domain features such as variance and frequency-domain features such as alpha and beta power, with a KNN model for classification. Insights from this study informed the development of a baseline system, which innovatively combined the common spatial patterns (CSP) method with continuous wavelet transform (CWT) for feature extraction and employed a GoogLeNet classifier with transfer learning. This system classified six unique pairs of events derived from the four motor functions, achieving remarkable accuracy, with the highest being 99.73% for the GP–RV pair and the lowest 80.87% for the FW–GP pair in intersubject classification. Building on this success, three additional systems were developed for four-way classification. The final model, ML-CSP-OVR, demonstrated the highest intersubject classification accuracy of 78.08% using all combined data and 76.39% for leave-one-out intersubject classification. This proposed model, featuring a novel combination of CSP-OVR, CWT, and GoogLeNet, represents a significant advancement in the field, showcasing strong potential as a general system for motor imagery (MI) tasks that is not dependent on the subject. This work highlights the prominence of the research contribution by demonstrating the effectiveness and robustness of the proposed approach in achieving high classification accuracy across different motor functions and subjects.

Localization and recognition of human action in 3D using transformers

Understanding a person’s behavior from their 3D motion sequence is a fundamental problem in computer vision with many applications. An important component of this problem is 3D action localization, which involves recognizing what actions a person is performing, and when the actions occur in the sequence. To promote the progress of the 3D action localization community, we introduce a new, challenging, and more complex benchmark dataset, BABEL-TAL (BT), for 3D action localization. Important baselines and evaluating metrics, as well as human evaluations, are carefully established on this benchmark. We also propose a strong baseline model, i.e., Localizing Actions with Transformers (LocATe), that jointly localizes and recognizes actions in a 3D sequence. The proposed LocATe shows superior performance on BABEL-TAL as well as on the large-scale PKU-MMD dataset, achieving state-of-the-art performance by using only 10% of the labeled training data. Our research could advance the development of more accurate and efficient systems for human behavior analysis, with potential applications in areas such as human-computer interaction and healthcare.

Future movement plans interact in sequential arm movements.

Real-world actions often comprise a series of movements that cannot be entirely planned before initiation. When these actions are executed rapidly, the planning of multiple future movements needs to occur simultaneously with the ongoing action. How the brain solves this task remains unknown. Here, we address this question with a new sequential arm reaching paradigm that manipulates how many future reaches are available for planning while controlling execution of the ongoing reach. We show that participants plan at least two future reaches simultaneously with an ongoing reach. Further, the planning processes of the two future reaches are not independent of one another. Evidence that the planning processes interact is twofold. First, correcting for a visual perturbation of the ongoing reach target is slower when more future reaches are planned. Second, the curvature of the current reach is modified based on the next reach only when their planning processes temporally overlap. These interactions between future planning processes may enable smooth production of sequential actions by linking individual segments of a long sequence at the level of motor planning.

Future movement plans interact in sequential arm movements

Real-world actions often comprise a series of movements that cannot be entirely planned before initiation. When these actions are executed rapidly, the planning of multiple future movements needs to occur simultaneously with the ongoing action. How the brain solves this task remains unknown. Here, we address this question with a new sequential arm reaching paradigm that manipulates how many future reaches are available for planning while controlling execution of the ongoing reach. We show that participants plan at least two future reaches simultaneously with an ongoing reach. Further, the planning processes of the two future reaches are not independent of one another. Evidence that the planning processes interact is twofold. First, correcting for a visual perturbation of the ongoing reach target is slower when more future reaches are planned. Second, the curvature of the current reach is modified based on the next reach only when their planning processes temporally overlap. These interactions between future planning processes may enable smooth production of sequential actions by linking individual segments of a long sequence at the level of motor planning.

Improving privacy-preserving multi-faceted long short-term memory for accurate evaluation of encrypted time-series MRI images in heart disease

In therapeutic diagnostics, early diagnosis and monitoring of heart disease is dependent on fast time-series MRI data processing. Robust encryption techniques are necessary to guarantee patient confidentiality. While deep learning (DL) algorithm have improved medical imaging, privacy and performance are still hard to balance. In this study, a novel approach for analyzing homomorphivally-encrypted (HE) time-series MRI data is introduced: The Multi-Faceted Long Short-Term Memory (MF-LSTM). This method includes privacy protection. The MF-LSTM architecture protects patient’s privacy while accurately categorizing and forecasting cardiac disease, with accuracy (97.5%), precision (96.5%), recall (98.3%), and F1-score (97.4%). While segmentation methods help to improve interpretability by identifying important region in encrypted MRI images, Generalized Histogram Equalization (GHE) improves image quality. Extensive testing on selected dataset if encrypted time-series MRI images proves the method’s stability and efficacy, outperforming previous approaches. The finding shows that the suggested technique can decode medical image to expose visual representation as well as sequential movement while protecting privacy and providing accurate medical image evaluation.

Visualization of the Cdc48 AAA+ ATPase protein unfolding pathway.

The Cdc48 AAA+ ATPase is an abundant and essential enzyme that unfolds substrates in multiple protein quality control pathways. The enzyme includes two conserved AAA+ ATPase motor domains, D1 and D2, that assemble as hexameric rings with D1 stacked above D2. Here, we report an ensemble of native structures of Cdc48 affinity purified from budding yeast lysate in complex with the adaptor Shp1 in the act of unfolding substrate. Our analysis reveals a continuum of structural snapshots that spans the entire translocation cycle. These data uncover elements of Shp1-Cdc48 interactions and support a 'hand-over-hand' mechanism in which the sequential movement of individual subunits is closely coordinated. D1 hydrolyzes ATP and disengages from substrate prior to D2, while D2 rebinds ATP and re-engages with substrate prior to D1, thereby explaining the dominant role played by the D2 motor in substrate translocation/unfolding.

Model-free decision-making underlies motor errors in rapid sequential movements under threat

Our movements, especially sequential ones, are usually goal-directed, i.e., coupled with task-level goals. Consequently, cognitive strategies for decision-making and motor performance are likely to influence each other. However, evidence linking decision-making strategies and motor performance remains elusive. Here, we designed a modified version of the two-step task, named the two-step sequential movement task, where participants had to conduct rapid sequential finger movements to obtain rewards (n = 40). In the shock session, participants received an electrical shock if they made an erroneous or slow movement, while in the no-shock session, they only received zero reward. We found that participants who prioritised model-free decision-making committed more motor errors in the presence of the shock stimulus (shock sessions) than those who prioritised model-based decision-making. Using a mediation analysis, we also revealed a strong link between the balance of the model-based and the model-free learning strategies and sequential movement performances. These results suggested that model-free decision-making produces more motor errors than model-based decision-making in rapid sequential movements under the threat of stressful stimuli.

The impact of Playdough games on the development of oral motor skills in preschool children with childhood apraxia of speech

The relevant challenges are the use of innovative approaches that conserve both the speech therapist’s and the child’s resources while addressing multiple goals in speech therapy. The authors of this article have adopted such an approach to develop oral motor skills, which are crucial for speech development in preschoolers, particularly those with childhood apraxia of speech. This study aimed to evaluate the effectiveness of an author-developed therapy using Playdough games for preschool children with childhood apraxia of speech. A controlled design with repeated measures was employed. The primary diagnosis of CAS was established based on clinical assessment. A comparative evaluation of two groups (control and experimental) was conducted using a developed methodology for assessing the level of manual and oral motor skills in preschool children before and after the author-developed therapy using Playdough games. Researchers discovered that the systematic use of their author-developed Playdough games positively impacted the motor skills of children with CAS. Children in the experimental group, who participated in Playdough therapy, demonstrated significant improvements in both manual and oral motor skills, increased precision and focus in movement execution, and enhanced accuracy and speed of articulatory movements. There was a reduction in delays and incorrect movement sequences during finger movement tasks. Children required less additional time and significantly less assistance from the speech therapist, indicating a positive impact of using Playdough in therapy. The following skills improved: transitions between articulatory positions, which helped children avoid coarticulation difficulties; planning of movement sequences; and smooth transitions between sounds and syllables. The practical significance of this study lies in the effectiveness of using Playdough games for significantly improving oral motor skills, movement precision, and concentration in children with childhood apraxia of speech, thereby reducing the need for additional speech therapy support

IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition

One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality, such as inertial measurement units (IMUs). With the emergence of generative AI models such as large language models (LLMs) and text-driven motion synthesis models, language has become a promising source data modality as well - as shown in proof of concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of language-based cross modality transfer to determine their effectiveness for HAR. Based on this study, we introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios: a motion filter capable of filtering out irrelevant motion sequences to ensure the relevance of the generated virtual IMU data, and a set of metrics that measure the diversity of the generated data facilitating the determination of when to stop generating virtual IMU data for both effective and efficient processing. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%, which opens up IMUGPT for practical use cases beyond a mere proof of concept.

Task and motion planning using mixed integer linear programming for solving fetch-and-carry tasks by a mobile manipulator

This manuscript describes a TAsk and Motion Planning (TAMP) method for mobile manipulators. We focus on fetch-and-carry tasks, and we aim to simultaneously generate both a sequence of actions and a motion sequence for each action. The number of factors to be considered in solving such a problem, and the interactions among them are complex due to the multifaceted characteristics of mobile manipulation. As a result, straightforwardly performing simultaneous TAMP may require a significant amount of processing time. Therefore, we reduce the complexity of the problem by formulating fetch-and-carry planning for a Mixed Integer Linear Programming (MILP) problem. The proposed method can obtain the sequence of actions and the movement of the mobile manipulator in less than one second in many cases. The effectiveness of the proposed method is verified in environments in which delivery objects and obstacles are placed in various patterns, using a robot with an omnidirectional mobile platform and a serial link manipulator.

Is the Articulatory Trajectory of Changing Syllables Important for Achieving Higher Syllable Rates Compared to Repeated Syllables?

The production of speech-like sequences composed of varying syllables has been reported to achieve higher syllable rates than the production of repeated syllables (commonly designed as sequential motion rate [SMR] and alternating motion rate [AMR] tasks, respectively). The faster rate for SMR relative to AMR sequences is explained by different interpretative hypotheses, which remain empirically unexplored. In the present study, we aimed to investigate whether the high syllable rates of SMR sequences are due to the specific sequences used in most studies that involve front-to-back articulatory movements. Syllable rates of SMR sequences composed of different articulatory trajectories (i.e., labial-alveolar-velar [/pateko/], alveolar-velar-labial [/tekopa/], and velar-labial-alveolar [/kopate/]) were compared with those of the AMR sequences /papapa/, /tetete/, and /kokoko/ in 20 participants. The results show higher syllable rates for each of the three SMR sequences as compared to AMR, suggesting that the trajectory of the articulatory movements in the sequential sequences is not the key to achieving higher syllable rates. The syllable rate advantage for SMR over AMR sequences is not explained by the articulatory trajectories included in the former task, indicating that the front-to-back movements generally included in SMR sequences (i.e., /pataka/) are not decisive in achieving a higher syllable rate.

Visual analysis of oceanic data for marine ecosystems

We present a data visualization approach that allows users to interactively analyse sequential movement patterns and automatically extract rules for marine animal behaviour. This enhances the ability of oceanic experts to understand marine ecosystems. Traditional time-series analysis, basic statistical methods, and single-view visualizations often fail to capture the intricate spatial and temporal dynamics of marine ecosystems. Our visual analytics approach, that leverages sequential pattern mining and rule mining techniques, integrates SPMF's Vertical Mining of Maximal Sequential Patterns (VMSP) algorithm and T-Rule Growth algorithm to efficiently extract and visualize movement patterns and behavioural rules of marine animals. The system also incorporates automated pre-processing to clean and prepare large datasets, reducing the time and effort required for manual handling. The visualization presents oceanic data from detection stations, allowing researchers to discover patterns in the movements of marine species. Our interface also provides geographical visualization of selected patterns or rules and helps users to navigate and find the relevant information. Key principles of our model include multi-view visualizations that provide comprehensive insights by displaying data from various perspectives simultaneously, and geo-spatial time-varying data visualization that captures and represents the dynamic movements of marine species. This approach allows domain experts to interactively explore and analyse sequential patterns and rules, enhancing their ability to make informed decisions. Additionally, feedback received, and insights gained from specialized professionals was used to determine if this approach would be a beneficial complement to existing practice.

Wearable high-density EMG sleeve for complex hand gesture classification and continuous joint angle estimation

High-density electromyography (HD-EMG) can provide a natural interface to enhance human–computer interaction (HCI). This study aims to demonstrate the capability of a novel HD-EMG forearm sleeve equipped with up to 150 electrodes to capture high-resolution muscle activity, decode complex hand gestures, and estimate continuous hand position via joint angle predictions. Ten able-bodied participants performed 37 hand movements and grasps while EMG was recorded using the HD-EMG sleeve. Simultaneously, an 18-sensor motion capture glove calculated 23 joint angles from the hand and fingers across all movements for training regression models. For classifying across the 37 gestures, our decoding algorithm was able to differentiate between sequential movements with 97.3±0.3%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$97.3 \\pm 0.3\\%$$\\end{document} accuracy calculated on a 100 ms bin-by-bin basis. In a separate mixed dataset consisting of 19 movements randomly interspersed, decoding performance achieved an average bin-wise accuracy of 92.8±0.8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$92.8 \\pm 0.8\\%$$\\end{document}. When evaluating decoders for use in real-time scenarios, we found that decoders can reliably decode both movements and movement transitions, achieving an average accuracy of 93.3±0.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$93.3 \\pm 0.9\\%$$\\end{document} on the sequential set and 88.5±0.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$88.5 \\pm 0.9\\%$$\\end{document} on the mixed set. Furthermore, we estimated continuous joint angles from the EMG sleeve data, achieving a R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^2$$\\end{document} of 0.884±0.003\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0.884 \\pm 0.003$$\\end{document} in the sequential set and 0.750±0.008\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0.750 \\pm 0.008$$\\end{document} in the mixed set. Median absolute error (MAE) was kept below 10° across all joints, with a grand average MAE of 1.8±0.04∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.8 \\pm 0.04^\\circ$$\\end{document} and 3.4±0.07∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3.4 \\pm 0.07^\\circ$$\\end{document} for the sequential and mixed datasets, respectively. We also assessed two algorithm modifications to address specific challenges for EMG-driven HCI applications. To minimize decoder latency, we used a method that accounts for reaction time by dynamically shifting cue labels in time. To reduce training requirements, we show that pretraining models with historical data provided an increase in decoding performance compared with models that were not pretrained when reducing the in-session training data to only one attempt of each movement. The HD-EMG sleeve, combined with sophisticated machine learning algorithms, can be a powerful tool for hand gesture recognition and joint angle estimation. This technology holds significant promise for applications in HCI, such as prosthetics, assistive technology, rehabilitation, and human–robot collaboration.

Audio2Gestures: Generating Diverse Gestures From Audio.

People may perform diverse gestures affected by various mental and physical factors when speaking the same sentences. This inherent one-to-many relationship makes co-speech gesture generation from audio particularly challenging. Conventional CNNs/RNNs assume one-to-one mapping, and thus tend to predict the average of all possible target motions, easily resulting in plain/boring motions during inference. So we propose to explicitly model the one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code. The shared code is expected to be responsible for the motion component that is more correlated to the audio while the motion-specific code is expected to capture diverse motion information that is more independent of the audio. However, splitting the latent code into two parts poses extra training difficulties. Several crucial training losses/strategies, including relaxed motion loss, bicycle constraint, and diversity loss, are designed to better train the VAE. Experiments on both 3D and 2D motion datasets verify that our method generates more realistic and diverse motions than previous state-of-the-art methods, quantitatively and qualitatively. Besides, our formulation is compatible with discrete cosine transformation (DCT) modeling and other popular backbones (i.e., RNN, Transformer). As for motion losses and quantitative motion evaluation, we find structured losses/metrics (e.g. STFT) that consider temporal and/or spatial context complement the most commonly used point-wise losses (e.g. PCK), resulting in better motion dynamics and more nuanced motion details. Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline.

Application of a Mixed Live and Online Live Streaming Exercise Program of Tai Chi and Its Effect on Dynamic Balance, Physical Function of Lower Limbs and Aerobic Capacity of Adults

This study aimed to investigate the effect of a mixed intervention program, with live and distance online Tai Chi (TC) practice on healthy adult practitioners and its effect on dynamic balance, physical function of lower limbs and aerobic capacity, which were measured at baseline and follow up. The sample of the study consisted of the experimental group (N = 19) of healthy subjects divided into two groups according to their level of experience in Tai Chi (beginner vs. experienced) and 15 healthy subjects (N = 15) of the control group who were sedentary individuals. The experimental group participated in a supervised Tai Chi program lasting 18 weeks, of which 9 weeks of live training sessions (before the pandemic) and 9 further weeks in online distance practice during the pandemic restrictive measures. The exercise frequency was 3 times/week, lasting 90 minutes during live lessons, while the online exercise duration was adjusted to 60 minutes. The Y-Test (Lower Quarter test) was used to measure the dynamic balance. The single limb single hop test measured physical function, while the 6-minute walking test was applied to measure aerobic capacity. The results of the present study showed that a statistically significant difference was observed in the dynamic balance of the left leg between the pre-and post-training measures for the experimental groups, with the experienced group scoring higher. Right leg results revealed improvements for TC practitioners, with the beginners group scoring significantly higher than experienced and control subjects. The asymmetry score changed positively for both experimental groups, with a higher index for the beginners group. Aerobic capacity scores were significantly improved between baseline and final measurements. The TC practitioners scored significantly higher than the control subjects. Furthermore, concerning the results in the physical function of the lower limbs, the TC-experienced group showed greater improvements compared to both TC beginners and the control group. Similar effects concerned right leg results. TC effectively improves balance and physical function of lower limbs in healthy adults due to the activation of different muscle groups, the sequence of movements, the duration, and the nature of the body mind. It should be incorporated in health prevention programs. Due to the pandemic, the implementation of distance learning became a solution for many exercise programs and complemented their content.

Multiparametric Assays Capture Sex- and Environment-Dependent Modifiers of Behavioral Phenotypes in Autism Mouse Models

BackgroundCurrent phenotyping approaches for murine autism models often focus on one selected behavioral feature, making the translation onto a spectrum of autistic characteristics in humans challenging. Furthermore, sex and environmental factors are rarely considered. Here, we aimed to capture the full spectrum of behavioral manifestations in 3 autism mouse models to develop a “behavioral fingerprint” that takes environmental and sex influences under consideration. MethodsTo this end, we employed a wide range of classical standardized behavioral tests and 2 multiparametric behavioral assays—the Live Mouse Tracker and Motion Sequencing—on male and female Shank2, Tsc1, and Purkinje cell–specific Tsc1 mutant mice raised in standard or enriched environments. Our aim was to integrate our high dimensional data into one single platform to classify differences in all experimental groups along dimensions with maximum discriminative power. ResultsMultiparametric behavioral assays enabled a more accurate classification of experimental groups than classical tests, and dimensionality reduction analysis demonstrated significant additional gains in classification accuracy, highlighting the presence of sex, environmental, and genotype differences in our experimental groups. ConclusionsTogether, our results provide a complete phenotypic description of all tested groups, suggesting that multiparametric assays can capture the entire spectrum of the heterogeneous phenotype in autism mouse models.

Interactive Character Control with Auto-Regressive Motion Diffusion Models

Real-time character control is an essential component for interactive experiences, with a broad range of applications, including physics simulations, video games, and virtual reality. The success of diffusion models for image synthesis has led to the use of these models for motion synthesis. However, the majority of these motion diffusion models are primarily designed for offline applications, where space-time models are used to synthesize an entire sequence of frames simultaneously with a pre-specified length. To enable real-time motion synthesis with diffusion model that allows time-varying controls, we propose A-MDM (Auto-regressive Motion Diffusion Model). Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on the previous frame. Despite its streamlined network architecture, which uses simple MLPs, our framework is capable of generating diverse, long-horizon, and high-fidelity motion sequences. Furthermore, we introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning (See Figure 1). These techniques enable a pre-trained A-MDM to be efficiently adapted for a variety of new downstream tasks. We conduct a comprehensive suite of experiments to demonstrate the effectiveness of A-MDM, and compare its performance against state-of-the-art auto-regressive methods.