Sequence Of Actions Research Articles

The hippocampus pre-orders movements for skilled action sequences.

Plasticity in the subcortical motor basal ganglia-thalamo-cerebellar network plays a key role in the acquisition and control of long-term memory for new procedural skills, from the formation of population trajectories controlling trained motor skills in the striatum to the adaptation of sensorimotor maps in the cerebellum. However, recent findings demonstrate the involvement of a wider cortical and subcortical brain network in the consolidation and control of well-trained actions, including of a brain region traditionally associated with declarative memory - the hippocampus. Here, we probe which role these subcortical areas play in skilled motor sequence control, from sequence feature selection during planning to their integration during sequence execution. An fMRI dataset (N=24, 14 female) collected after participants learnt to produce four finger press sequences entirely from memory with high movement and timing accuracy over several days was examined for both changes in BOLD activity and their informational content in subcortical regions of interest. Although there was a widespread activity increase in effector-related striatal, thalamic and cerebellar regions, in particular during sequence execution, the associated activity did not contain information on the motor sequence identity. In contrast, hippocampal activity increased during planning and predicted the order of the upcoming sequence of movements. Our findings suggests that the hippocampus pre-orders movements for skilled action sequences, thus contributing to the higher-order control of skilled movements that require flexible retrieval. These findings challenge the traditional taxonomy of episodic and procedural memory and carry implications for the rehabilitation of individuals with neurodegenerative disorders.Significance statement Skilled actions like handwriting, typing, and playing music, involve recombining movements into various sequences and producing them from memory. This fMRI study suggests that the hippocampus, traditionally associated with episodic memory and spatial navigation, pre-orders well-trained movement sequences prior to their execution. These findings challenge the canonical separation between episodic and procedural memory systems and the neural basis of skilled motor behaviours.

MAFormer: A cross-channel spatio-temporal feature aggregation method for human action recognition

Human action recognition has been widely used in fields such as human–computer interaction and virtual reality. Despite significant progress, existing approaches still struggle with effectively integrating hierarchical information and processing data beyond a certain frame count. To address these challenges, we introduce the Multi-AxisFormer (MAFormer) model, which is organized in terms of spatial, temporal, and channel dimensions of the action sequence, thereby enhancing the model’s understanding of correlations and intricate structures among and within features. Drawing on the Transformer architecture, we propose the Cross-channel Spatio-temporal Aggregation (CSA) structure for more refined feature extraction and the Multi-Axis Attention (MAA) module for more comprehensive feature aggregation. Moreover, the integration of Rotary Position Embedding (RoPE) boosts the model’s extrapolation and generalization abilities. MAFormer surpasses the known state-of-the-art on multiple skeleton-based action recognition benchmarks with the accuracy of 93.2% on NTU RGB+D 60 cross-subject split, 89.9% on NTU RGB+D 120 cross-subject split, and 97.2% on N-UCLA, offering a novel paradigm for hierarchical modeling in human action recognition.

Sequential Reservoir Computing for Log File‐Based Behavior Process Data Analyses

AbstractThe use of process data in assessment has gained attention in recent years as more assessments are administered by computers. Process data, recorded in computer log files, capture the sequence of examinees' response activities, for example, timestamped keystrokes, during the assessment. Traditional measurement methods are often inadequate for handling this type of data. In this paper, we proposed a sequential reservoir method (SRM) based on a reservoir computing model using the echo state network, with the particle swarm optimization and singular value decomposition as optimization. Designed to regularize features from process data through a computational self‐learning algorithm, this method has been evaluated using both simulated and empirical data. Simulation results suggested that, on one hand, the model effectively transforms action sequences into standardized and meaningful features, and on the other hand, these features are instrumental in categorizing latent behavioral groups and predicting latent information. Empirical results further indicate that SRM can predict assessment efficiency. The features extracted by SRM have been verified as related to action sequence lengths through the correlation analysis. This proposed method enhances the extraction and accessibility of meaningful information from process data, presenting an alternative to existing process data technologies.

LG-AKD: Application of a lightweight GCN model based on adversarial knowledge distillation to skeleton action recognition

Human action recognition, a pivotal topic in computer vision, is a highly complex and challenging task. It requires the analysis of not only spatial dependencies of targets but also temporal changes in these targets. In recent decades, the advancement of deep learning has led to the development of numerous action recognition methods based on deep neural networks. Given that the skeleton points of the human body can be treated as a graph structure, graph neural networks (GNNs) have emerged as an effective tool for modeling such data, garnering significant interest from researchers. This paper aims to address the issue of low test speed caused by over-complicated deep graph convolutional models. To achieve this, we compress the network structure using knowledge distillation from a teacher-student architecture, leading to a compact and lightweight student GNN. To enhance the model’s robustness and generalization capabilities, we introduce a data augmentation mechanism that generates diverse action sequences while maintaining consistent behavior labels, thereby providing a more comprehensive learning basis for the model. The proposed model integrates three distinct knowledge learning paths: teacher networks, original datasets, and derived data. The fusion of knowledge distillation and data augmentation enables lightweight student networks to outperform their teacher networks in terms of both performance and efficiency. Experimental results demonstrate the efficacy of our approach in the context of skeleton-based human action recognition, highlighting its potential to simplify state-of-the-art models while enhancing their performance.

Interactive probes: Towards action-level evaluation for dialogue systems

Measures of ‘humanness’, ‘coherence’ or ‘fluency’ are the mainstay of dialogue system evaluation, but they don’t target system capabilities and rarely offer actionable feedback. Reviewing recent work in this domain, we identify an opportunity for evaluation at the level of action sequences, rather than the more commonly targeted levels of whole conversations or single responses. We introduce interactive probes, an evaluation framework inspired by empirical work on social interaction that can help to systematically probe the capabilities of dialogue systems. We sketch some first probes in the domains of tellings and repair, two sequence types ubiquitous in human interaction and challenging for dialogue systems. We argue interactive probing can offer the requisite flexibility to keep up with developments in interactive language technologies and do justice to the open-endedness of action formation and ascription in interaction.

Hand-Jaw Coordination as Mice Handle Food Is Organized around Intrinsic Structure-Function Relationships.

Rodent jaws evolved structurally to support dual functionality, for either biting or chewing food. Rodent hands also function dually during food handling, for actively manipulating or statically holding food. How are these oral and manual functions coordinated? We combined electrophysiological recording of muscle activity and kilohertz kinematic tracking to analyze masseter and hand actions as mice of both sexes handled food. Masseter activity was organized into two modes synchronized to hand movement modes. In holding/chewing mode, mastication occurred as rhythmic (∼5 Hz) masseter activity while the hands held food below the mouth. In oromanual/ingestion mode, bites occurred as lower-amplitude aperiodic masseter events that were precisely timed to follow regrips (by ∼200 ms). Thus, jaw and hand movements are flexibly coordinated during food handling: uncoupled in holding/chewing mode and tightly coordinated in oromanual/ingestion mode as regrip-bite sequences. Key features of this coordination were captured in a simple model of hierarchically orchestrated mode-switching and intramode action sequencing. We serendipitously detected an additional masseter-related action, tooth sharpening, identified as bouts of higher-frequency (∼13 Hz) rhythmic masseter activity, which was accompanied by eye displacement, including rhythmic proptosis, attributable to masseter contractions. Collectively, the findings demonstrate how a natural, complex, and goal-oriented activity is organized as an assemblage of distinct modes and complex actions, adapted for the divisions of function arising from anatomical structure. These results reveal intricate, high-speed coordination of disparate effectors and show how natural forms of dexterity can serve as a model for understanding the behavioral neurobiology of multi-body-part coordination.

ORGANIZATIONAL AND ECONOMIC MECHANISM FOR DAIRY INDUSTRY ORGANIZATION COMPETITIVENESS ENSURING

The article is devoted to the current problem of increasing the effectiveness of the competitiveness strategy of a dairy industry organization through the formation of a model of an organizational and economic mechanism that allows choosing strategic vectors of development, taking into account the integration impact of various factors. The conducted theoretical analysis of the categories “organizational-economic mechanism” and competitiveness made it possible to define an approach to understanding the content of the interaction of the studied categories as a logical sequence of actions of the organization to achieve the goal, including marketing research of the internal and external environment, identifying development problems, assessing the ways and possibilities of their solution, taking into account the maximum number of influencing factors at various levels. An analysis of statistical data related to the development of the dairy products market at the level of trends in the Russian Federation and the Republic of Tatarstan was carried out. A conclusion is made about the possibility of increasing the volume of dairy production, and the need to build effective inter-industry connections through the formation of an organizational and economic mechanism for increasing the competitiveness of organizations in the dairy industry of the Republic of Tatarstan. Using the example of Kasymovsky Dairy Plant LLC, marketing tools for analyzing competitiveness were tested and calculations were made, some of which are presented in the article (calculation of the need for dairy products in 2026, analysis of nomenclature, sales dynamics and assessment of consumer awareness of dairy consumers). A methodological approach to building an organizational and economic mechanism for achieving a certain level of competitiveness of a dairy industry organization is proposed. The theoretical three-level model of the organizational and economic mechanism for increasing the competitiveness of dairy industry organizations determines the vectors of regulation and criteria for the effectiveness of selected strategies. Within the framework of the methodological approach, a generalized algorithm for the development of the organization and a scheme for forming the process of selling dairy products are proposed.

Hippocampal neuronal activity is aligned with action plans.

Neurons in the hippocampus are correlated with different variables, including space, time, sensory cues, rewards, and actions, where the extent of tuning depends on ongoing task demands. However, it remains uncertain whether such diverse tuning corresponds to distinct functions within the hippocampal network or if a more generic computation can account for these observations. To disentangle the contribution of externally driven cues versus internal computation, we developed a task in mice where space, auditory tones, rewards, and context were juxtaposed with changing relevance. High-density electrophysiological recordings revealed that neurons were tuned to each of these modalities. By comparing movement paths and action sequences, we observed that external variables had limited direct influence on hippocampal firing. Instead, spiking was influenced by online action plans modulated by goal uncertainty. Our results suggest that internally generated cell assembly sequences are selected and updated by action plans toward deliberate goals. The apparent tuning of hippocampal neuronal spiking to different sensory modalities might emerge due to alignment to the afforded action progression within a task rather than representation of external cues.

Contextual visual and motion salient fusion framework for action recognition in dark environments

Infrared (IR) human action recognition (AR) exhibits resilience against shifting illumination conditions, changes in appearance, and shadows. It has valuable applications in numerous areas of future sustainable and smart cities including robotics, intelligent systems, security, and transportation. However, current IR-based recognition approaches predominantly concentrate on spatial or local temporal information and often overlook the potential value of global temporal patterns. This oversight can lead to incomplete representations of body part movements and prevent accurate optimization of a network. Therefore, a contextual-motion coalescence network (CMCNet) is proposed that operates in a streamlined and end-to-end manner for robust action representation in darkness in a near-infrared (NIR) setting. Initially, data are preprocessed to separate foreground, normalized, and resized. The framework employs two parallel modules: the contextual visual features learning module (CVFLM) for local feature extraction, and the temporal optical flow learning module (TOFLM) for acquiring motion dynamics. These modules focus on action-relevant regions used shift window-based operations to ensure accurate interpretation of motion information. The coalescence block harmoniously integrates the contextual and motion features within a unified framework. Finally, the temporal decoder module discriminatively identifies the boundaries of the action sequence. This sequence of steps ensures the synergistic optimization of both CVFLM and TOFLM and thorough competent feature extraction for precise AR. Evaluations of CMCNet are carried out on publicly available datasets, InfAR and NTURGB-D, where superior performance is achieved. Our model yields the highest average precision of 89% and 85% on these datasets, respectively, representing an improvement of 2.25% (on InfAR) compared to conventional methods operating at spatial and optical flow levels which underscores its efficacy.

Tactics of surgical treatment of recurrent glomus tumors of the skull base: A clinical case

AimThe purpose of the study is to build effective tactics of surgical treatment of pathology, and to increase the effectiveness of surgical treatment of patients with this diagnosis by optimising preoperative preparation, and the correct sequence of actions that will help prevent relapse. MethodsThe leading approach to the study is the description of a clinical case, which will help to carefully consider this issue from all angles, analyse the methodology of patient examination, and build highly effective tactics of surgical intervention to eliminate tumour-like masses, minimising the risks of recurrence. The second clinical case was also considered, and a comparative analysis was carried out on certain parameters. ResultsThe paper presents a clinical case with a practical aspect of surgical treatment of the pathology. The issue of detailed symptoms and manifestations of the disease is disclosed, and the course of surgical interventions is described in stages, in which no nerve is damaged, no complications are caused, and the desired result is achieved. The peculiarity of this case is the recurrence of the disease, and the ineffectiveness of previous treatment methods, including embolisation of the neoplasm and antromastoidotomy. ConclusionsThe results obtained in the course of this study and the formulated conclusions are of great importance for surgeons who are faced with patients with the stated diagnosis.

Optimizing ZX-diagrams with deep reinforcement learning

Abstract ZX-diagrams are a powerful graphical language for the description of quantum processes with applications in fundamental quantum mechanics, quantum circuit optimization, tensor network simulation, and many more. The utility of ZX-diagrams relies on a set of local transformation rules that can be applied to them without changing the underlying quantum process they describe. These rules can be exploited to optimize the structure of ZX-diagrams for a range of applications. However, finding an optimal sequence of transformation rules is generally an open problem. In this work, we bring together ZX-diagrams with reinforcement learning, a machine learning technique designed to discover an optimal sequence of actions in a decision-making problem and show that a trained reinforcement learning agent can significantly outperform other optimization techniques like a greedy strategy, simulated annealing, and state-of-the-art hand-crafted algorithms. The use of graph neural networks to encode the policy of the agent enables generalization to diagrams much bigger than seen during the training phase.

Towards equitable infrastructure asset management: Scour maintenance strategy for aging bridge systems in flood-prone zones using deep reinforcement learning

Bridges play a critical role in transportation networks; however, they are vulnerable to deterioration, aging, and degradation, especially in the face of climate change and extreme weather events such as floodings. Furthermore, bridges can significantly affect social vulnerability; their damage or destruction can isolate communities, inhibit emergency responses, and disrupt essential services. Maintaining critical bridges in a cost-effective and sustainable manner is crucial to ensure their longevity and protect vulnerable communities. To address the maintenance optimization problem of bridge systems considering the effects of time deterioration, flood degradation, and social vulnerability, this study proposes a deep reinforcement learning algorithm to optimally allocate resources to bridges that are at expected cost of failure due to scour. The algorithm considers the effects of flood degradation with different return periods and is trained using a Markov Decision Process as the environment. The study conducts four flood simulation scenarios using Geographic Information System data. The findings suggest that the deep reinforcement learning algorithm proposes a sequence of repair actions that outperforms the status quo, currently employed by bridge managers. The significance of this study lies in its valuable insights for cities worldwide on how to effectively optimize their limited resources for the maintenance and rehabilitation of critical infrastructure systems to decrease portfolio cost and increase social equity.

Negotiating difficult (in)tangible heritage: the intricate journey of museum making for the “Dai-ichi Saloon” comfort station in Shanghai

ABSTRACT The heritage of comfort women has always remained on the periphery of authoritative heritage, classified as a “difficult heritage,” facing a predicament of social marginalization, government concealment, and international controversy. This article addresses the negotiations, discussions, and compromises involved in the process of safeguarding the Dai-ichi Saloon, which is on the verge of becoming a museum. It is a priority for the most difficult heritage to be musealized, which is one of the most common methods of “normalizing” to ensure preservation and dissemination. The museumization of Dai-ichi Saloon has involved four stages, through which, scholars, residents, Shanghai citizens, and the government, as the four major stakeholders, have developed different understandings of the Dai-ichi Saloon as a difficult heritage. This article argues that the key reason for this misalignment lies in the misplaced sequence of actions among agencies during the “normalization” process. Through the intricate process of the museum making for Dai-ichi Saloon, this article aims to offer lessons for the museum’s approach to the normalization of difficult heritage.

From Recognition to Prediction: Leveraging Sequence Reasoning for Action Anticipation

The action anticipation task refers to predicting what action will happen based on observed videos, which requires the model to have a strong ability to summarize the present and then reason about the future. Experience and common sense suggest that there is a significant correlation between different actions, which provides valuable prior knowledge for the action anticipation task. However, previous methods have not effectively modeled this underlying statistical relationship. To address this issue, we propose a novel end-to-end video modeling architecture that utilizes attention mechanisms, named Anticipation via Recognition and Reasoning (ARR). ARR decomposes the action anticipation task into action recognition and sequence reasoning tasks, and effectively learns the statistical relationship between actions by next action prediction (NAP). In comparison to existing temporal aggregation strategies, ARR is able to extract more effective features from observable videos to make more reasonable predictions. In addition, to address the challenge of relationship modeling that requires extensive training data, we propose an innovative approach for the unsupervised pre-training of the decoder, which leverages the inherent temporal dynamics of video to enhance the reasoning capabilities of the network. Extensive experiments on the Epic-kitchen-100, EGTEA Gaze+, and 50salads datasets demonstrate the efficacy of the proposed methods. The code is available at https://github.com/linuxsino/ARR .

The origin of great ape gestural forms.

Two views claim to account for the origins of great ape gestural forms. On the Leipzig view, gestural forms are ontogenetically ritualised from action sequences between pairs of individuals. On the St Andrews view, gestures are the product of natural selection for shared gestural forms. The Leipzig view predicts within- and between-group differences between gestural forms that arise as a product of learning in ontogeny. The St Andrews view predicts universal gestural forms comprehensible within and between species that arise because gestural forms were a target of natural selection. We reject both accounts and propose an alternative "recruitment view" of the origins of great ape gestures. According to the recruitment view, great ape gestures recruit features of their existing behavioural repertoire for communicative purposes. Their gestures inherit their communicative functions from visual (and sometimes tactile) presentations of familiar and easily recognisable action schemas and states and parts of the body. To the extent that great ape species possess similar bodies, this predicts mutual comprehensibility within and between species - but without supposing that gestural forms were themselves targets of natural selection. Additionally, we locate great ape gestural communication within a pragmatic framework that is continuous with human communication, and make testable predications for adjudicating between the three alternative views. We propose that the recruitment view best explains existing data, and does so within a mechanistic framework that emphasises continuity between human and non-human great ape communication.

Decision-making in action: How international-level professional football players gain an advantage

Understanding how both visual and contextual in-game information influences player’s attempts to gain an advantage over their opponent is key to understanding skilled decision-making in fast-ball sports. In the present study, we conducted semi-structured interviews with 15 male professional football players to explore their reported behaviours and perspectives on their in-game decision-making and the ways in which they adapt to gain an advantage over their opponent. Professional players who have competed internationally at either Under 17, Under 19, Under 21, or senior level took part in semi-structured interviews. The data were analysed using reflexive thematic analysis which generated four higher-order themes about players reported behaviours and perspectives on their decision-making and ways of gaining an advantage: (i) being “unpredictable”; (ii) option generation and invitation, comprising two sub-themes: (iia) generating and realising options in action; and (iib) act on what invites you in the moment; (iii) anticipation and awareness; and (iv) dictating and controlling the game. A key finding was that players attempts to gain an advantage were largely grounded in the aim of ‘being unpredictable’ through the use of deception and disguise, as well as deliberate manipulation of action sequences in order to make their actions ‘hard to read’. A further important finding was that players do not always seek a universal optimal decision, as their individual action capabilities influence their available options. These findings have important implications for the assessment of decision-making performance in future empirical research.

Task and motion planning using mixed integer linear programming for solving fetch-and-carry tasks by a mobile manipulator

This manuscript describes a TAsk and Motion Planning (TAMP) method for mobile manipulators. We focus on fetch-and-carry tasks, and we aim to simultaneously generate both a sequence of actions and a motion sequence for each action. The number of factors to be considered in solving such a problem, and the interactions among them are complex due to the multifaceted characteristics of mobile manipulation. As a result, straightforwardly performing simultaneous TAMP may require a significant amount of processing time. Therefore, we reduce the complexity of the problem by formulating fetch-and-carry planning for a Mixed Integer Linear Programming (MILP) problem. The proposed method can obtain the sequence of actions and the movement of the mobile manipulator in less than one second in many cases. The effectiveness of the proposed method is verified in environments in which delivery objects and obstacles are placed in various patterns, using a robot with an omnidirectional mobile platform and a serial link manipulator.

Distributionally robust sequential load restoration of distribution system considering random contingencies

AbstractNatural disasters would destroy power grids and lead to blackouts. To enhance resilience of distribution systems, the sequential load restoration strategy can be adopted to restore outage portions using a sequence of control actions, such as switch on/off, load pickup, distributed energy resource dispatch etc. However, the traditional strategy may be unable to restore the distribution system in extreme weather events due to random sequential contingencies during the restoration process. To address this issue, this paper proposes a distributionally robust sequential load restoration strategy to determine restoration actions. Firstly, a novel multi‐time period and multi‐zone contingency occurrence uncertainty set is constructed to model spatial and temporal nature of sequential line contingencies caused by natural disasters. Then, a distributionally robust load restoration model considering uncertain line contingency probability distribution is formulated to maximize the expected restored load amount with respect to the worst‐case line contingency probability distribution. Case studies were carried out on the modified IEEE 123‐node system. Simulation results show that the proposed distributionally robust sequential load restoration strategy can produce a more resilient load restoration strategy against random sequential contingencies. Moreover, as compared with the conventional robust restoration strategy, the proposed strategy yields a less conservative restoration solution.

DRL-based progressive recovery for quantum-key-distribution networks

With progressive network recovery, operators restore network connectivity after massive failures along multiple stages, by identifying the optimal sequence of repair actions to maximize carried live traffic. Motivated by the initial deployments of quantum-key-distribution (QKD) over optical networks appearing in several locations worldwide, in this work we model and solve the progressive QKD network recovery (PQNR) problem in QKD networks to accelerate the recovery after failures. We formulate an integer linear programming (ILP) model to optimize the achievable accumulative key rates during recovery for four different QKD network architectures, considering different capabilities of using trusted relay and optical bypass. Due to the computational limitations of the ILP model, we propose a deep reinforcement learning (DRL) algorithm based on a twin delayed deep deterministic policy gradients (TD3) framework to solve the PQNR problem for large-scale topologies. Simulation results show that our proposed algorithm approaches well compared to the optimal solution and outperforms several baseline algorithms. Moreover, using optical bypass jointly with trusted relay can improve the performance in terms of the key rate by 14% and 18% compared to the cases where only optical bypass and only trusted relay are applied, respectively.

Optimization of truss structures using multi-objective cheetah optimizer

In this study, a multi-objective version of the recently proposed cheetah optimizer called multi-objective cheetah optimizer (MOCO) has been proposed. MOCO draws inspiration from the targeted hunting strategy employed by cheetahs, which involves a sequence of actions: searching for prey, patiently waiting for the right moment to attack, swiftly launching the attack, and then retreating from the prey and returning to their habitat. MOCO is the result of modification and enhancement from its single-objective counterpart, utilizing a Pareto dominance-based approach. This adaptation allows MOCO to efficiently handle multiple objectives, explores and exploits promising areas in the optimization landscape, and identifies non-dominated solutions, offering valuable tradeoff choices for decision-makers. To demonstrate its practical applications, the MOCO method has been employed to address five intricate structural design problems. These problems involve a pair of competing objectives: the minimization of structural weight and the reduction of maximum nodal displacement. To gauge the efficacy and efficiency of the proposed algorithm, a comparative analysis is conducted against three alternative state-of-the-art multi-objective algorithms. Furthermore, a rigorous evaluation is carried out utilizing hypervolume testing. The findings reveal that the MOCO algorithm surpasses the performance of the other algorithms, underscored by its capacity to uncover a diverse array of non-dominated solutions. To delve deeper into the experimental results, statistical analysis employing Friedman’s rank test is employed. The solutions generated and the convergence patterns exhibited by the MOCO approach underscore its exceptional proficiency in resolving intricate design problems.