Action Representation Research Articles

OTM-HC: Enhanced Skeleton-Based Action Representation via One-to-Many Hierarchical Contrastive Learning

Human action recognition has become crucial in computer vision, with growing applications in surveillance, human–computer interaction, and healthcare. Traditional approaches often use broad feature representations, which may miss subtle variations in timing and movement within action sequences. Our proposed One-to-Many Hierarchical Contrastive Learning (OTM-HC) framework maps the input into multi-layered feature vectors, creating a hierarchical contrast representation that captures various granularities within a human skeleton sequence temporal and spatial domains. Using sequence-to-sequence (Seq2Seq) transformer encoders and downsampling modules, OTM-HC can distinguish between multiple levels of action representations, such as instance, domain, clip, and part levels. Each level contributes significantly to a comprehensive understanding of action representations. The OTM-HC model design is adaptable, ensuring smooth integration with advanced Seq2Seq encoders. We tested the OTM-HC framework across four datasets, demonstrating improved performance over state-of-the-art models. Specifically, OTM-HC achieved improvements of 0.9% and 0.6% on NTU60, 0.4% and 0.7% on NTU120, and 0.7% and 0.3% on PKU-MMD I and II, respectively, surpassing previous leading approaches across these datasets. These results showcase the robustness and adaptability of our model for various skeleton-based action recognition tasks.

EXPRESS: Disgust drivers do not impact on the altered body in action representation in anorexia nervosa.

Disgust is a powerful emotion, that evolved to protect us from contamination and diseases; it also cores to very human feelings, such as shame. In anorexia nervosa, most of the knowledge on disgust regards food. However, disgust can be elicited by varied drivers, including body-related self-disgust, which may be more central to this condition. Here, we investigate in depth how disgust triggers related to the body influence altered representations in anorexia nervosa.Women with anorexia nervosa and healthy women performed the Hand Laterality Task, in which they were asked to judge the laterality of hands without and with a disgust charging feature (i.e. with a body product or with a body violation). We computed accuracy and reaction time for the effect of biomechanical constraints, an index of motor imagery. We also measured the general disgust sensitivity through a self-report questionnaire.Participants with anorexia nervosa were overall less accurate and slower compared to controls, suggesting a non-canonical (i.e. not based on motor imagery) approach to solving the task. However, they showed the same pattern of responses as controls for disgust-charged stimuli, despite reporting higher levels of disgust sensitivity.Our results suggested the absence of specific effects of disgust drivers on the (altered) body in action representation in anorexia nervosa. We discuss this evidence focusing on the role of the psychopathological symptoms characterizing anorexia nervosa. We also reflect on the efficacy of experimental methodologies used to detect alterations in body representation in this clinical condition.

A Business Management Model in the Circular Economy: Technological Solutions for Plastic Recycling

Objectives: this article develops a model for managing business processes in a circular economy using technologies for plastic recycling. Theoretical Framework: The theoretical idea and basis of the article consists in the development of project management methodology, process approach in the closed-loop economy, development of methods and economic tools aimed at effective resource management taking into account waste reduction and increased reuse of plastic fractions in Municipal Solid Waste (MSW). The application of digital technologies and solutions in the context of business process management in the field of plastics recycling will be substantiated, which will allow to identify new opportunities and reserves for optimising business processes in this field, increasing the efficiency of resource use, improving the environment and reducing costs. Method: the work uses a project approach, a process approach, digital modeling methods, and the “polluter pays” method. To implement the process approach, it is proposed to use the concept of Business Process Management (BPM). Competent business process management allows the company to react quickly to changing market conditions and customer orientation. Digital modelling in this paper means the representation of actions in the form of business processes in the organisation of solid waste management, the 'polluter pays' method means the economic assessment and accounting of environmental damage in waste management in the costs of business processes. Results and Discussion: The results of the study can be used to introduce the process approach to plastics recycling in individual regions and countries for the development of a green economy. According to the estimates of the authors, the optimisation and cost reduction of business processes, taking into account the environmental factor and the increase of the revenue part of the project from the reuse and inclusion in the economic turnover of plastic waste recycling products on the scale of the considered city (population of 1800 thousand people, area of 2856 ha, 18.14 tonnes of plastic waste accumulated per day) will reduce the costs of business process management by more than 200 thousand euros. Research Implications: the results of the study can be used to justify projects for the processing and recycling of solid municipal waste, including plastic waste, to introduce environmentally friendly production, to reduce the negative impact on the environment, to develop a process approach to the organisation of all activities for the collection of plastic waste in the city, district. In addition, the application of the proposed approach allows to obtain economic benefits for companies through the sale of secondary resources for the processing of plastic waste, as well as to reduce the negative impact on the environment. Originality/ Value: The originality of the research and its value lies in a new scientific formulation of the problem of reducing environmental pollution from plastic waste through the use of business processes in the field of plastic recycling and digital solutions, as well as the possibility of practical application of the results obtained in the field of green economy.

A new bi-level deep human action representation structure based on the sequence of sub-actions

A new bi-level deep human action representation structure based on the sequence of sub-actions

Region-aware image-based human action retrieval with transformers

Human action understanding is a fundamental and challenging task in computer vision. Although there exists tremendous research on this area, most works focus on action recognition, while action retrieval has received less attention. In this paper, we focus on the neglected but important task of image-based action retrieval which aims to find images that depict the same action as a query image. We establish benchmarks for this task and set up important baseline methods for fair comparison. We present a Transformer-based model that learns rich action representations from three aspects: the anchored person, contextual regions, and the global image. A fusion transformer is designed to model the relationships among different features and effectively fuse them into an action representation. Experiments on both the Stanford-40 and PASCAL VOC 2012 Action datasets show that the proposed method significantly outperforms previous approaches for image-based action retrieval.

Background debiased class incremental learning for video action recognition

In this work, we tackle class incremental learning (CIL) for video action recognition, a relatively under-explored problem despite its practical importance. Directly applying image-based CIL methods does not work well in the video action recognition setting. We hypothesize the major reason is the spurious correlation between the action and background in video action recognition datasets/models. Recent literature shows that the spurious correlation hampers the generalization of models in the conventional action recognition setting. The problem is even more severe in the CIL setting due to the limited exemplars available in the rehearsal memory. We empirically show that mitigating the spurious correlation between the action and background is crucial to the CIL for video action recognition. We propose to learn background invariant action representations in the CIL setting by providing training videos with diverse backgrounds generated from background augmentation techniques. We validate the proposed method on public benchmarks: HMDB-51, UCF-101, and Something-Something-v2.

Eyes and hand are both reliable at localizing somatosensory targets.

Body representations (BR) for action are critical to perform accurate movements. Yet, behavioral measures suggest that BR are distorted even in healthy people. However, the upper limb has mostly been used as a probe so far, making difficult to decide whether BR are truly distorted or whether this depends on the effector used as a readout. Here, we aimed to assess in healthy humans the accuracy of the eye and hand effectors in localizing somatosensory targets, to determine whether they may probe BR similarly. Twenty-six participants completed two localization tasks in which they had to localize an unseen target (proprioceptive or tactile) with either their eyes or hand. Linear mixed model revealed in both tasks a larger horizontal (but not vertical) localization error for the ocular than for the manual localization performance. However, despite better hand mean accuracy, manual and ocular localization performance positively correlated to each other in both tasks. Moreover, target position also affected localization performance for both eye and hand responses: accuracy was higher for the more flexed position of the elbow in the proprioceptive task and for the thumb than for the index finger in the tactile task, thus confirming previous results of better performance for the thumb. These findings indicate that the hand seems to beat the eyes along the horizontal axis when localizing somatosensory targets, but the localization patterns revealed by the two effectors seemed to be related and characterized by the same target effect, opening the way to assess BR with the eyes when upper limb motor control is disturbed.

Figurative Narratives in Political Blogging: Linguocognitive Perspective

Introduction. Recent years have seen a significant shift towards the pervasive use of figurative narratives in mediated political communication. Narrative practices, previously analyzed exclusively within the framework of literary theory, have become a kind of «language games» of modern politicians in their communication with potential voters and opponents, debates and intricate negotiation processes. The article analyzes figurative narratives of the English- and German-speaking political blogs as a powerful linguistic-cognitive tool for mediated political communication. The purpose of the paper is deep semantic analysis of figurative narratives of political blogs, which, as the authors claim, is a promising direction of research due to the narrative nature of human reasoning. The paper argues that the skillful use of figurative narrative practices based on archetypical plots and cultural references makes it possible to manipulate the consciousness of the electorate to achieve political goals.Methodology and sources. Methodologically, the study is based on the cognitive-discursive approach to narrative analysis and the basic principles of the neural theory of language and metaphor.Results and discussion. Pointing to the manipulative effect of metaphorical framing in narrative, the authors attempt to present a holistic view of narrative as a tool for the formation of political views. The study is based on the premise that narrative employs the rich structure of human representations of events and actions, and the type of metaphor promoted in the narrative affects human understanding of events and the adoption of certain positions on an important public issue by readers of political blogs. The set of metaphors, politicians use, forms a special type of cultural narrative, which the authors interpret as an extended metaphorical narrative.Conclusion. Summarizing the effects of using metaphorically framed narratives in politics, the authors conclude that the narrative communication strategy in political blogging is a targeted impact on a communication partner by referring to various narrative plots; the modification or transformation of the potential and diversity of political narratives, as well as their effectiveness, speaks in favor of their use in domestic and foreign policy communication to contribute to a symmetrical communication space aimed at successful cooperation rather than confrontation.

Enhancing visual reinforcement learning with State–Action Representation

Despite the remarkable progress made in visual reinforcement learning (RL) in recent years, sample inefficiency remains a major challenge. Many existing approaches attempt to address this by extracting better representations from raw images using techniques like data augmentation or introducing some auxiliary tasks. However, these methods overlook the environmental dynamic information embedded in the collected transitions, which can be crucial for efficient control. In this paper, we present STAR: State-Action Representation Learning, a simple yet effective approach for visual continuous control. STAR learns a joint state–action representation by modeling the dynamics of the environment in the latent space. By incorporating the learned joint state–action representation into the critic, STAR enhances the value estimation with latent dynamics information. We theoretically show that the value function can still converge to the optima when involving additional representation inputs. On various challenging visual continuous control tasks from DeepMind Control Suite, STAR achieves significant improvements in sample efficiency compared to strong baseline algorithms.

Contextual visual and motion salient fusion framework for action recognition in dark environments

Infrared (IR) human action recognition (AR) exhibits resilience against shifting illumination conditions, changes in appearance, and shadows. It has valuable applications in numerous areas of future sustainable and smart cities including robotics, intelligent systems, security, and transportation. However, current IR-based recognition approaches predominantly concentrate on spatial or local temporal information and often overlook the potential value of global temporal patterns. This oversight can lead to incomplete representations of body part movements and prevent accurate optimization of a network. Therefore, a contextual-motion coalescence network (CMCNet) is proposed that operates in a streamlined and end-to-end manner for robust action representation in darkness in a near-infrared (NIR) setting. Initially, data are preprocessed to separate foreground, normalized, and resized. The framework employs two parallel modules: the contextual visual features learning module (CVFLM) for local feature extraction, and the temporal optical flow learning module (TOFLM) for acquiring motion dynamics. These modules focus on action-relevant regions used shift window-based operations to ensure accurate interpretation of motion information. The coalescence block harmoniously integrates the contextual and motion features within a unified framework. Finally, the temporal decoder module discriminatively identifies the boundaries of the action sequence. This sequence of steps ensures the synergistic optimization of both CVFLM and TOFLM and thorough competent feature extraction for precise AR. Evaluations of CMCNet are carried out on publicly available datasets, InfAR and NTURGB-D, where superior performance is achieved. Our model yields the highest average precision of 89% and 85% on these datasets, respectively, representing an improvement of 2.25% (on InfAR) compared to conventional methods operating at spatial and optical flow levels which underscores its efficacy.

Beyond balance: The role of the Vestibular system in action recognition

BackgroundAction recognition is a fundamental aspect of human interaction. This process is mediated by the activation of shared sensorimotor representations during action execution and observation. Although complex movements involving balance or head and trunk rotations require vestibular signals for effective execution, their role in the recognition of others' actions is still unknown. ObjectiveTo investigate the causal involvement of the vestibular system in the discrimination of actions performed by others and whether this is influenced by motor familiarity. MethodsIn a single-blind design involving 25 healthy participants, Galvanic Vestibular Stimulation (GVS) was administered during an Action Discrimination Task (ADT), in which videos of actions categorized as vestibular/non-vestibular and familiar/unfamiliar were presented. Following each video, participants were required to identify the climax of the previously viewed action between two image options, using a two-alternative forced choice paradigm. The ADT was performed in active and sham GVS conditions, with left or right anodal montages. Response Times (RTs), Accuracy, and subjective motor familiarity were recorded for each action category. ResultsIn sham GVS condition, an overall familiarity effect was observed, where RTs for familiar actions were faster than RTs for unfamiliar ones, regardless of vestibular engagement (p < .001; ηp2 = .80). Conversely, under active GVS, a selective interference of the identification of vestibular familiar actions was observed compared to sham. Specifically, GVS prolonged RTs for recognizing familiar vestibular actions (p = .004, d = .59) while concurrently enhancing visual sensitivity (d’) for the same actions (p = .03, r = .21). ConclusionThese findings demonstrate the contribution of the vestibular system to action recognition. GVS disrupted the sensorimotor representation of vestibular actions and led to increased reliance on an alternative processing system focused on visual analysis of limb positions. This dissociation provides valuable insights for future investigations into the complex relationship between vestibular signals and cognitive processes involved in action identification, essential for developing innovative GVS interventions, particularly for individuals with sensorimotor or vestibular disorders.

Language-guided temporal primitive modeling for skeleton-based action recognition

Human actions describe the complex body dynamics, which are defined by an ordered set of sub-action primitives. Existing methods in skeleton-based action recognition primarily focus on designing various networks to learn the entire action representation, in alignment with the action label. However, training the heterogeneous action as a whole increases the burden on the network and is not conducive to learning the temporal structure of the action. In this paper, the traditional action label is extended into the ordered primitive language description by employing a large language model, which not only unveils the temporal structure of the action but also enriches its connotation. Based on this, we propose a language-guided skeleton representation learning network (LGS-Net) that leverages the language description to guide the skeleton feature learning from the ordered primitive and global action levels. Especially, the primitive guidance aligns the features of skeleton clips with those of the ordered primitive descriptions while preserving temporal order, which promotes the network to model the temporal primitive and capture the internal structure within the action from skeletons. To enhance the cross-modal alignment performance, we develop an innovative temporal alignment loss supplemented with diversity and sparsity regularization terms to generate the discriminative multi-modal representation. Evaluated on three benchmark datasets, NTU-60, NTU-120 and N-UCLA, the proposed LGS-Net achieves comparable results to state-of-the-art methods, which proves the effectiveness of the language-guided learning mechanism.

Leveraging Generative AI to Accelerate Biocuration of Medical Actions for Rare Disease.

Structured representations of clinical data can support computational analysis of individuals and cohorts, and ontologies representing disease entities and phenotypic abnormalities are now commonly used for translational research. The Medical Action Ontology (MAxO) provides a computational representation of treatments and other actions taken for the clinical management of patients. Currently, manual biocuration is used to assign MAxO terms to rare diseases, enabling clinical management of rare diseases to be described computationally for use in clinical decision support and mechanism discovery. However, it is challenging to scale manual curation to comprehensively capture information about medical actions for the more than 10,000 rare diseases. We present AutoMAxO, a semi-automated workflow that leverages Large Language Models (LLMs) to streamline MAxO biocuration for rare diseases. AutoMAxO first uses LLMs to retrieve candidate curations from abstracts of relevant publications. Next, the candidate curations are matched to ontology terms from MAxO, Human Phenotype Ontology (HPO), and MONDO disease ontology via a combination of LLMs and post-processing techniques. Finally, the matched terms are presented in a structured form to a human curator for approval. We used this approach to process 4,918 unique medical abstracts and identified annotations for 21 rare genetic diseases, we extracted 18,631 candidate disease-treatment curations, 538 of which were confirmed and transferred to the MAxO annotation dataset. The results of this project underscore the potential of generative AI to accelerate precision medicine by enabling a robust and comprehensive curation of the primary literature to represent information about diseases and procedures in a structured fashion. Although we focused on MAxO in this project, similar approaches could be taken for other biomedical curation tasks.

A Hippocampal-parietal Network for Reference Frame Coordination.

Navigating space and forming memories based on spatial experience are crucial for survival, including storing memories in an allocentric (map-like) framework and conversion into body-centered action. The hippocampus and parietal cortex (PC) comprise a network for coordinating these reference frames, though the mechanism remains unclear. We used a task requiring remembering previous spatial locations to make correct future action and observed that hippocampus can encode the allocentric place, while PC encodes upcoming actions and relays this to hippocampus. Transformation from location to action unfolds gradually, with 'Came From' signals diminishing and future action representations strengthening. PC sometimes encodes previous spatial locations in a route-based reference frame and conveys this to hippocampus. The signal for the future location appears first in PC, and then in hippocampus, in the form of an egocentric direction of future goal locations, suggesting egocentric encoding recently observed in hippocampus may originate in PC (or another "upstream" structure). Bidirectional signaling suggests a coordinated mechanism for integrating map-like, route-centered, and person-centered spatial reference frames at the network level during navigation.

Force reflections of auditory and tactile action-effect weighting in motor planning

Most voluntary actions have only few goals, which provides considerable freedom in the selection of action parameters. Recent studies showed that task-irrelevant aspects of the task context influence the motor parameters of the actions in a way which seems to reflect the relative importance of these aspects within the underlying action representation. The present study investigated how the intensity of auditory action-effects affected force exertion patterns in a self-paced action production task. Participants applied force impulses with their index finger on a force-sensitive resistor every three seconds. In four separate conditions, force impulses elicited no sound, or elicited tones with 69, 59 or 49 dB intensity. The results showed that participants applied more force when tone intensity was lower, and when tones were absent. These force differences were also present in the first 60 ms following tone onset, implying that these reflected differences in motor planning. The results are compatible with the notion that actions are represented in terms of their sensory effects, which are weighted differently—presumably to maintain an optimal level of overall auditory and tactile stimulation in the present case. These results hint at the potential usefulness of motor parameters as readouts of action intentions.

Variation-aware directed graph convolutional networks for skeleton-based action recognition

Directed Graph convolutional networks (DGCNs) have been indeed gaining attention and being applied in skeleton-based action recognition tasks to capture the hierarchical relationships of skeleton via directed graph topology. However, they typically pay the same attention to regions with variations and regions with relative statics of the skeleton, which leads to low accuracy when recognizing fine-grained actions. To this end, we design an innovative variation-aware directed graph convolutional network (VA-DGCN) to focus on the regions where the variation takes place. VA-DGCN comprises a variation-aware directed spatial convolution (VDSC) module and a multi-scale contrastive temporal convolution (MCTC) module. Specifically, in order to capture subtle variations in fine-grained actions, VDSC introduce the average posture of the action sequence as the static anchor. In VDSC, the channel-specific topology branch is designed to model the different kinematic properties of different channels to extract global features, to which global-attention graph convolution is added for unnaturally connected joints. Subsequently, we identify the regions with variations by comparing global features acquired from action sequence and average posture. In temporal, MCTC comprises multiple branches for extracting temporal features at different scales. Moreover, to maximize the mutual information between branches, we introduce contrastive learning to drive the module to learn more meaningful action representations. We conduct extensive experiments on three public datasets to validate the feasibility and efficacy of our proposed VA-DGCN.

Multi-performance enhanced eco-driving strategy for connected fuel cell hybrid electric bus based on stein soft actor-3-critic

Eco-driving strategies increasingly emphasize enhancing multi-performance including safety, comfort, durability, and traffic efficiency under optimal energy-saving. This paper introduces a multi-performance enhanced eco-driving strategy for connected fuel cell hybrid electric buses (FCHEBs) based on the stein soft actor-3-critic (SSA3C) algorithm. By leveraging the correlation between the international roughness index (IRI) and weighted root mean square acceleration (WRMSA), energy management system (EMS) and adaptive cruise control (ACC) are integrated the to enhancing multi-performance including vertical comfort and energy efficiency. To address the insufficient action representation and overestimation of Q-values of traditional soft actor-critic (SAC) algorithm, improvements are made to the actor and critic networks using stein variational gradient descent (SVGD) and clipped triple Q-learning (3C), respectively. Composite cycles integrating the standard cycle and IRI are developed to enhance the strategy's generalization. Experimental results indicate that the proposed eco-driving strategy, while maintaining dynamics, safety, durability, and efficiency, efficiently achieves 98.1 % energy savings and equivalent comfort level as dynamic programming (DP). Ablation experiments on vertical comfort demonstrate improvements of 1.24 % in energy efficiency and 2.94 % in WRMSA. Finally, the effectiveness of the network improvements and the adaptability of the proposed strategy are demonstrated through comparative experiments.

Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semisupervised Skeleton-Based Action Recognition.

Contrastive learning has been successfully leveraged to learn action representations for addressing the problem of semisupervised skeleton-based action recognition. However, most contrastive learning-based methods only contrast global features mixing spatiotemporal information, which confuses the spatial- and temporal-specific information reflecting different semantic at the frame level and joint level. Thus, we propose a novel spatiotemporal decouple-and-squeeze contrastive learning (SDS-CL) framework to comprehensively learn more abundant representations of skeleton-based actions by jointly contrasting spatial-squeezing features, temporal-squeezing features, and global features. In SDS-CL, we design a new spatiotemporal-decoupling intra-inter attention (SIIA) mechanism to obtain the spatiotemporal-decoupling attentive features for capturing spatiotemporal specific information by calculating spatial- and temporal-decoupling intra-attention maps among joint/motion features, as well as spatial- and temporal-decoupling inter-attention maps between joint and motion features. Moreover, we present a new spatial-squeezing temporal-contrasting loss (STL), a new temporal-squeezing spatial-contrasting loss (TSL), and the global-contrasting loss (GL) to contrast the spatial-squeezing joint and motion features at the frame level, temporal-squeezing joint and motion features at the joint level, as well as global joint and motion features at the skeleton level. Extensive experimental results on four public datasets show that the proposed SDS-CL achieves performance gains compared with other competitive methods.

Representations of group actions and their applications in cryptography

Cryptographic group actions provide a flexible framework that allows the instantiation of several primitives, ranging from key exchange protocols to PRFs and digital signatures. The security of such constructions is based on the intractability of some computational problems. For example, given the group action (G,X,⋆), the weak unpredictability assumption (Alamati et al. (2020) [1]) requires that, given random xi's in X, no probabilistic polynomial time algorithm can compute, on input {(xi,g⋆xi)}i=1,…,Q and y, the set element g⋆y.In this work, we study such assumptions, aided by the definition of group action representations and a new metric, the q-linear dimension, that estimates the “linearity” of a group action, or in other words, how much it is far from being linear. We show that under some hypotheses on the group action representation, and if the q-linear dimension is polynomial in the security parameter, then the weak unpredictability and other related assumptions cannot hold. This technique is applied to some actions from cryptography, like the ones arising from the equivalence of linear codes, as a result, we obtain the impossibility of using such actions for the instantiation of certain primitives.As an additional result, some bounds on the q-linear dimension are given for classical groups, such as Sn, GL(Fn) and the cyclic group Zn acting on itself.

Entertainment type robots based on machine learning and game teaching mode applied in dance action planning of art teaching

We studied the use of machine learning models for training dance action models, collected dance action data, annotated and classified it, and established a training set for dance action models. We trained the training set to learn the feature representation and pattern recognition capabilities of dance actions. Through training and tuning the model, a model that can accurately recognize and generate dance movements was obtained. Evaluate the similarity between two dance movements and select the appropriate dance movements to form a smooth dance sequence. A planning algorithm was designed based on the kinematic and dynamic characteristics of robots to generate dance action paths suitable for the robot’s body conditions. Considering factors such as joint limitations, body stability, and smooth movement of the robot, generate a reasonable dance motion path while ensuring safety. Through on-site testing and data analysis of the system, it has been verified that it can effectively generate diverse and expressive dance movements, bringing a unique viewing experience to entertainment venues.