Features Of Skeleton Research Articles

Gaitdlf: global and local fusion for skeleton-based gait recognition in the wild

A new trend in long-range biometrics, gait recognition, is finding application in a number of different fields including video surveillance. Recently, with the increase in robustness of the pose estimator and the presence of various unpredictable factors in realistic gait recognition, skeleton-based methods with higher robustness have emerged to better meet the challenging gait recognition needs. However, existing approaches primarily focus on extracting global skeletal features, neglecting the intricate motion information of local body parts and overlooking inter-limb relationships. Our solution to these challenges is the dynamic local fusion network (GaitDLF), a novel gait neural network for complex environments that includes a detail-aware stream in addition to the previous direct extraction of global skeleton features, which provides an enhanced representation of gait features. To extract discriminative local motion information, we introduce predefined body part assignments for each joint in the skeletal structure. By segmenting and mapping the overall skeleton based on these limb site divisions, limb-level motion features can be obtained. In addition, we will dynamically fuse the motion features from different limbs and enhance the motion feature representation of each limb by global context information and local context information of the limb-level motion features. The ability to extract gait features between individuals can be improved by aggregating local motion features from different body parts. Based on experiments on CASIA-B, Gait3D, and GREW, we show that our model extracts more comprehensive gait features than the state-of-the-art skeleton-based method, demonstrating that our method is better suited to detecting gait in complex environments in the wild than the appearance-based method.

GAMNet: A deep learning approach for precise gesture identification

Deep learning has gained popularity across several industries, including object recognition and classification. In the case of Convolutional Neural Networks (CNN), the first layers extract the most noticeable elements, such as shape and margin. As the model progresses, it learns to extract more complex features such as texture and color; conversely, skeleton features encompass significant locations (joints) that do not naturally align with the grid-like architecture intended for these networks. This study emphasizes the importance of structural features in enhancing the performance of deep learning models. It introduces the Gesture Analysis Module Network (GAMNet), which computes abstract structural values within the architecture for feature extraction, prioritization, and classification. These values go through a rigorous evaluation process along with the cutting-edge deep learning model, CNN, and result in intermediate representations, leading to better performance in gesture analysis. An automated dance gesture identification system can address the challenges of recognizing hand movements in unpredictable lighting, varied backgrounds, noise, and changing camera angles. Despite these challenges, GAMNet performed remarkably well, surpassing renowned models like VGGNet, ResNet, EfficientNet, and CNN, achieving a classification accuracy of 96.80%, even in challenging image circumstances. This paper highlights how GAMNet can revolutionize the world of classical Indian dance, opening up new opportunities for research and development in this field.

Research on human behavior recognition algorithm based on HRNet and LSTM with model lightweight

This paper proposes a lightweight human action recognition algorithm for pedestrian behavior recognition. First, the skeleton feature information is input into the HRNet network model. In order to selectively enhance more details containing the target features and suppress irrelevant or weak features, an external attention mechanism is added to the HRN child model. Secondly, in order to extract the temporal characteristics of the target feature vector and ensure the continuity of actions in human behavior recognition, a dual-stream network based on HRNet and Long Short-Term Memory (LSTM) is constructed; finally, due to the huge model, it cannot be well transplanted to embedded. Therefore, this paper uses depthwise separable convolution to lightweight the network model. The experimental results show that in terms of human behavior recognition, the method in this paper has better recognition accuracy than Two-stream, Multi-streamCNN, Cov3DJ, ConvNets, JTM, ASM-3, RF+SW, hd-CNN and TPSMMs.

Seismoelectric wave propagation through a fluid-saturated porous sandwiched interlayer

The seismoelectric waves generated in a fluid-saturated porous interlayer and the reflection/transmission characteristics at the interfaces of sandwiched structure are studied in this paper. Aiming to describe the strong attenuation and dispersion simultaneously at seismic and sonic frequencies, a new fractional viscoelastic model for the solid skeleton is introduced to the Pride's seismoelectric coupling theory. Using the interfacial continuity of stress, displacements, magnetic and electric fields, the reflection/transmission coefficients are determined. The numerical results are provided and shown graphically. The effects of seismoelectric coupling and the viscoelastic characteristics of solid skeleton on the reflection and transmission of seismoelectric waves are mainly studied. It is observed that the thickness, porosity and the viscoelastic characteristics of solid skeleton have evident effects not only on the electromagnetic waves but also on the seismic waves.

Recognising pedestrian behaviour using a multi-channel spatiotemporal fusion network

To protect public safety, video abnormal behaviour detection is essential. The low accuracy of identifying abnormal behaviour in pedestrians is the focus of this study. To address this, an abnormal target identification method based on multi-feature fusion of trajectory skeleton is proposed. The process begins with defining the type of abnormal behaviour in accordance with the environmental requirements. Next, pedestrian identification is carried out in the designated area, the pedestrian is tracked to determine its movement trajectory, and the image coordinates of the corresponding relevant nodes are calculated by analysing the human posture. Lastly, the track and skeleton features are integrated to classify the normal and abnormal behaviours, and the identification of abnormal target behaviours is accomplished. Test results on the behavioural analysis database indicate that this algorithm's accuracy is 87.08%, which is higher than the single-feature identification method's detection efficiency of abnormal behaviours.

Work procedure action recognition based on skeleton and video features

In the field of industrial engineering, traditional methods for analyzing manufacturing process actions have limitations such as time-consuming, labor-intensive, and experience-dependent. To address these challenges in action analysis, we propose an intelligent action recognition method based on both skeleton and video features, aiming to replace manual decomposition of action elements. The MediaPipe framework is used for human posture estimation to obtain the skeleton sequence, and the CNN-GRU model is constructed action recognition based on skeleton features. For hand movements involving the use of industrial gloves, an enhanced TimeSformer video understanding model is introduced for action recognition based on video features. This improvement incorporates uniform attention and external attention mechanisms, resulting in enhanced model performance. The final experimental validation for the self-constructed process action dataset shows that the online detection speed of the skeleton action recognition model reaches 25 FPS, and the accuracy of the end-to-end video action recognition model is improved by 10.5 percentage points compared to the base model.

Expanding Multiple-Resonance Structure of a Double-Borylated Skeleton by Fusing with Indolocarbazole a Multiple-Resonance Donor for Narrowband Green Emission.

Double-borylated multiple-resonance (MR) skeletons are promising templates for high performance, while the chemical design space is relatively limited. Peripheral segments are often used to decorate/fuse MR skeletons and modulate the photophysics but they can also cause unwanted spectral broadening. Herein, a narrowband MR emitter ICzDBA by fusing an MR-featured donor segment indolocarbazole into a double-borylated MR skeleton is developed. In ICzDBA, the nitrogen atom located away from the core benzene ring can also contribute to the generation of the overall MR-featured distribution through the long-range conjugation effect, along with the other boron/nitrogen atoms on the phenyl center. Thus, ICzDBA in toluene displays a narrowband emission peaking at 507nm with a full width at half maximum of merely 20nm (0.09eV). Moreover, organic light-emitting diode devices using ICzDBA emitter exhibit ultrapure green emission with Commission Internationale de l'Eclairage (CIE) coordinates of (0.27, 0.70) and a high external quantum efficiency of 32.5%. These results manifest the importance of MR characters of peripheral decorations/fusions in preserving the narrowband features of MR skeletons, which provides a solution for further expanding MR structures with well-maintained narrowband characters.

Object Knowledge Integration for Skeleton-Based Action Recognition

3D-Skeleton-based action recognition has been widely adopted due to its efficiency and robustness to complex backgrounds. While it is capable of conveying a significant amount of information regarding the dynamics of human poses, we argue that its performance is curtailed when confronted with actions involving interactions between humans and objects due to the absence of the study of the surrounding objects. It is of great importance to delve deeper into the study of human-object interactions for skeleton-based action recognition. This paper proposes a novel approach to represent the spatial-temporal skeleton features, along with the present nearby objects and their dynamics. To accomplish this, a new formulation named object knowledge is presented, which entails the categorization of object characteristics, based on whether or not the object necessitates a motion analysis. With a piece of prior knowledge, in cases where it is required, the motion is calculated, while for those where it is not necessary, only the category of object is considered. This object knowledge is then early-fusion along with the skeleton representation, in such a way that it fits into the self-attention model. The experimental results on different popular action recognition datasets (NTU RGB+D 60, NTU RGB+ D 120 illustrate that the proposed approach outperforms the current state-of-the-art methods.

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition.

Skeleton-based human interaction recognition is a challenging task in the field of vision and image processing. Graph Convolutional Networks (GCNs) achieved remarkable performance by modeling the human skeleton as a topology. However, existing GCN-based methods have two problems: (1) Existing frameworks cannot effectively take advantage of the complementary features of different skeletal modalities. There is no information transfer channel between various specific modalities. (2) Limited by the structure of the skeleton topology, it is hard to capture and learn the information about two-person interactions. To solve these problems, inspired by the human visual neural network, we propose a multi-modal enhancement transformer (ME-Former) network for skeleton-based human interaction recognition. ME-Former includes a multi-modal enhancement module (ME) and a context progressive fusion block (CPF). More specifically, each ME module consists of a multi-head cross-modal attention block (MH-CA) and a two-person hypergraph self-attention block (TH-SA), which are responsible for enhancing the skeleton features of a specific modality from other skeletal modalities and modeling spatial dependencies between joints using the specific modality, respectively. In addition, we propose a two-person skeleton topology and a two-person hypergraph representation. The TH-SA block can embed their structural information into the self-attention to better learn two-person interaction. The CPF block is capable of progressively transforming the features of different skeletal modalities from low-level features to higher-order global contexts, making the enhancement process more efficient. Extensive experiments on benchmark NTU-RGB+D 60 and NTU-RGB+D 120 datasets consistently verify the effectiveness of our proposed ME-Former by outperforming state-of-the-art methods.

Controlling Media Player with Hands: A Transformer Approach and a Quality of Experience Assessment

In this article, we propose a Hand Gesture Recognition (HGR) system based on a novel deep transformer (DT) neural network for media player control. The extracted hand skeleton features are processed by separate transformers for each finger in isolation to better identify the finger characteristics to drive the following classification. The achieved HGR accuracy (0.853) outperforms state-of-the-art HGR approaches when tested on the popular NVIDIA dataset. Moreover, we conducted a subjective assessment involving 30 people to evaluate the Quality of Experience (QoE) provided by the proposed DT-HGR for controlling a media player application compared with two traditional input devices, i.e., mouse and keyboard. The assessment participants were asked to evaluate objective (accuracy) and subjective (physical fatigue, usability, pragmatic quality, and hedonic quality) measurements. We found that (i) the accuracy of DT-HGR is very high (91.67%), only slightly lower than that of traditional alternative interaction modalities; and that (ii) the perceived quality for DT-HGR in terms of satisfaction, comfort, and interactivity is very high, with an average Mean Opinion Score (MOS) value as high as 4.4, whereas the alternative approaches did not reach 3.8, which encourages a more pervasive adoption of the natural gesture interaction.

Somatotopic features of the collagen skeleton of the knee meniscus in men

Somatotopic features of the collagen skeleton of the knee meniscus in men

Phenotypic features, transcriptomic signatures and genomic regions associated with body ellipticity in the flatfish Senegalese sole

Body shape is a complex and plastic trait with a high impact on fish performance and commercialization. Shape is particularly relevant in flatfish with highly asymmetric bodies. In this study we investigated the skeleton features, transcriptomic profiles and genomic regions in two Senegalese sole groups from a full-sib family with positive (named as HE) or negative (LE) breeding values for body ellipticity (ELL). Soles from HE group were heavier and displayed more elongated bodies than LE. A skeleton analysis showed that HE had a higher number of vertebrae, longer vertebral bodies and intervertebral spaces and a lower incidence of skeletal fusions than LE. With respect to the gender effect, females showed a higher height of ventral body and some differences in the ratio urostyle to skeleton length than males for similar ELL values. The PCA analysis using 28 dimensionless morphological traits indicated that PC1 and PC2 explained 51.3% of the total variance and they were mainly associated with ELL and gender variation, respectively. Expression analysis using two factors (ELL and gender) revealed a major effect of gender in muscle, pterygiophores and liver with 6764, 6506 and 7559 differentially expressed transcripts (DETs) between sexes, respectively. In brain, only 16 DETs were identified. When HE and LE were compared controlling by gender, we identified the subset of DETs with conserved responses in both sexes or those regulated in a sex-specific manner in each tissue. A comparison across tissues identified 31 ubiquitous transcripts as differentially expressed in at least two of them. Functional analysis showed that liver and pterygiophores had the highest number of enriched gene ontology categories (24 and 14, respectively). Some of these enriched pathways were related to regulation of cell cycle, cell structure and shape, locomotory behavior, immune system, hormonal responses and homeostasis. A BSR-Seq analysis confirmed five significant genomic windows associated with ELL by using the tricube-smoothed G' and Fst statistics. Windows located in linkage groups (LG) 5 and LG14 included the malt1 and vtg3 genes, respectively, that were also detected within the subset of ubiquitous DETs. Moreover, malt1 appeared located very close to the bmpr1b gene within the significance hotspot of LG5 indicating that mediators of the inflammatory responses and BMP signaling pathway could be responsible for the differences found in the skeleton features and body ellipticity.

Multi-stimuli responsive behaviors in a new chiral hybrid nitroprusside salt (R-3-hydroxypyrrolidinium)2[Fe(CN)5(NO)]

Multi-stimuli responsive materials controlled and coupled by two or more channels have a broad range of applications in the field of switches, memories, and molecular machines. The exploration of the material is currently focused on the pure organic system, which limits the development of such materials greatly. In this work, we present a new chiral organic-inorganic hybrid salt, (R-3-hydroxypyrrolidinium)2[Fe(CN)5(NO)] (1), which exhibits rare multi-stimuli responsive behaviors in thermal, mechanical and optical channels. In detail, 1 undergoes a C2-P21221 phase transition deriving from the thermal motion of organic cations with the increase of temperature, but the reverse transition can only be induced by mechanical pressure. Moreover, polycrystalline hybrid salt showed photo-responsive performance, i.e., the ground-state N-bound nitrosyl ligand adopts two configurations in excited states caused by light in 532 nm irradiation, accompanying with a photo-induced structural transformation of the anionic framework. Namely, the thermal motion characteristics of organic cations, the photoresponse characteristics of anionic inorganic skeleton and the pressure characteristics from hydrogen bonds are simultaneously integrated in 1. This unprecedented coupling mechanism of multi-stimuli responses makes 1 a potential candidate for future multichannel data storage applications.

Skeleton-based coordinate system construction method for non-cooperative targets

Due to the lack of structure and part information in non-cooperative targets, the task of pose estimation suffers from unstable and unreliable initial coordinate system. In this paper, we introduce a curve-skeleton-based method to construct the robust coordinate system automatically for non-cooperative targets. First, a nonmanifold-Laplacian-based method is proposed to extract the curve skeleton from point clouds. The proposed method solves the over-contraction problem while preserves the topology and structure of skeleton well during point clouds contraction. Additionally, we create the skeleton graph to detect the skeleton structure and key points automatically. Then, the detected skeleton feature is used to construct target coordinate system, and improve the stability and robustness. Specifically, a real-scan dataset is constructed in this paper, and the proposed point cloud contraction method outperforms previous methods in structural representation and visualization. The mean rotation error of proposed coordinate system construct method performs 0.1249° and 1.2135° in 0%–30% noise ratio of distortions and outliers, and performs 7.04° and 4.21° in real-scan aircraft and satellite datasets. The results in complete and incomplete datasets verify the robustness and reliability of our skeleton-based coordinate system.

Disentangling Modality and Posture Factors: Memory-Attention and Orthogonal Decomposition for Visible-Infrared Person Re-Identification.

Striving to match the person identities between visible (VIS) and near-infrared (NIR) images, VIS-NIR reidentification (Re-ID) has attracted increasing attention due to its wide applications in low-light scenes. However, owing to the modality and pose discrepancies exhibited in heterogeneous images, the extracted representations inevitably comprise various modality and posture factors, impacting the matching of cross-modality person identity. To solve the problem, we propose a disentangling modality and posture factors (DMPFs) model to disentangle modality and posture factors by fusing the information of features memory and pedestrian skeleton. Specifically, the DMPF comprises three modules: three-stream features extraction network (TFENet), modality factor disentanglement (MFD), and posture factor disentanglement (PFD). First, aiming to provide memory and skeleton information for modality and posture factors disentanglement, the TFENet is designed as a three-stream network to extract VIS-NIR image features and skeleton features. Second, to eliminate modality discrepancy across different batches, we maintain memory queues of previous batch features through the momentum updating mechanism and propose MFD to integrate features in the whole training set by memory-attention layers. These layers explore intramodality and intermodality relationships between features from the current batch and memory queues under the optimization of the optimal transport (OT) method, which encourages the heterogeneous features with the same identity to present higher similarity. Third, to decouple the posture factors from representations, we introduce the PFD module to learn posture-unrelated features with the assistance of the skeleton features. Besides, we perform subspace orthogonal decomposition on both image and skeleton features to separate the posture-related and identity-related information. The posture-related features are adopted to disentangle the posture factors from representations by a designed posture-features consistency (PfC) loss, while the identity-related features are concatenated to obtain more discriminative identity representations. The effectiveness of DMPF is validated through comprehensive experiments on two VIS-NIR pedestrian Re-ID datasets.

Advanced Camera-Based Scoliosis Screening via Deep Learning Detection and Fusion of Trunk, Limb, and Skeleton Features.

Scoliosis significantly impacts quality of life, highlighting the need for effective early scoliosis screening (SS) and intervention. However, current SS methods often involve physical contact, undressing, or radiation exposure. This study introduces an innovative, non-invasive SS approach utilizing a monocular RGB camera that eliminates the need for undressing, sensor attachment, and radiation exposure. We introduce a novel approach that employs Parameterized Human 3D Reconstruction (PH3DR) to reconstruct 3D human models, thereby effectively eliminating clothing obstructions, seamlessly integrated with an ISANet segmentation network, which has been enhanced by Multi-Scale Fusion Attention (MSFA) module we proposed for facilitating the segmentation of distinct human trunk and limb features (HTLF), capturing body surface asymmetries related to scoliosis. Additionally, we propose a Swin Transformer-enhanced CMU-Pose to extract human skeleton features (HSF), identifying skeletal asymmetries crucial for SS. Finally, we develop a fusion model that integrates the HTLF and HSF, combining surface morphology and skeletal features to improve the precision of SS. The experiments demonstrated that PH3DR and MSFA significantly improved the segmentation and extraction of HTLF, whereas ST-based CMU-Pose substantially enhanced the extraction of HSF. Our final model achieved a comparable F1 (0.895 ±0.014) to the best-performing baseline model, with only 0.79% of the parameters and 1.64% of the FLOPs, achieving 36 FPS-significantly higher than the best-performing baseline model (10 FPS). Moreover, our model outperformed two spine surgeons, one less experienced and the other moderately experienced. With its patient-friendly, privacy-preserving, and easily deployable solution, this approach is particularly well-suited for early SS and routine monitoring.

Bottom-up deposition of lithium on 3D lithiophobic–lithiophilic host for long-life lithium metal anodes

The features of 3D carbon skeletons can regulate the composition of the SEI layer to achieve bottom-up Li deposition behavior.

Research on the evolution of aggregate skeleton characteristics of asphalt mixture under uniaxial compression loading

The aggregate skeleton is the intrinsic core of asphalt mixtures for load transfer, which is quite different under various gradations. For evaluating the loading characteristics of the aggregate skeleton, this research combined the 3D blue-ray scanning technology and discrete element method (DEM) to establish the numerical models of different graded asphalt mixtures considering the real aggregate morphology. A linear parallel bond model was used to simulate the adhesion effect of asphalt, and the uniaxial compression numerical simulation was verified by indoor tests. In addition, the contact force, contact number, anisotropy, and rotation angle were used to evaluate the aggregate skeleton evolution under uniaxial compression loading. Furthermore, the contribution of each grade aggregate in the gradation to the skeleton structure was analyzed. Results show that the average contact force is higher in large-size aggregates above 9.5 mm, which exhibit greater load transfer efficiency. Moreover, 4.75–9.5 mm size aggregates combine the functions of bearing and filling, and its role is significantly affected by the gradation. It is also found that the internal skeleton structure can be optimized and adjusted under stress. The contact numbers of different graded mixtures follow a close decrease, and the strong contacts are mostly provided by coarse aggregates above 4.75 mm. The anisotropy development of aggregate skeleton contact force is induced by the stress, and the deviator fabric of asphalt mixtures show a good linear positive correlation with the uniaxial compressive strength (UCS). Besides, fine aggregates exhibit greater cumulative rotation angle and instability effect, it is feasible to detect asphalt mixture deformation by the rotation angle of particles.

Demonstration and Analysis of DRN Deep Learning Architecture for Step-by-Step Disassembly of Dahongquan

Abstract The Dahongquan martial arts style is characterized by its intricate and varied maneuvers, posing challenges for traditional video demonstrations to fully satisfy learners’ needs. This study introduces a cutting-edge approach utilizing a visual neural network to deconstruct and map these martial arts movements in a 3D virtual environment. Initially, preprocessing involves extracting data from exercise videos, employing a combination of video annotation and skeleton feature extraction. Subsequently, a deep residual network (DRN) with multi-scale feature extraction and dense residual attention mechanisms is employed, facilitating a systematic breakdown of the martial arts movements. The third phase involves constructing a 3D action mapping and demonstration process, utilizing cubic B-spline curve fitting and cylinder deformation techniques to enhance the representation of movement based on the skeletal model. Remarkably, the DRN network, without the need for extensive training datasets, achieves flawless recognition of various gongfu combinations and records a recognition rate of 92.92% for the most challenging boxing movements. The efficacy of this model is further corroborated by its enhancement of the demonstration process, achieving an overall learner satisfaction rating of 5.26. The disassembly and demonstration strategy proposed in this study has proven effective during experimental sessions, demonstrating significant potential for advancing martial arts training methodologies.

Human Behavior Recognition Based on CNN-LSTM Hybrid and Multi-Sensing Feature Information Fusion

To address the human activity recognition problem and its application in practical situations, a CNN-LSTM hybrid neural network model capable of automatically extracting sensor data features and memorizing temporal activity data is designed and improved by integrating CNN and gated recurrent units as a variant of RNN. A multi-channel spatiotemporal fusion network-based two-person interaction behavior recognition method is proposed for two-person skeletal sequential behavior recognition. Firstly, a viewpoint invariant feature extraction method is used to extract two-player skeleton features, then a two-layer cascaded spatiotemporal fusion network model is designed, and finally, a multi-channel spatiotemporal fusion network is used to learn multiple sets of two-player skeleton features separately to obtain multi-channel fusion features, and the fusion features are used to recognize the interaction behavior, and the weights are shared among the channels. Applying the algorithm in the paper to the UCF101 dataset for experiments, the accuracy of the two-person cross-object experiment can reach 96.42% and the accuracy of the cross-view experiment can reach 97.46%. The method in the paper shows better performance in two-player interaction behavior recognition compared to typical methods in this field.