Pose Classification Research Articles

Development and performance evaluation of an indoor thermal environment control algorithm incorporating MET estimation model with object detection

As people increasingly spend time indoors, the significance of the indoor environment in influencing occupant quality of life is becoming more pronounced. Traditionally, indoor environment control primarily relied on fixed temperature settings, which failed to accommodate the diverse circumstances of occupants. This approach limited the creation of a comfortable indoor environment and the enhancement of energy efficiency. Consequently, there is growing interest in occupant-centric control (OCC), which integrates metabolic rate (MET) information, which is a critical factor in determining the thermal sensation of occupants. Previously, a method was developed to estimate MET by classifying occupant poses and detecting the objects they interact with from indoor images. This study aims to develop and experimentally validate an indoor thermal environment control algorithm (ITEC-algorithm) using the MET estimation model and assess its effectiveness and applicability in real building environments.The performance evaluation revealed that the ITEC-algorithm significantly enhanced the comfort ratios, achieving improvements of up to 59% compared to the fixed temperature control and 28% compared to the control methods that only used the pose classification model for MET estimation. The energy consumption varied depending on the activity and control method, with a reduction of up to 88% compared to fixed temperature control. These results indicate that thermal comfort can be enhanced while minimizing unnecessary energy consumption by incorporating the MET of the occupants. Consequently, it has been confirmed that the ITEC-algorithm effectively improves thermal comfort by managing the MET of various occupants.

Automated Unmanned Aerial System for Camera-Based Semi-Automatic Triage Categorization in Mass Casualty Incidents

Using drones to obtain vital signs during mass-casualty incidents can be extremely helpful for first responders. Thanks to technological advancements, vital parameters can now be remotely assessed rapidly and robustly. This motivates the development of an automated unmanned aerial system (UAS) for patient triage, combining methods for the automated detection of respiratory-related movements and automatic classification of body movements and body poses with an already published algorithm for drone-based heart rate estimation. A novel UAS-based triage algorithm using UAS-assessed vital parameters is proposed alongside a robust UAS-based respiratory rate assessment and pose classification algorithm. A pilot concept study involving 15 subjects and 30 vital sign measurements under outdoor conditions shows that with our approach, an overall triage classification accuracy of 89% and an F1 score of 0.94 can be achieved, demonstrating its basic feasibility.

Human Pose Classification Based on Pose Long Short-Term Memory

Objective: At present, pose classification research is a hot topic in the field of computer vision, playing a crucial role in surveillance security, and motion data mining. The research serves as a crucial follow-up to three dimensional (3D) pose estimation and makes a significant contribution to the advancement of artificial intelligence. Pose classification is a complex problem involving a large amount of complicated data, making its digital modeling and further processing challenging. Methods: This article proposes the Pose long short - term memory (LSTM) method to classify 3D poses reconstructed from 2D image sequences. Result: Compared with traditional methods, the experimental tests show that the performance of the Pose LSTM is more superior in all aspects. Conclusion: Pose LSTM is particularly effective for recognizing continuous action poses because they can capture temporal dependencies in sequential data. In the context of pose recognition, it can analyze the sequence of poses over time and understand how these poses transition from one to another, making it possible to accurately classify or predict actions that unfold over a series of frames.

Productivity Measurement with the Use of Pose Classification and Machine Learning with Fuzzy Approximate Reasoning

Productivity Measurement with the Use of Pose Classification and Machine Learning with Fuzzy Approximate Reasoning

Object classification through heterogeneous fog with a fast data-driven algorithm using a low-cost single-photon avalanche diode array

This study presents a framework for classifying a wooden mannequin’s poses using a single-photon avalanche diode (SPAD) array in dynamic and heterogeneous fog conditions. The target and fog generator are situated within an enclosed fog chamber. Training datasets are continuously collected by configuring the temporal and spatial resolutions on the sensor's firmware, utilizing a low-cost SPAD array sensor priced below $5, consisting of an embedded SPAD array and diffused VCSEL laser. An extreme learning machine (ELM) is trained for rapid pose classification, as a benchmark against CNN. We quantitatively justify the selection of nodes in the hidden layer to balance the computing speed and accuracy. Results demonstrate that ELM can accurately classify mannequin poses when obscured by dynamic heavy fog to 35 cm away from the sensor, enabling real-time applications in consumer electronics. The proposed ELM achieves 90.65% and 89.58% accuracy in training and testing, respectively. Additionally, we demonstrate the robustness of both ELM and CNN as the fog density increases. Our study also discusses the sensor’s current optical limitations and lays the groundwork for future advancements in sensor technology.

ClassyPose: A Machine‐Learning Classification Model for Ligand Pose Selection Applied to Virtual Screening in Drug Discovery

Determining the target‐bound conformation of a drug‐like molecule is a crucial step in drug design, as it affects the outcome of virtual screening (VS), and paves the way for hit‐to‐lead and lead optimization. While most docking programs usually manage to produce at least a near‐native pose for a bioactive molecule inside its binding pocket, their integrated classical scoring functions (SFs) generally fail to prioritize this pose. Many studies have been carried out to tackle this SF problem, offering multiple pose refinement and/or classification methods, albeit with limitations. This study presents a new support vector machine model for pose classification, called “ClassyPose”, which predicts the probability that a receptor‐bound ligand conformation could be near‐native, without any additional pose optimization step. Trained on protein‐ligand extended connectivity features extracted from over 21 600 crystal and docking poses of diverse ligands, this model outperformed other machine‐learning algorithms and three existing SFs in terms of docking power, identifying the native ligand pose as top‐ranked solution for more than 90% of entries in two test sets. It also achieved high specificity (above 0.96), and improved VS performance when used for pose selection. This efficient, user‐friendly tool and all related data are available at https://github.com/vktrannguyen/Classy_Pose.

Deep Learning Based Technical Classification of Badminton Pose with Convolutional Neural Networks

Deep Learning Based Technical Classification of Badminton Pose with Convolutional Neural Networks

Machine learning for hand pose classification from phasic and tonic EMG signals during bimanual activities in virtual reality.

Myoelectric prostheses have recently shown significant promise for restoring hand function in individuals with upper limb loss or deficiencies, driven by advances in machine learning and increasingly accessible bioelectrical signal acquisition devices. Here, we first introduce and validate a novel experimental paradigm using a virtual reality headset equipped with hand-tracking capabilities to facilitate the recordings of synchronized EMG signals and hand pose estimation. Using both the phasic and tonic EMG components of data acquired through the proposed paradigm, we compare hand gesture classification pipelines based on standard signal processing features, convolutional neural networks, and covariance matrices with Riemannian geometry computed from raw or xDAWN-filtered EMG signals. We demonstrate the performance of the latter for gesture classification using EMG signals. We further hypothesize that introducing physiological knowledge in machine learning models will enhance their performances, leading to better myoelectric prosthesis control. We demonstrate the potential of this approach by using the neurophysiological integration of the "move command" to better separate the phasic and tonic components of the EMG signals, significantly improving the performance of sustained posture recognition. These results pave the way for the development of new cutting-edge machine learning techniques, likely refined by neurophysiology, that will further improve the decoding of real-time natural gestures and, ultimately, the control of myoelectric prostheses.

Survey on Yogic Posture Recognition using Deep Learning

The abstract emphasizes the growing importance of yoga pose detection, especially in integrating technology into yoga practice. It highlights the various benefits of yoga, such as physical health, mental well-being, and stress reduction. The paper discusses the significance of automated yoga pose detection in providing real-time feedback and enhancing self-correction. The research explores different approaches to yoga pose classification, focusing on PoseNet and KNN classifier. Using deep learning algorithms and computer vision techniques like Open Pose, human pose estimation is employed to identify yoga postures. A combination of CNN and LSTM is utilized for real-time yoga pose recognition from monitored videos. Existing techniques for yoga pose recognition often fail in real-world situations, prompting the need for more efficient methods. The paper presents a computationally efficient approach using deep learning for yoga pose recognition in complex environments. Artificial intelligence is proposed as a substitute for yoga instructors, assisting individuals with their poses. The paper introduces algorithms for creating skeletons of yoga poses and hand mudras, extracting joint angles as features for machine learning and deep learning models. Furthermore, deep learning techniques are developed to detect incorrect yoga postures, enabling users to upload recorded videos for analysis. The system advises users on improving their poses by identifying abnormal angles between the actual and desired poses

471 Using Computer Vision and Wearable Devices to Improve Care of Parkinson’s Disease

OBJECTIVES/GOALS: Inexpensive, accurate home monitoring is the standard-of-care in many diseases like hypertension or diabetes; however, it has yet to be widely used in neurodegenerative diseases. We used wearable activity monitors and computer-vision evaluated assessments to estimate Parkinson’s disease (PD)-related disease burden. METHODS/STUDY POPULATION: We recruited 22 people from the University of Iowa Movement Disorders Clinic. Each person completed a standardized set of 3 fine motor tasks using their hands. We recorded a video of this activity, which was evaluated using MediaPipe - an open-source pose classification program from Alphabet - as well as had an nurse-practitioner evaluate the performance on a validated scale (UPDRS). Participants wore a Fitbit Inspire 3 activity tracker at home for the next two weeks. We quantified disease burden using the Parkinson’s Disease Questionnaire 39 - a validated 39-item survey about the intensity of PD-related impairment. Using data from the videos and activity trackers, we estimated 1) the standardized UPDRS assessment of motor impairment and 2) the total PDQ-39 score. RESULTS/ANTICIPATED RESULTS: We found observationally recorded fastest sustained (at least 5 minutes) walking speed was a strong predictor of PDQ-39, explaining over one third of the variability in the measure. Range of motion in the videos was a significant predictor of UPDRS scores; however, was only weakly related to the overall PDQ-39 score. Further processing of the signals from the video, including wavelets and frequency domain analysis, may provide better predictive capabilities. PDQ-39 subscores (e.g., cognition, social support, mobility) will be the subject of further analysis. DISCUSSION/SIGNIFICANCE: Home monitoring has become the standard in other fields because of the better generalizability of home measurements. Improving the detection and evaluation of PD using home monitoring will lead to more timely and accurately changes in medication and less need for clinic visits - especially off levodopa.

HUMAN POSE ESTIMATION

In "Human Pose Estimation" with integrated feedback mechanisms to assess and guide users in achieving correct poses. Utilizing advanced deep learning techniques in computer vision, the system swiftly detects key points on the human body and provides instant feedback on pose accuracy. Built on convolutional neural networks trained on extensive pose datasets, the system includes pose detection, classification, and feedback stages. By comparing detected poses with predefined correct poses, the system delivers positive feedback for accurate poses and corrective guidance for deviations. Key Words: Human Pose Estimation, Pose Detection, Pose Classification, Correct Pose Assessment, Fitness Training, Key Points Detection, Correct Pose Thresholds

Enhancing Docking Accuracy with PECAN2, a 3D Atomic Neural Network Trained without Co-Complex Crystal Structures

Decades of drug development research have explored a vast chemical space for highly active compounds. The exponential growth of virtual libraries enables easy access to billions of synthesizable molecules. Computational modeling, particularly molecular docking, utilizes physics-based calculations to prioritize molecules for synthesis and testing. Nevertheless, the molecular docking process often yields docking poses with favorable scores that prove to be inaccurate with experimental testing. To address these issues, several approaches using machine learning (ML) have been proposed to filter incorrect poses based on the crystal structures. However, most of the methods are limited by the availability of structure data. Here, we propose a new pose classification approach, PECAN2 (Pose Classification with 3D Atomic Network 2), without the need for crystal structures, based on a 3D atomic neural network with Point Cloud Network (PCN). The new approach uses the correlation between docking scores and experimental data to assign labels, instead of relying on the crystal structures. We validate the proposed classifier on multiple datasets including human mu, delta, and kappa opioid receptors and SARS-CoV-2 Mpro. Our results demonstrate that leveraging the correlation between docking scores and experimental data alone enhances molecular docking performance by filtering out false positives and false negatives.

CAM based fine-grained spatial feature supervision for hierarchical yoga pose classification using multi-stage transfer learning

In this paper, we propose a technique for hierarchical yoga pose classification (YPC) in a multi-stage multi-tasking framework. We propose a three-stage transfer learning based end-to-end training methodology. Novelty lies in (a) proposed supervised contrastive combined loss function for stage-1 training, (b) proposed Encoder–Decoder network architecture with attention mechanism for stage-3 training, (c) proposed spatial context aware multi-tasking combined loss function for stage-3 training. Firstly, for stage-1 training, we propose the usage of linear combination of three loss functions: cross-entropy, self-supervised contrastive loss and supervised contrastive loss in a multi-tasking manner. We introduce radial and cosine margin into the formulation of self-supervised and supervised contrastive loss to pull feature embeddings of same class closer together compared to feature embeddings of different classes. Weights learned over stage-1 training are subsequently fine-tuned over cross-entropy multi-tasking loss in stage-2. These stage-2 weights are transfer learned and are further fine-tuned in stage-3 training. For stage-3 training, we propose the usage of spatial context aware multi-tasking combined loss function. This loss function leverages on the fine-grained spatial features obtained from HiResCAM. These are processed in parallel with features obtained from XGrad-CAM (sensitivity and conservation axioms satisfying features) to further supervise the learning of hierarchical yoga pose classifier. We exemplify our methodology on the publicly available Yoga-82 large-scale dataset. We report peak Top-1 YPC accuracy of 95.89% over 6 pose classes (Yoga-6), 93.85% over 20 pose classes (Yoga-20) and 90.0% over 82 pose classes (Yoga-82). Our proposed method achieves 6.1% improvement over Top-1 classification accuracy in Yoga-6 hierarchy, 9.3% improvement in Yoga-20 hierarchy and 10.9% improvement in Yoga-82 hierarchy in comparison with state-of-the-art (SOTA) methodology. We achieve the current best Top-1 classification accuracies in all the three YPC hierarchies.

Skeleton-based Human Activity Classification in Sparse Image Sequences

Research results on human activity classification in video are described, based on initial human skeleton estimation in video frames. Both single person actions and two-person interactions are considered. The initial skeleton data is estimated in selected video frames by OpenPose, HRNet or other dedicated library. Important contributions of presented work are computational steps of skeleton tracking and -refinement, and relational feature extraction from pairs of skeleton joints. It is shown, that this feature engineering significantly increases the classification accuracy. Regarding the final neural network encoder-classifier, two different architectures are designed and tested. The first solution is a lightweight MLP network, implementing the idea of a "mixture of pose experts". Several pose classifiers (experts) are trained independently on different time periods (snapshots) of single-person visual actions (or 2-person interactions), while the final classification is a time-related pooling of weighted expert classifications. All pose experts use the same deep encoding network. The second (middle weight) solution is based on a LSTM network.Both solutions are trained and tested on the action set of the well-known NTU RGB+D dataset, although only 2D data are used.Our results show comparable performance with some of the best reported STM- and CNN-based classifiers for this dataset. We conclude that by reducing the noise of skeleton data, highly successful lightweight- and midweight-approaches to visual activity recognition in image sequences can be achieved.

An efficient multi-task learning CNN for driver attention monitoring

Driver Monitoring System (DMS), usually equipped with a camera, is an emerging vehicle safety system that can monitor driver attentiveness and trigger timely alarms when signs of inattention are detected. Since a single indicator (e.g., eye blink rate) is insufficient and unreliable to analyze driver attentiveness, almost all existing solutions train several independent models to identify driver facial states, such as face landmark, head pose, yawning, eye state, etc. However, apart from neglecting the inherent correlations between these related tasks, multiple models also raise challenges for vehicle safety-critical systems (e.g., hardware resources, software compatibility, and real-time response). In this paper, we propose a multi-task learning CNN framework (DANet) to unify the relevant tasks into one model and simultaneously output various driver facial states. By sharing the common features and parameters of highly related tasks, DANet avoids repetitive computations and mitigates single task overfitting. More importantly, the model provides a comprehensive overview of facial states while maintaining low complexity. We also propose two novel designs: (1) Dual-loss Block, which decomposes the pose estimation task into pose classification and coarse-to-fine regression; (2) Head Pose Penalization, which constrains the network to predict gaze direction based on predicted head pose. Our method achieves compelling results in both speed and accuracy on a vehicle computing platform, marking a momentous step in this field.

Comparison of CNN-based methods for yoga pose classification

Yoga is an exercise developed in ancient India. People perform yoga in order to have mental, physical, and spiritual benefits. While yoga helps build strength in the mind and body, incorrect postures might result in serious injuries. Therefore, yoga exercisers need either an expert or a platform to receive feedback on their performance. Since access to experts is not an option for everyone, a system to provide feedback on the yoga poses is required. To this end, commercial products such as smart yoga mats and smart pants are produced; Kinect cameras, sensors, and wearable devices are used. However, these solutions are either uncomfortable to wear or not affordable for everyone. Nonetheless, a system that employs computer vision techniques is a requirement. In this paper, we propose a deep-learning model for yoga pose classification, which is the first step of a quality assessment and personalized feedback system. We introduce a wavelet-based model that first takes wavelet transform of input images. The acquired subbands, i.e., approximation, horizontal, vertical, and diagonal coefficients of the wavelet transform are then fed into separate convolutional neural networks (CNN). The obtained probability results for each group are fused to predict the final yoga class. A publicly available dataset with 5 yoga poses is used. Since the number of images in the dataset is not enough for a deep learning model, we also perform data augmentation to increase the number of images. We compare our results to a CNN model and the three models that employ the subbands separately. Results obtained using the proposed model outperforms the accuracy output achieved with the compared models. While the regular CNN model has 61% and 50% accuracy for the training and test data, the proposed model achieves 91% and 80%, respectively.

Designing and Prototyping of AI-based Real-time Mobile Detectors for Calisthenic Push-up Exercise

Fitness exercises, including push-ups, are very beneficial to personal health. Many Artificial Intelligence (AI)-based fitness trainers are developed based on human pose estimation models or assisted by Internet of Things (IoT) devices. However, many of them require access to a graphing processing unit (GPU) for model training or IoT sensors to deploy, less accessible for individuals. In our work, we designed and prototyped real-time mobile push-up detectors using three distinctive approaches: (1) Push-up pose classification, (2) Angle-heuristic estimation and (3) Optical flow detection. We trained our deep-learning model with over 2000 images to achieve a high accuracy for real time deployment. Models are tested on our video dataset applied data augmentation techniques to simulate real-world environmental conditions to evaluate model performance based on accuracy metrics (precision, recall, F1 score) and processing frame rate (FPS). From the results, we concluded that the angle-heuristic estimation method has the best overall performance and we analysed the reasons for the relatively poorer performance of the push-up pose classification and optical flow detection methods. All methods developed are capable of working on mobile devices without the need of GPU or IoT sensors.

Workpiece Pose Classification Framework for Flexible Part Feeding Systems Using Machine Learning and Synthetic Datasets

In previous work, a concept for a novel flexible part feeding system based on aerodynamic feeding was presented. In contrast to conventional part feeding systems, such as vibratory bowl feeders, aerodynamic part feeding systems use controlled pressurized air jets to reorient the workpieces. The proposed concept aims to increase flexibility and eliminate retooling times to meet the challenges of modern automated production with regard to increasing uncertainties, shorter product life cycles and higher cost pressure. Regarding these objectives, the determination of the workpiece poses in the feeding system requires flexible image processing, which will be enabled using machine learning and synthetic datasets.This work presents a framework which enables the classification of workpiece poses using a standard industrial camera and convolutional neural networks (CNN). The training of CNNs usually requires large datasets. However, in order to eliminate the need to create and label datasets of real images, which would cause machine downtimes and increase the retooling effort, the CNNs will be trained with synthetic datasets. Based on the CAD-data of the workpieces, artificial images of each pose are rendered using the open source engine Blender. Several CNN architectures are selected, trained with the artificial datasets and evaluated using real images of the workpieces. The results show that high classification accuracies of over 99 % can be achieved. It is concluded that the presented approach enables the classification of workpiece poses with high accuracy, while eliminating the need for the cumbersome creation of real image datasets. The approach is not limited to the use in the aerodynamic part feeding system but can be transferred to other applications in which the orientation of a workpiece has to be classified.

Pose-Aware Facial Expression Recognition Assisted by Expression Descriptions

Although expression descriptions provide additional information about facial behaviors despite of different poses, and pose features are beneficial to adapt to pose variety, neither has been fully leveraged in facial expression recognition. This paper proposes a pose-aware text-assisted facial expression recognition method using cross-modality attention. Specifically, the method contains three components. The pose feature extractor extracts pose-related features from facial images, and then cooperates with a fully-connected layer for pose classification. When poses can be clearly discriminated and classified, features obtained from the extractor can represent the corresponding poses. To eliminate bias due to appearance and illumination, cluster centers are taken as the final pose features. The text feature extractor obtains embeddings from expression descriptions. These descriptions are first passed through Intra-Exp attention to obtain preliminary embeddings. To leverage the correlations among expressions, all expression embeddings are then concatenated and passed through Inter-Exp attention. The cross-modality module attempts to learn attention maps that distinguish the importance of facial regions by using prior knowledge about poses and expression descriptions. The image features weighted by the attention maps are utilized to recognize pose and expression jointly. Experiments on three benchmark datasets demonstrate the superiority of the proposed method.

Exploring the Use of Contrastive Language-Image Pre-Training for Human Posture Classification: Insights from Yoga Pose Analysis

Accurate human posture classification in images and videos is crucial for automated applications across various fields, including work safety, physical rehabilitation, sports training, or daily assisted living. Recently, multimodal learning methods, such as Contrastive Language-Image Pretraining (CLIP), have advanced significantly in jointly understanding images and text. This study aims to assess the effectiveness of CLIP in classifying human postures, focusing on its application in yoga. Despite the initial limitations of the zero-shot approach, applying transfer learning on 15,301 images (real and synthetic) with 82 classes has shown promising results. The article describes the full procedure for fine-tuning, including the choice for image description syntax, models and hyperparameters adjustment. The fine-tuned CLIP model, tested on 3826 images, achieves an accuracy of over 85%, surpassing the current state-of-the-art of previous works on the same dataset by approximately 6%, its training time being 3.5 times lower than what is needed to fine-tune a YOLOv8-based model. For more application-oriented scenarios, with smaller datasets of six postures each, containing 1301 and 401 training images, the fine-tuned models attain an accuracy of 98.8% and 99.1%, respectively. Furthermore, our experiments indicate that training with as few as 20 images per pose can yield around 90% accuracy in a six-class dataset. This study demonstrates that this multimodal technique can be effectively used for yoga pose classification, and possibly for human posture classification, in general. Additionally, CLIP inference time (around 7 ms) supports that the model can be integrated into automated systems for posture evaluation, e.g., for developing a real-time personal yoga assistant for performance assessment.