Posture Estimation Research Articles

Special Issue on Navigation and Control Technologies for Autonomous Mobility

Autonomous mobility, as exemplified by self-driving cars, autonomous mobile robots, drones, etc., is essential to the acceleration and practical application of transportation services and the automation of delivery, guidance, security, and inspection. Therefore, in recent years, expectations have been building for autonomous mobility to grow as a technology that not only improves the convenience and comfort of transportation and the efficiency of logistics but also leads to solutions to various social problems. Various technological elements are required to ensure the safety and quality of autonomous mobility. For example, technology is needed to create environmental maps and automatically determine obstacles based on data acquired by cameras and sensors such as LiDAR. Technologies for planning appropriate routes and controlling robots safely and comfortably are also essential. This special issue highlights 24 exciting papers, including 20 research papers, three letters, and one development report. They are related to “recognition,” “decision and planning,” and “control” technologies for autonomous mobile robots, such as self-driving cars and drones. The papers’ keywords are as follows: • Collision avoidance, path planning, path tracking control • Motion control, attitude control • Measurement, position and posture estimation, modeling • Point cloud processing We would like to express our gratitude to all authors and reviewers, and we hope that this special issue contributes to future research and development in autonomous mobility.

IoT based Human Activity Recognition using Deep learning

Artificial intelligence and the Internet of things (IoT) are the fastest and latest growing technologies that can handle a huge amount of data in computing services. This paper presents a smart human activity recognition system based on IoT that can be used for surveillance purposes working as IoT-based armour. Pose estimation model viz. MoveNet has been employed to extract the anatomical key points from RGB video frames. Different subjects from different camera angles were employed to make the approach person-independent. Diverse Machine learning models such as Decision tree, support vector machines, XGboost, and random forest classifiers were employed using extracted keypoints for training the model for estimating human activity during posture estimation monitoring. SMS are sent to the designated person with the raising of buzzer alarm in case of anomalous behaviour detection.

Estimation for Human Motion Posture and Health Using Improved Deep Learning and Nano Biosensor

To improve the technical level of human motion posture and health estimation, a human motion posture and health estimation algorithm based on Nano biosensor and improved deep learning is proposed. First, we use Nano biological acceleration sensor and Nano biological angular velocity sensor to obtain human motion posture and health data. Second, after the fusion processing of human motion posture and health data, we use the motion posture coordinate system conversion unit and the physiological information recognition unit to convert the coordinate system of human motion angular velocity and acceleration data and recognize the physiological information of blood pressure and heart rhythm. Finally, the convolution neural networks (CNN) in deep learning is improved to obtain the deformable CNN. The transformed angular velocity, physiological information recognition results and other human posture data are input into the deformable CNN, and the human posture estimation results are output. Experiments show that proposed algorithm can accurately obtain human posture data, can quickly and accurately estimate human posture, and has a good application effect. It has important application value in identity recognition and sports performance analysis.

A Soft Enveloping Gripper with Enhanced Grasping Ability via Morphological Adaptability

Hitherto, automated grasping with robotic grippers requires adjusting the posture and force of the fingers based on the size, geometry, stiffness, and pose of the objects. To provide a simpler but efficient grasping methodology, a soft enveloping gripper is presented and investigated how its morphological adaptability improves the grasping ability by comparing its performance with fingered grippers. Results show that this enveloping gripper can omnidirectionally envelop objects via active–passive interaction, which allows the gripper to 100% grasp the object located at different positions within range and keep their orientations. However, the grasping success rate and orientation error of the fingered grippers highly depend on the relative position and angle of the objects to the grippers, as well as the number of fingers. The dynamic vibration and decay time of the enveloping gripper when grasping a 500 g weight are, both, approximately one‐sixth of those of the two‐fingered gripper when grasping a 12.37 g cube. This enveloping gripper can automatically grasp objects (including deformable ones) lying in different poses without posture estimation and force control with a simple vision‐based automatic grasping method. The enveloping grasping method may open an avenue for simple, low cost yet powerful automatic grasping applications.

Yoga Pose Estimation along with Human Posture Detection using Deep Learning Approach

The yoga is the ancient art for keeping the human body healthy and fit and acquiring mental peace. This position in yoga almost matches with human posture for comfortable physical activity. The different sequence of body joint action leads to a specific yoga asana that has its own benefit for health. If asana are done in improper way, it would create a bad impact not only in health but also in mental peace. The advancement in computer vision technology help the yoga practitioner identify if they are doing right asana by applying various machine learning and deep learning algorithms for pose estimation. This article has done the concise literature survey on different machine and deep learning algorithm available in computer vision technology. The survey portraits the different classification for human posture estimation and yoga pose estimation along with their contributions and specific difference in between them. Especially the CNN, RNN and LSTM algorithm evolution is briefly narrated in this concise survey for yoga pose estimation.

Design of Metaverse-Based Physical Fitness Service for the Enhancement of Exercise Capability for Youth

This study explains the case study to develop a metaverse-based training system for the management of the physical fitness of youth. The study targeted developing a smart mirror-based AR physical fitness system and a bone age-based physical fitness management service to enhance exercise capability in youth. Accordingly, the data on youth’s bone age, physical strength, and physique were collected. Based on the collected data, the stage of the human development in youth was categorized, and the physical fitness management system providing the exercise program customized for the phases of physical development was implemented. Furthermore, a solution was devised so that youth with less physical exercise experience could practice physical fitness training without a coach using the smart mirror-based AR physical fitness system. This system uses an RGB-D camera and a three-dimensional posture estimation algorithm to recognize a user’s movement, determine movement accuracy, and provide feedback for the improved exercise effect. The XR Physical Fitness, using a smart mirror implemented in this article, features two features of metaverse’s augmented reality and lifelogging, enabling the youth to enjoy more immersive and enjoyable exercise experiences.

Single 24-GHz FMCW Radar-Based Indoor Device-Free Human Localization and Posture Sensing With CNN

The position information and posture information of a device-free human in a confined indoor environment have multiple uses in health monitoring and other areas. In this study, we aim to use a single-input-single-output (SISO) frequency-modulated continuous-wave (FMCW) radar at 24-GHz band for simultaneous localization and posture estimation of one device-free human target. This device is easy to deploy in new setups. We use image formation to convert temporal measurements of the radar signal into image-like data and convert the problem of simultaneous position estimation and posture perception into an image classification problem. Leveraging the use of convolutional neural networks (CNNs) for image classification, we design a variety of tests to explore the most favorable parameters of the CNN model and then use the best practice model to accomplish the classification task involving a fusion of localization and pose recognition. To explain the primary causes of inaccuracies, we examine not only cases based on a fused position and posture dataset but also cases based on position-only and posture-only datasets. On both real-life datasets, our proposed scheme can achieve 98% classification accuracy and less than 1-m localization accuracy within 0.95 cumulative error probability within the area of interest, outperforming traditional classification methods.

KILO - Fitness application with posture estimation

The lockdown measures taken by countries across the globe in light of the COVID-19 pandemic has made people realize the importance of a healthy lifestyle. As more people try to change their habits to adopt fitness routines, they tend to perform exercises without knowing which ones are for beginners and how long they should be working out for. In proposed work, we focus on implementing a fully functional workout app built using flutter, a cross-platform development framework to develop applications for iOS and android, supported by a backend service provided by firebase. This paper surveys different technologies used in pose estimation /classification as well as the different architectures used for the app development in Flutter. It also focuses on the importance of yoga and a technology aided approach for performing yoga poses. The grave nature of challenges faced while doing yoga such as performing it with incorrect form or postures as well as the cost of classes. With novel computer vision techniques, human pose estimation sees a great future. The work delves into the means of deploying yoga pose estimation on an android application, the modeling of the app and the working of each component.

Motion Capture Benchmark of Real Industrial Tasks and Traditional Crafts for Human Movement Analysis

Human movement analysis is a key area of research in robotics, biomechanics, and data science. It encompasses tracking, posture estimation, and movement synthesis. While numerous methodologies have evolved over time, a systematic and quantitative evaluation of these approaches using verifiable ground truth data of three-dimensional human movement is still required to define the current state of the art. This paper presents seven datasets recorded using inertial-based motion capture. The datasets contain professional gestures carried out by industrial operators and skilled craftsmen performed in real conditions in-situ. The datasets were created with the intention of being used for research in human motion modeling, analysis, and generation. The protocols for data collection are described in detail, and a preliminary analysis of the collected data is provided as a benchmark. The Gesture Operational Model, a hybrid stochastic-biomechanical approach based on kinematic descriptors, is utilized to model the dynamics of the experts’ movements and create mathematical representations of their motion trajectories for analysis and quantifying their body dexterity. The models allowed accurate the generation of human professional poses and an intuitive description of how body joints cooperate and change over time through the performance of the task.

Deep learning-based activity recognition and fine motor identification using 2D skeletons of cynomolgus monkeys.

Video-based action recognition is becoming a vital tool in clinical research and neuroscientific study for disorder detection and prediction. However, action recognition currently used in non-human primate (NHP) research relies heavily on intense manual labor and lacks standardized assessment. In this work, we established two standard benchmark datasets of NHPs in the laboratory: MonkeyinLab (MiL), which includes 13 categories of actions and postures, and MiL2D, which includes sequences of two-dimensional (2D) skeleton features. Furthermore, based on recent methodological advances in deep learning and skeleton visualization, we introduced the MonkeyMonitorKit (MonKit) toolbox for automatic action recognition, posture estimation, and identification of fine motor activity in monkeys. Using the datasets and MonKit, we evaluated the daily behaviors of wild-type cynomolgus monkeys within their home cages and experimental environments and compared these observations with the behaviors exhibited by cynomolgus monkeys possessing mutations in the MECP2 gene as a disease model of Rett syndrome (RTT). MonKit was used to assess motor function, stereotyped behaviors, and depressive phenotypes, with the outcomes compared with human manual detection. MonKit established consistent criteria for identifying behavior in NHPs with high accuracy and efficiency, thus providing a novel and comprehensive tool for assessing phenotypic behavior in monkeys.

Posture Estimation by Clustering Pressure Information and Control Implementation for Pneumatically Driven Gait-Assistive Robot

Pneumatic artificial muscles (PAMs) are light and soft, and are expected to be applied to gait assistance robots with multiple actuators on the human body. The PAMs can be used as not only actuators, but also sensors to detect the gait phase by using their deformable bodies whose internal pressure changes in response to the wearer’s gait. However, conventional methods to detect the gait phase by PAMs have targeted single point detection in a gait phase and used for only ON/OFF control of the PAM actuators. In this study, we proposed an algorithm to estimate the postural state of the wearer, especially the state of both hip joints, from the internal pressure information of the PAMs with a small amount of calculation by using clustering method, and succeeded in controlling the PAMs’ pressure continuously based on this algorithm. The effectiveness of the proposed control method was verified through gait-assistive experiments using a treadmill. We measured the electromyogram of the adductor longus muscle under 3 subjects and a one-sided significant difference test was performed. As a result, we confirmed significant differences at the 1% significance level for 2 subjects and at the 10% significance level for the remaining subject, allowing us to evaluate the effectiveness of the proposed PAM control strategy.

Estimation of upper body posture while riding a bicycle based on load measurement

Estimation of upper body posture while riding a bicycle based on load measurement

Markerless Posture Estimation of Flexible Cord-like Objects by Video Processing with Deep Learning

Markerless Posture Estimation of Flexible Cord-like Objects by Video Processing with Deep Learning

A hybrid neural network-based intelligent body posture estimation system in sports scenes.

Body posture estimation has been a hot branch in the field of computer vision. This work focuses on one of its typical applications: recognition of various body postures in sports scenes. Existing technical methods were mostly established on the basis of convolution neural network (CNN) structures, due to their strong visual information sensing ability. However, sports scenes are highly dynamic, and many valuable contextual features can be extracted from multimedia frame sequences. To handle the current challenge, this paper proposes a hybrid neural network-based intelligent body posture estimation system for sports scenes. Specifically, a CNN unit and a long short-term memory (LSTM) unit are employed as the backbone network in order to extract key-point information and temporal information from video frames, respectively. Then, a semi-supervised learning-based computing framework is developed to output estimation results. It can make training procedures using limited labeled samples. Finally, through extensive experiments, it is proved that the proposed body posture estimation method in this paper can achieve proper estimation effect in real-world frame samples of sports scenes.

Posture Estimation of Nursing Care using Plantar Pressure Distribution for Prevention of Lower Back Pain

Posture Estimation of Nursing Care using Plantar Pressure Distribution for Prevention of Lower Back Pain

Integrated Task Sensing and Whole Body Control for Mobile Manipulation With Series Elastic Actuators

In this paper, an integrated framework consisting of the sensing, navigation and control is proposed for an autonomous mobile manipulator driven by series elastic actuators (SEAs) to preform mobile manipulation tasks in unknown environments. First, ORB-SLAM2 technique is combined into the environment sensing by extracting the ORB features, automatic initialization, repositioning and loop detection for real-time posture estimation. Then, the navigation function is designed for generating collision-free trajectory in an environment with obstacles. To realize kinematic and dynamic control of the mobile manipulator with the developed SEA joints, the whole body dynamics is considered and described. And to handle dynamic uncertainties and the SEA inherent saturation limits, a novel adaptive neural network control considering the whole body dynamics is proposed. Without knowing the exact parameters of the whole body model, the designed controller merely requires the position and velocity of the actuators and links, which can make the tracking errors converge to zero and keep all signals uniformly bounded in the closed-loop system. The performance and efficiency of the proposed method are verified by extensive experiments. Note to Practitioners—This paper is motivated by issues of manipulation control of autonomous unmanned system. Traditional manipulation frameworks focus either on sensing or control by assuming that the environment is known, which would result in lacking of autonomy for a specified task. Since mobile manipulation tasks often consist of nonholonomic and holonomic constraints for wheeled mobile manipulators with differential steering. In addition, most current works for whole body control are based on the condition that robot dynamic parameters are known beforehand. Therefore, it is necessary to establish an enhanced framework to simultaneously deal with these problems. In this paper, an integrated navigation and control framework is proposed. To make the mobile manipulator work in the unknown environment, task sensing and whole body control for mobile manipulators are also developed. The framework is partitioned into the task sensing, navigation and control, where the mobile manipulator can fulfill the mobile manipulation tasks in the unstructured environments.

Researches Advanced in Human Pose Estimation Based on Deep Learning

Human pose estimation has always been a popular research topic in the discipline of computer vision, which aims to predict the spatial position of the human body’s key points (parts/joints) from a given image or video. Benefiting from the powerful feature representation ability of the convolutional neural networks, the pose estimation methods based on the deep learning have become the mainstream framework of this task, which obtain the human pose estimation results by extracting the human pose feature information at different scales with specific neural networks and using the corresponding methods to process it. This paper, based on the depth study of human body posture estimation can be divided into single-task and multi-person missions. The time and method of correlation are classified and summarized, respectively from the network architecture, algorithm and feature extraction of performance that are analyzed. The current problems existing in the research field and methods are summarized, and the research prospect for the future has been prospected.

An Extended Spatial Transformer Convolutional Neural Network for Gesture Recognition and Self-calibration Based on Sparse sEMG Electrodes.

sEMG-based gesture recognition is widely applied in human-machine interaction system by its unique advantages. However, the accuracy of recognition drops significantly as electrodes shift. Besides, in applications such as VR, virtual hands should be shown in reasonable posture by self-calibration. We propose an armband fusing sEMG and IMU with autonomously adjustable gain, and an extended spatial transformer convolutional neural network (EST-CNN) with feature enhanced pretreatment (FEP) to accomplish both gesture recognition and self-calibration via a one-shot processing. Different from anthropogenic calibration methods, spatial transformer layers (STL) in EST-CNN automatically learn the transformation relation, and explicitly express the rotational angle for coarse correction. Due to the shape change of feature pattern as rotational shift, we design the fine tuning layer (FTL) which is able to regulate rotational angle within 45°. By combining STL, FTL and IMU-based posture, EST-CNN is able to calculate non-discretized angle, and achieves high resolution of posture estimation based on sparse sEMG electrodes. Experiments collect frequently-used 3 gestures of 4 subjects in equidistant angles to evaluate EST-CNN. The results under electrodes shift show that the accuracy of gesture recognition is 97.06%, which is 5.81% higher than CNN, the fitness between estimated and true rotational angle is 99.44%.

Research on the posture estimation algorithm for installation and docking of marine engineering structures

This paper innovatively proposes a method of posture estimation using beacon coordinates for the installation process of marine engineering structures, aiming to solve the problem that posture changes cannot be obtained in real-time during the installation of marine engineering structures and has a guiding role in installation docking. Taking the jacket of an underwater wellhead base plate as an example, the ultra-short baseline (USBL) positioning system is used to obtain the coordinates of the three beacons on the jacket before and after the movement, and the rotation matrix and translation vector of this process are obtained through calculation. We construct a posture estimation model, which is obtained from the coordinates of three one-to-one corresponding points as input, and a mutual conversion method of the rotation matrix and the Euler angle is given. To verify the accuracy of the posture estimation algorithm, we conducted a USBL positioning experiment in an anechoic pool and simulated the movement process by rotating and translating the USBL transceiver. Finally, the experimental data was processed, and the root mean square error of the algorithm's theoretical coordinates and the USBL measured coordinates was less than 0.1 m, which proved the effectiveness of the algorithm.

Three-Dimensional Action Recognition for Basketball Teaching Coupled with Deep Neural Network

This study proposes a 3D attitude estimation algorithm using the RMPE algorithm coupled with a deep neural network that combines human pose estimation and action recognition, which provides a new idea for basketball auxiliary training. Compared with the traditional single-action recognition method, the present method makes the recognition accuracy better and the display effect more intuitive. The flipped classroom teaching mode based on this algorithm is applied to the college sports basketball optional course to explore the influence of this teaching mode on the classroom teaching effect. Compared with the evaluation index of action recognition, the experimental results of various action recognition methods and datasets are compared and analyzed, and it is verified that the method has a good recognition effect. The values of Topi and Top5 of the proposed method are 42.21% and 88.77%, respectively, which are 10.61% and 35.09% higher than those of the Kinetics-skeleton dataset. However, compared with the NTU RGM dataset, the recognition rate of Topi is significantly reduced. Compared with the traditional single-action recognition method, this method has better recognition accuracy and a more intuitive display effect. The fusion method of human posture estimation and motion recognition provides a new idea for basketball auxiliary training.