Posture Estimation Research Articles

A robust covariate‐invariant gait recognition based on pose features

Gait recognition uses video of human gait processed by computer vision methods to identify people based on walking style. The complexity introduced by covariates makes the previous methods less efficient and inaccurate. This study proposes an approach based on pose features to attempt gait recognition of people with an overcoat, carrying objects, or other covariates. It aims to estimate human locomotion using Convolutional Neural Networks. Gathering video data, extracting video frames in a particular order, posture estimation for each frame, using multilayer RNN for gait recognition from the pose, and obtaining one-dimensional object vectors, are all critical steps. Furthermore, these one-dimensional identification vectors are stored in a data set along with the name of the person walking in the video. The proposed data set is used to train a classification model to predict the person in a new video by first processing it to get its identification vector and then to use it as a test case in the classification model. A graphical user interface was also developed so that anyone with no programming or technical experience can easily use the tool. The developed application does everything for gait detection from mp4 videos by obtaining the identification vectors and saving them into the data set. Using this application, one can quickly identify the person walking in a video. The results obtained offered an accuracy from 60.88% to 95.23%.

A health monitoring system with posture estimation and heart rate detection based on millimeter-wave radar

Millimeter-wave radar is a device gaining popularity in many applications such as occupancy detection and people counting, vital sign detection and robotics. It has recently been used for human detection and tracking within home health care as a potential application area. Furthermore, increasing number of researchers utilize mmWave radar for heart rate detection. In this paper, we introduce a new home health monitoring system based on millimeter-wave radar, which is composed of posture estimation and heart rate detection. We focus on categorization of postures (standing, sitting and lying) for a single person in indoor environments. When the person is estimated as lying down, our system can further detect the heart rate. We establish a new point cloud dataset and show that, with simple algorithms employed, the system has an accuracy of around 99% for posture estimation and approximately 91% accuracy for detecting heart rate with only a single millimeter-wave radar.

Human elbow flexion behaviour recognition based on posture estimation in complex scenes

Aiming at the difficulty of recognising the smoking and making phone calls behaviours of people in the complex background of construction sites, a method of recognising human elbow flexion behaviour based on posture estimation is proposed. The human upper body key points needed are retrained based on AlphaPose to achieve human object localization and key points detection. Then, a mathematical model for human elbow flexion behaviour discrimination (HEFBD model) is proposed based on human key points, as well as locating the region of interest for small object detection and reducing the interference of complex background. A super-resolution image reconstruction method is used for pre-processing some blurred images. In addition, YOLOv5s is improved by adding a small object detection layer and integrating a convolutional block attention model to improve the detection performance. The detection precision of this method is improved by 5.6%, and the false detection rate caused by complex background is reduced by 13%, which outperforms other state-of-the-art detection methods and meets the requirement of real-time performance.

Assessing Posture And Activity Intensity Agreement Between Wearable Camera Images And Criterion Direct Observation

PURPOSE: To compare posture and activity intensity estimates annotated from wearable camera images (WCI) to criterion video-recorded direct observation (DO). METHODS: 19 healthy adults were observed over 53 visits (mean visit length = 89.7 ± 19.2 minutes), in a location of their choice. During these visits, participants were video-recorded performing free-living activities of daily living, while wearing a wearable camera at chest level. The camera was set to a high capture rate, where it took first-person images. Researchers annotated the WCI and DO videos for posture (sitting, standing, and movement). Activity intensity (sedentary, light, moderate-vigorous) from DO videos was determined from posture, while WCI were coded using a schema with matching Taylor codes to the Compendium of Physical Activities to extrapolate activity intensity. For analysis, a linear mixed-effects model calculated statistical bias and 95% CIs of WCI posture and activity intensity time in minutes to DO. Cohen’s weighted Kappa statistic (Κ) was calculated to assess the agreement of posture and activity intensity annotations to criterion DO. RESULTS: 4754 minutes of WC and DO data were collected. Significant bias was calculated for all outcome measures except movement posture (Figure 1). Kappa showed substantial agreement between WCI and DO to assess posture (K = 0.76), and moderate agreement between WCI and DO for estimating intensity (K = 0.55). CONCLUSIONS: When assessing posture in healthy adults, WCI show substantial agreement with criterion DO. WCI show moderate agreement with DO when used to estimate activity intensity. Future studies should explore WCI’s estimates of posture and activity intensity in more diverse populations and validate WCI activity intensity estimations against gold standard measurements of energy expenditure.

Detecting change in the routine of the elderly

Recent studies shown that elderly persons have a well-defined, regular life routine, organized around their postures, habits, environment, and social relations. Monitoring the daily routine will allow to detect person behavior changes. In this paper, we propose two solutions for detecting changes in the elderly person behavior based on the daily living activities monitoring. For both proposed methods, the person’s activity is recorded with a depth sensor to preserve his anonymity. In a first step, the recorded images are classified according to the posture of the person (sitting, standing, lying down, absent and fall) by a modified ResNet-18 model. The first method is a binary decision problem and consists in classifying routine and non routine day according to the postures. Then a behavior index is built using the output of the classifier. The second method consists in describing the routine day by the temporal succession in postures. A similarity between the recorded day and the routine day postures is computed using the minimum edit distance which acts as a behavior index. The number of transformations reflects the change in the person’s behavior. Both methods have been tested on a database of depth images recorded in a nursing home over a 85 days period. Inferring the whole dataset gives an accuracy of 89.09% and a F1-score of 85.84% for posture estimation. Using Random Forest, an accuracy of 92% was retrieved for routine/non routine day classification. The two proposed indexes proved their reliability in monitoring behavior changes. The first approach proposes a behavioral index with a median magnitude error less that 0.01 compared to the ground truth. The second one provides a significant difference in the number of transformations between the postures in routine days (Interquartile range 25th-75th=[28, 48]) and non routine days (Interquartile range 25th-75th=[55, 62]).

Depressive Symptoms Are Associated With Accelerometer-Measured Physical Activity and Time in Bed Among Working-Aged Men and Women

Background: Depression is a significant health problem, whereas higher physical activity (PA) associates with fewer depressive symptoms. We examined how self-reported depressive symptoms are associated with accelerometer-measured PA, standing, sedentary behavior, and time in bed (TIB) among 20- to 69-year-old men and women. Methods: The study is a part of the cross-sectional, population-based FinFit2017 study, in which depressive symptoms were assessed by modified nine-item Finnish version of the Patient Health Questionnaire, and physical behavior in terms of PA, sedentary behavior, standing, and TIB was assessed 24/7 by a triaxial accelerometer. During waking hours, the accelerometer was hip worn. Intensity of PA was analyzed by mean amplitude deviation and body posture by angle for posture estimation algorithms. During TIB, the device was wrist worn, and the analysis was based on the wrist movements. A total of 1,823 participants answered the nine-item Finnish version of the Patient Health Questionnaire and used the accelerometer 24 hr at least 4 days per week. Results: Men without depressive symptoms had on average more standing, light, and moderate to vigorous PA and steps, and less low and high movement TIB than the men with at least moderate symptoms, when age group, education, work status, marital status, and fitness were adjusted for. The asymptomatic women had more moderate to vigorous PA and steps and less high movement TIB than the women with at least moderate symptoms. Conclusions: Depressive symptoms were associated with lower levels of PA and longer TIB. It is important to identify these symptoms as early as possible to be able to initiate and target preventive actions, including PA promotion, to these symptomatic persons on time.

A Gaussian High-Dimensional Random Matrix-Based Method for Detecting Abnormal Student Behaviour in Chinese Language Classrooms

In this paper, a Gaussian high-dimensional random matrix approach is used to conduct an in-depth study and analysis of the detection of abnormal student behaviour in Chinese language classrooms, and a corresponding model is designed for practical application. The Gaussian high-dimensional random matrix technique and recurrent neural network technique are applied to build a basic technical framework for intelligent proctoring. An innovative method of human posture estimation is used to complete the proctoring task, and the structure of the Gaussian high-dimensional random matrix method is simplified and the model is compressed to achieve real-time and parallelism of the method. A PolSAR sparse representation classification (OGRM_SRC) algorithm based on orthogonal Gaussian random matrices (OGRM) is proposed to construct an observation dictionary based on polarisation features and OGRM to select typical ground target samples in polarized feature images. The system not only functions as a psychometric assessment of students but also is capable of detecting and analysing abnormal student behaviour based on the improved forest algorithm. The residuals of the pixel to be classified are calculated relative to each atom in the observation dictionary, and the minimum reconstruction residual is used as the classification criterion. The OGRM_SRC algorithm is applied to classify the pixel, and the final classification results are obtained and evaluated for accuracy. We propose a fine-grained action recognition optimisation method for recurrent neural networks, fusing temporal attention information and spatial attention information to improve the action recognition method and improve the model’s ability to recognise fine-grained abnormal behaviours such as looking left, looking right, and copying with head down. Based on this, we conducted some comparative experiments to validate the effectiveness of our work and verified that the system achieved a recognition efficiency of 76.8 FPS and a recognition accuracy of 98.5% in a graphical computing processor.

S08-3 Physical activity, sedentary behavior and sleep-time of Eupasmos-project participants in 16 European countries: European Union Physical Activity and Sport Monitoring System (EUPASMOS) project

Abstract Background Increasing number of studies use device-based measurements to report levels of physical activity (PA) and sedentary behavior (SB) of populations. The studies have used different devices, device placements and analysis methods, and thus the comparison of the results between studies and populations is very challenging. The purpose of the present study was to present preliminary results of PA, SB and sleep-time measured with a similar method among adults of 16 European countries. Methods The study is based on the Eupasmos-project where PA, SB and sleep-time of the participants (n = 6674) were measured by a tri-axial accelerometer 24/7 (UKK RM42, UKK Terveyspalvelut Oy, Tampere, Finland). Accelerometer was worn on an elastic hip-band during waking-hours and on a wrist-band attached to a non-dominant wrist during bed-time (sleeping). PA-parameters were based on mean amplitude deviation (MAD) of acceleration analyzed in 1min exponential moving average (epoch length 6s). Assessment of SB (e.g. sitting and laying down) and standing were based on the angle for posture estimation (APE). Sleep measurement was based on the movement of a non-dominant wrist during bed-time. Parameters analyzed were total daily times and different bout lengths of SB, standing, light PA, moderate-to-vigorous PA, sleep and mean number of daily steps. Results The preliminary analysis included 47-1952 participants/country. In all countries participants spent most of their waking-hours sedentary. On weekdays the country-mean of SB varied from 8h18min to 10h7min. Participants took on average 6907-9716 steps/day. Light PA varied from 3h46min to 5h10min and moderate-to-vigorous PA from 45 to 78min. Participants wore the device on their wrist (sleeping-time) 6h42min-8h37min on weekdays. Although the total times of these behaviors were quite similar between the countries, the variation increased when different bout lengths were analyzed: the longer the bout length, the larger the variation. The hour-by-hour pattern of these behaviors varied a lot between the countries as well. Conclusions General pattern of SB, PA and sleep-time was quite similar in all countries, but the deeper analysis revealed clear differences. To be able to compare levels and patterns of these behaviors it is essential to use the exactly same method in all participating countries.

PE Classroom Teaching Innovation Based on Deep Learning from the Perspective of New Curriculum Standard

In recent years, due to the influence of various factors, most of the physical quality of primary and secondary school students in China are in a state of subhealth. According to relevant studies, nearly 70 percent of Chinese students lack daily physical exercise. Physical quality is the primary factor and prerequisite of study and work, so how to improve the physical quality of primary and middle school students has become the top priority of physical education. Based on the requirements and guidance of China’s physical education in the new curriculum standard, this paper innovates China’s physical education classroom teaching to a certain extent based on deep learning algorithm. The final results show that the students who choose PE class based on deep learning algorithm account for about 85% of the total number of students, which exceeds the students who choose traditional PE class by 70%. Therefore, we believe that the estimation and recognition of students’ movement posture in PE class can not only greatly improve students’ enthusiasm for physical exercise but also avoid sports injuries caused by inaccurate operation in the process of exercise. Human posture estimation is to detect the position of each part of the human body from the image and calculate its direction and scale information. The advent of the era of big data is based on the relationship between multiple frames of images, while human posture recognition is based on the processing of single-frame static images. Correctly recognizing the pose information of multiple frames of continuous images still makes it possible to realize correct behavior analysis and understanding.

WSN-Driven Posture Recognition and Correction Towards Basketball Exercise

In order to enhance the daily training for basketball, this paper establishes a human posture estimation framework by using monocular camera and wireless sensor network. First, the daily basketball training images are collected by monocular camera and transmitted through wireless sensor network. Second, the collected images are processed by an observation and reasoning model which is based on component and graph reasoning. The basketball player's posture is depicted by the rotation invariant features of edge field. The extracted features are used to learn a boosting classifier as the observation model. The experimental results show that the posture recognition rate can achieve more than 88% for basketball player's action.

Research on Training Model of Volleyball Based on Flexible Strain Sensing Network for Training

This paper provides an in-depth investigation and analysis of volleyball training patterns using a sensing network composed of flexible strain sensors. This paper models the motion of the hand and leg joints based on an improved human posture estimation technique. The pattern recognition of human motion used the fusion of acceleration sensor and spiral meter data, the output of human motion information from the spiral meter, combined with GA-BP neural network algorithm to repair and fuse the output information, effectively improving the accuracy of posture angle measurement. Based on wireless positioning technology, an improved RFID positioning algorithm combining the LANDMARC algorithm and weighted center-of-mass algorithm in wireless sensor network is proposed, and the positioning accuracy reaches 89.2%; and the error feedback mechanism is introduced to improve the improved algorithm, and the positioning accuracy is improved again by 3.2%. Finally, the two-way dynamic time regularization algorithm is used to align the input action with the standard action, and the action pose evaluation index is compared and analyzed for the action sequence after alignment to obtain the final action quality comparison analysis results. In this paper, the proposed method is applied in practical training, which can effectively locate the stance of athletes and evaluate their movement quality without disturbing them and can provide valuable analysis information to support coaches’ guidance.

Combination of Deep Cross-Stage Partial Network and Spatial Pyramid Pooling for Automatic Hand Detection

The human hand is involved in many computer vision tasks, such as hand posture estimation, hand movement identification, human activity analysis, and other similar tasks, in which hand detection is an important preprocessing step. It is still difficult to correctly recognize some hands in a cluttered environment because of the complex display variations of agile human hands and the fact that they have a wide range of motion. In this study, we provide a brief assessment of CNN-based object identification algorithms, specifically Densenet Yolo V2, Densenet Yolo V2 CSP, Densenet Yolo V2 CSP SPP, Resnet 50 Yolo V2, Resnet 50 CSP, Resnet 50 CSP SPP, Yolo V4 SPP, Yolo V4 CSP SPP, and Yolo V5. The advantages of CSP and SPP are thoroughly examined and described in detail in each algorithm. We show in our experiments that Yolo V4 CSP SPP provides the best level of precision available. The experimental results show that the CSP and SPP layers help improve the accuracy of CNN model testing performance. Our model leverages the advantages of CSP and SPP. Our proposed method Yolo V4 CSP SPP outperformed previous research results by an average of 8.88%, with an improvement from 87.6% to 96.48%.

Prediction of anatomically and biomechanically feasible precision grip posture of the human hand based on minimization of muscle effort

We developed a method to estimate a biomechanically feasible precision grip posture of the human hand for a given object based on a minimization of the muscle effort. The hand musculoskeletal model was constructed as a chain of 21 rigid links with 37 intrinsic and extrinsic muscles. To grasp an object, the static force and moment equilibrium condition of the object, force balance between the muscle and fingertip forces, and static frictional conditions must be satisfied. We calculated the hand posture, fingertip forces, and muscle activation signals for a given object to minimize the square sum of the muscle activations while satisfying the above kinetic constraints using an evolutionary optimization technique. To evaluate the estimated hand posture and fingertip forces, a wireless fingertip force-sensing device with two six-axis load cells was developed. When grasping the object, the fingertip forces and hand posture were experimentally measured to compare with the corresponding estimated values. The estimated hand postures and fingertip forces were in reasonable agreement to the corresponding measured data, indicating that the proposed hand posture estimation method based on the minimization of muscle effort is effective for the virtual ergonomic assessment of a handheld product.

Human Pose Estimation Based on Multi-Spectral Attention and High Resolution Network

In view of the problem of feature information loss during multi-resolution feature fusion in human pose estimation, a lightweight high resolution human pose estimation network named Lite MSA-HRNet is designed based on Lite-HRNet and multi-spectral attention mechanism, which integrates multi-spectral attention mechanism into Lite-HRNet. Multiple frequency components are used to extract richer feature information, contributing to the repeated fusion of different resolution feature. A deconvolution module is used behind the main network to fuse the higher resolution features generated by itself with the high resolution features generated by the main network. Channel shuffle, pointwise group convolutions and depthwise separable convolution are introduced to lighten the residual block in the deconvolution module and improve the speed of network positioning key points. The experimental results on the COCO2017 data set show that Lite MSA-HRNet achieves a better balance between the accuracy and complexity of human posture estimation compared with other networks.

Development and Evaluation of Squatting Exercise Support System Using PoseNet Posture Estimation Library

Development and Evaluation of Squatting Exercise Support System Using PoseNet Posture Estimation Library

Matching Model of Dance Movements and Music Rhythm Features Using Human Posture Estimation.

The essential issue in music understanding and dance synthesis research is how to improve the degree of matching between dance motions and music rhythm elements. The matching model of dance movements and music rhythm features is created in this study based on human posture estimation research to tackle the problems that existing matching methods are prone to rapid changes in movements and cannot keep the original movement features. The rhythm properties of movement and music data are first analyzed, and then the degree of matching between movement and music pieces is measured using a dynamic time warping technique. A constraint-based dynamic planning approach is also used to synthesize the dance action sequence that best matches the supplied music. Experiments reveal that this model has a matching accuracy of 95.1 percent, which is higher than the two comparative models' matching accuracy of 5.2 percent and 7.6 percent, respectively. The matching score of this paper is high, reaching around 94 percent, according to the user research results. The method suggested in this work has apparent advantages in that it may efficiently help users in the arrangement of desired dance moves or backing music, and it can be applied in the field of actual choreography and scoring.

Privacy-Preserving In-Bed Pose and Posture Tracking on Edge.

In-bed behavior monitoring is commonly needed for bed-bound patient and has long been confined to wearable devices or expensive pressure mapping systems. Meanwhile, vision-based human pose and posture tracking while experiencing a lot of attention/success in the computer vision field has been hindered in terms of usability for in-bed cases, due to huge privacy concerns surrounding this topic. Moreover, the inference models for mainstream pose and posture estimation often require excessive computing resources, impeding their implementation on edge devices. In this paper, we introduce a privacy-preserving in-bed pose and posture tracking system running entirely on an edge device with added functionality to detect stable motion as well as setting user-specific alerts for given poses. We evaluated the estimation accuracy of our system on a series of retrospective infrared (LWIR) images as well as samples from a real-world test environment. Our test results reached over 93.6% estimation accuracy for in-bed poses and achieved over 95.9% accuracy in estimating three in-bed posture categories.

Determining human upper limb postures with a developed inverse kinematic method

AbstractBody posture determination methods have many applications, including product design, ergonomic workplace design, human body simulation, virtual reality, and animation industry. Initiated in robotics, inverse kinematic (IK) method has been widely applied to proactive human body posture estimation. The analytic inverse kinematic (AIK) method is a convenient and time-saving type of IK methods. It is also indicated that, based on AIK methods, a specific body posture can be determined by the optimization of an arbitrary objective function. The objective of this paper is to predict the postures of human arms during reaching tasks. In this research, a human body model is established in MATLAB, where the middle rotation axis analytic kinematic method is accomplished, based on this model. The joint displacement function and joint discomfort function are selected to be initially applied in this AIK method. Results show that neither the joint displacement function nor the joint discomfort function predicts postures that are close enough to natural upper limb postures of human being, during reaching tasks. Therefore, a bi-criterion objective function is proposed by integrating the joint displacement function and joint discomfort function. The accuracy of the arm postures, predicted by the proposed objective function, is the most satisfactory, while the optimal value of the coefficient, in the proposed objective function, is determined by golden section search.

Application of Human Posture Recognition Based on the Convolutional Neural Network in Physical Training Guidance.

The application of sports game video analysis in athlete training and competition analysis feedback has attracted extensive attention, but the traditional sports human body posture estimation method has a large error between the athlete's human body posture estimation results and the actual results in the complex environment and the athlete's body parts are blocked. Therefore, this study proposes a convolutional neural network for athlete pose estimation in sports game video. Based on the improved model, multiscale model, and large perception model, a superimposed hourglass network is constructed, and the gradient disappearance problem of the convolutional neural network is solved using intermediate supervision. The experimental results show that the athlete pose estimation model based on the convolutional neural network can improve the accuracy of athlete pose estimation and reduce the negative impact of occlusion environment on athlete pose estimation to a certain extent. In addition, compared with other athletes' standing posture estimation methods, the model has competitive advantages and high accuracy under widely used standard conditions.

Deep Learning-Based Device-Free Localization Scheme for Simultaneous Estimation of Indoor Location and Posture Using FMCW Radars.

Indoor device-free localization (DFL) systems are used in various Internet-of-Things applications based on human behavior recognition. However, the usage of camera-based intuitive DFL approaches is limited in dark environments and disaster situations. Moreover, camera-based DFL schemes exhibit certain privacy issues. Therefore, DFL schemes with radars are increasingly being investigated owing to their efficient functioning in dark environments and their ability to prevent privacy issues. This study proposes a deep learning-based DFL scheme for simultaneous estimation of indoor location and posture using 24-GHz frequency-modulated continuous-wave (FMCW) radars. The proposed scheme uses a parallel 1D convolutional neural network structure with a regression and a classification model for localization and posture estimation, respectively. The two-dimensional location information of the target is estimated for localization, and four different postures, namely standing, sitting, lying, and absence, are estimated simultaneously. We experimentally evaluated the proposed scheme and compared its performance with that of conventional schemes under identical conditions. The results indicate that the average localization error of the proposed scheme is 0.23 m, whereas that of the conventional scheme is approximately 0.65 m. The average posture estimation error of the proposed scheme is approximately 1.7%, whereas that of the conventional correlation, CSP, and SVM schemes are 54.8%, 42%, and 10%, respectively.