Human Shape Research Articles

3D human pose estimation with single image and inertial measurement unit (IMU) sequence

Three-dimensional human pose estimation plays an important role in the field of computer vision, such as in healthcare, sports, activity recognition, motion capture, and augmented reality. However, monocular image or video based methods are sensitive to occlusions, while multi-view methods usually require enormous computation resources. Currently, inertial measurement unit (IMU)-based methods have begun to overcome the occlusion problem and can potentially achieve real-time inference. Yet, they still suffer from insufficient precision and scale drift error over time. In this paper, we propose a novel, efficient framework to fuse a single image with temporal sequence from IMU sensors to estimate human poses and reconstruct human shapes. Our method achieves 46 mm Mean Per Joint Positional Error (MPJPE) on the Total Capture dataset with 30 frames time segment, and surpasses state-of-the-art pure IMU-based methods. Moreover, in comparison with other vision-based methods, the proposed method shows great advantage in reducing computing floating point operations per second (FLOPS) quota while still achieving competitive estimation precision. Our method achieves 74 FPS on an IPhone 12 for offline processing. In addition, our method can easily be generalized for outdoor cases.

Positioning errors of anatomical landmarks identified by fixed vertices in homologous meshes

BackgroundHuman movement analysis is usually achieved by tracking markers attached to anatomical landmarks with photogrammetry. Such marker-based systems have disadvantages that have led to the development of markerless procedures, although their accuracy is not usually comparable to that of manual palpation procedures. New motion acquisition systems, such as 3D temporal scanners, provide homologous meshes that can be exploited for this purpose. Research questionCan fixed vertices of a homologous mesh be used to identify anatomical landmarks with an accuracy equivalent to that of manual palpation? MethodsWe used 3165 human shape scans from the CAESAR dataset, with labelled locations of anatomical landmarks. First, we fitted a template mesh to the scans, and assigned a vertex of that mesh to 53 anatomical landmarks in all subjects. Then we defined a nominal vertex for each landmark, as the more centred vertex out of the set assigned for that landmark. We calculated the errors of the template-fitting and the nominal vertex determination procedures, and analysed their relationship to subject’s sex, height and body mass index, as well as their size compared to manual palpation errors. ResultsThe template-fitting errors were below 5 mm, and the nominal vertex determination errors reached maximum values of 24 mm. Except for the trochanter, those errors were the same order of magnitude or smaller than inter-examiner errors of lower limb landmarks. Errors increased with height and body mass index, and were smaller for men than for women of the same height and body mass index. SignificanceWe defined a set of vertices for 53 anatomical landmarks in a homologous mesh, which yields location errors comparable to those obtained by manual palpation for the majority of landmarks. We also quantified how the subject’s sex and anthropometric features can affect the size of those errors.

Human shape representations are not an emergent property of learning to classify objects.

Humans are particularly sensitive to relationships between parts of objects. It remains unclear why this is. One hypothesis is that relational features are highly diagnostic of object categories and emerge as a result of learning to classify objects. We tested this by analyzing the internal representations of supervised convolutional neural networks (CNNs) trained to classify large sets of objects. We found that CNNs do not show the same sensitivity to relational changes as previously observed for human participants. Furthermore, when we precisely controlled the deformations to objects, human behavior was best predicted by the number of relational changes while CNNs were equally sensitive to all changes. Even changing the statistics of the learning environment by making relations uniquely diagnostic did not make networks more sensitive to relations in general. Our results show that learning to classify objects is not sufficient for the emergence of human shape representations. Instead, these results suggest that humans are selectively sensitive to relational changes because they build representations of distal objects from their retinal images and interpret relational changes as changes to these distal objects. This inferential process makes human shape representations qualitatively different from those of artificial neural networks optimized to perform image classification. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Perception of 3D shape integrates intuitive physics and analysis-by-synthesis.

Many surface cues support three-dimensional shape perception, but humans can sometimes still see shape when these features are missing-such as when an object is covered with a draped cloth. Here we propose a framework for three-dimensional shape perception that explains perception in both typical and atypical cases as analysis-by-synthesis, or inference in a generative model of image formation. The model integrates intuitive physics to explain how shape can be inferred from the deformations it causes to other objects, as in cloth draping. Behavioural and computational studies comparing this account with several alternatives show that it best matches human observers (total n = 174) in both accuracy and response times, and is the only model that correlates significantly with human performance on difficult discriminations. We suggest that bottom-up deep neural network models are not fully adequate accounts of human shape perception, and point to how machine vision systems might achieve more human-like robustness.

Bridging the Gap: Development of frontal crash mode ATD Analogous human body models

Anthropomorphic variation is an important factor in computational studies using Human Body Models (HBMs), particularly regarding how such differences can influence observed kinematics and loading. Currently, a gap exists between Anthropomorphic Test Devices (ATDs) and human body models (HBMs). By necessity, there are differences in constitutive behaviors at a material level, however segment mass distribution and anthropometry differences can make matched simulations of ATDs and HBMs difficult to interpret, which has real-world implications for current or future regulatory applications. In response to this gap, we present Global Human Body Model Consortium (GHBMC) 50th percentile male models (M50-OS) analogous to Hybrid III and THOR 50th percentile male models to serve as intermediaries between the HBMs and ATDs in computational biomechanics studies.Statistical human shape models were sized to each ATD, individually, using measurement metrics from ANSUR I. The M50-OS model surface and body shape surface were then converted to both polygon data (STL) and nodal coordinates. Landmark registration was performed, and the morphed nodes were imported back into the existing M50-OS models to generate two morphed versions analogous to the ATDs. The qualification suites were obtained from each respective ATD manual and run against the baseline and morphed models. The models were also simulated in a frontal NCAP crash pulses (56 km/hr) to assess variation in loading in a regulatory crash environment. 33 simulations were run between all models. The LSTC Hybrid III version 151,214 (open access software, ANSYS, Canonsburg, PA), Adult Shape Parametric Model (open access model, UMTRI, Ann Arbor, MI), GHBMC M50-OS (Elemance, LLC, Winston-Salem, NC), Thor v2.7 (University of Virginia School of Engineering and Applied Science/NHTSA) were used in this study. All simulations were run using LS-Dyna R. 9.3.1 (ANSYS) on an in-house computational cluster.The Male 50th Percentile THOR Analogous Occupant Simplified (M50 THAN-OS) has 364 × 103 elements and weighs 77.1 kg, the Hybrid III Analogous Occupant Simplified (M50 HAN-OS) has 370 × 103 elements and weighs 79.2 kg. The M50 HAN-OS and M50 THAN-OS response data exhibited a closer match to their respective models overall when compared to the baseline M50-OS human. In the vehicle test environment, the airbag and seatbelt peaks between M50 HAN-OS and M50 Hybrid III at nearly the same time.The results indicate that the ATD analogous models exhibit more similar response characteristics to their ATD counterparts after correcting for shape and mass distribution discrepancies. The ATD analogous models therefore represent a novel means to compare anticipated biomechanical loading on an occupant under similar testing conditions that use ATDs. Furthermore, the models represent a potential bridge towards testing modes of interest that ATDs are not well suited or are not validated for, such as low speed maneuvering, or oblique and far-side modes of crash.

Climate Change Predictive of Body Size and Proportionality in Humans

The effects of climate change on plants and animals have been examined across numerous species, yet little evidence has been produced to demonstrate an influence on human evolution. Here the impact of climate change on human body size and scaling is examined over a period of 700,000 years using five independent paleoclimate records. Across 247 Homo specimens, body mass averaged significantly smaller during periods of climatic warming as compared to cooler cycles. Body proportions also changed significantly, appearing more ectomorphic during warmer periods and more endomorphic during periods of cooling across a sample of 87 specimens. The results indicate a relationship between climate change and body size and shape in humans that is driven by natural selection in response to thermoregulatory demands. The findings suggest that body size adaptations in response to climate change occurred early and potentially implicated cultural adaptations in later periods, muting the morphological response to extreme climates. Because morphological variation has been used as a factor for classification within the genus Homo, taxonomic and phylogenic decisions may need to be reconsidered in the context of temporal climate differences.

‘I am not data’: A GAN simulation in tandem with Second Nature

‘I am not data’ has been produced in collaboration with a creative coder from the Netherlands. Over a period of six months around 4600 images were chosen from approximately 20,000 images of LatinX femmes. The images were then fed into a generative adversarial network (GAN) simulation that produced this film. The resulting work seeks to elucidate a process of lumping together foreign bodies as a method of amalgamation that works to create ‘new’ information. It exists in tandem with ‘Second Nature’, a body of work on Mexican femme bodies and algorithmic web searches, from which some images are included in this article. The process behind the GAN used for this film becomes a mere reference to a human shape. Its processes of synthesis work as a metaphor for systems of human classification in which the bodies of foreigners become fuel for monstrous stereotypes. There is a sense of history repeating itself through the ways that these technological tools are used in artificial intelligence (AI) which launches questions regarding equality and progress in pursuit of social justice.

Spatiotemporal Learning Transformer for Video-Based Human Pose Estimation

Multi-frame human pose estimation has long been an appealing and fundamental issue in visual perception. Owing to the frequent rapid motion and pose occlusion in videos, this task is extremely challenging. Current state-of-the-art methods seek to model spatiotemporal features by equally fusing each frame in the local sequence, which weakens the target frame information. In addition, existing approaches usually emphasize more on deep features while ignoring the detailed information implied in the shallow feature maps, resulting in the dropping of crucial features. To address the above problems, we propose an effective framework, namely spatiotemporal learning transformer for video-based human pose estimation (SLT-Pose), which consists of a Personalized Feature Extraction Module (PFEM), Self-feature Refinement Module (SRM), Cross-frame Temporal Learning Module (CTLM) and Disentangled Keypoint Detector (DKD). To be specific, we propose PFEM which extracts and modulates the individual frame features to adapt to the varying human shape, and integrates single-frame features to obtain the spatiotemporal features. We further present SRM to establish global correlation spatial cues on the target frame to attain the refinement feature. Then, a CTLM is designed to search for the information most closely related to the target frame from the spatiotemporal features to intensify the interaction between the target frame and the local sequence, using both the shallow detailed and the deep semantic representations. Finally, we employ DKD to extract the disentangled characteristics of each joint and encode the articulated joint pairs in the human body, promoting the model to reasonably and accurately predict the keypoint heatmaps. Extensive experiments on three huamn motion benchmarks, including PoseTrack2017, PoseTrack2018, and Sub-JHMDB dataset, demonstrate that SLT-Pose plays favorably against state-of-the-art approaches in terms of both objective evaluation and subjective visual performance.

BOSS: Bones, organs and skin shape model

A virtual anatomical model of a patient can be a valuable tool for enhancing clinical tasks such as workflow automation, patient-specific X-ray dose optimization, markerless tracking, positioning, and navigation assistance in image-guided interventions. For these tasks, it is highly desirable that the patient’s surface and internal organs are of high quality for any pose and shape estimate. At present, the majority of statistical shape models (SSMs) are restricted to a small number of organs or bones or do not adequately represent the general population. To address this, we propose a deformable human shape and pose model that combines skin, internal organs, and bones, learned from CT images. By modeling the statistical variations in a pose-normalized space using probabilistic PCA while also preserving joint kinematics, our approach offers a holistic representation of the body that can be beneficial for automation in various medical applications. In an interventional setup, our model could, for example, facilitate automatic system/patient positioning, organ-specific iso-centering, automated collimation or collision prediction. We assessed our model’s performance on a registered dataset, utilizing the unified shape space, and noted an average error of 3.6 mm for bones and 8.8 mm for organs. By utilizing solely skin surface data or patient metadata like height and weight, we find that the overall combined error for bone-organ measurement is 8.68 mm and 8.11 mm, respectively. To further verify our findings, we conducted additional tests on publicly available datasets with multi-part segmentations, which confirmed the effectiveness of our model. In the diverse TotalSegmentator dataset, the errors for bones and organs are observed to be 5.10mm and 8.72mm, respectively. Our work shows that anatomically parameterized statistical shape models can be created accurately and in a computationally efficient manner. The proposed approach enables the construction of shape models that can be directly integrated into to various medical applications.

DeepPortraitDrawing: Generating human body images from freehand sketches

Various methods for generating realistic images of objects and human faces from freehand sketches have been explored. However, generating realistic human body images from sketches is still a challenging problem. It is, first because of the sensitivity to human shapes, second because of the complexity of human images caused by body shape and pose changes, and third because of the domain gap between realistic images and freehand sketches. In this work, we present DeepPortraitDrawing, a deep generative framework for converting roughly drawn sketches to realistic human body images. To encode complicated body shapes under various poses, we take a local-to-global approach. Locally, we employ semantic part auto-encoders to construct part-level shape spaces, which are useful for refining the geometry of an input pre-segmented hand-drawn sketch. Globally, we employ a cascaded spatial transformer network to refine the structure of body parts by adjusting their spatial locations and relative proportions. Finally, we use a style-based generator as the global synthesis network for the sketch-to-image translation task which is modulated by segmentation maps for semantic preservation. Extensive experiments have shown that given roughly sketched human portraits, our method produces more realistic images than the state-of-the-art sketch-to-image synthesis techniques.

3D shape recovery algorithm from image orientations of textured surfaces.

Previous psychophysical studies have demonstrated that the image orientation of textured surfaces guides human 3D shape perception. However, the accuracy of computational 3D shape reconstruction solely from image orientation requires further study. This paper proposes a 3D shape recovery algorithm from the image orientation of a single textured surface image. The evaluation of the proposed algorithm uses computer-generated textured complex 3D surfaces. The depth correlations between the recovered and true surface shapes achieved or exceeded 0.8, which is similar to the accuracy of human shape perception, as shown in a previous psychophysical study, indicating that the image orientations contain adequate information for 3D shape recovery from textured surface images.

Revisiting mandibular symphyseal shape in juvenile early hominins and modern humans using a deformation-based approach.

The juvenile mandible is important in the investigation of ontogenetic and evolutionary changes among early hominins. We revisit the mandibular symphysis in juvenile specimens of Australopithecus africanus and Paranthropus robustus with two main contributions. First, we employ, for the first time, methods of computational anatomy to model complex symphyseal shape differences. Second, we present new fossil evidence from Kromdraai to improve our knowledge of symphyseal morphology. We describe differences between shapes by landmark-free diffeomorphism needed to align them. We assess which features of the mandibular symphysis best discriminate the juvenile symphysis in these fossil species, relative to the intraspecific variation observed among modern humans. Our approach eliminates potential methodological inconsistencies with traditional approaches (i.e., the need for homologous anatomical landmarks, assumption of linearity). By enabling detailed comparisons of complex shapes in juvenile mandibles, our proposed approach offers new perspectives for more detailed comparisons among Australopithecus, Paranthropus and early Homo.

Robotic Information System (RIS): Design of Humanoid Robot’s Head Based on Human Biomechanics

Robotic Information System (RIS) is a new information system architecture that provides data, information processing and services for users in a wide geographical space based on intelligent robots. The front end of the RIS is a humanoid robot head that interacts with users by different means to provide different services. Developing a robot which is similar to a human shape, function and socially accepted by human beings was and is still a challenging subject. Arslan is a new humanoid robot head that has been designed to be close to real human skeletal structure, where its physical structure is designed to be identical to the real human skull, jaw and the nick vertebrae. In order to develop a high-quality and low-cost system, Arslan has a smart mechanism that is controlled by 16 servo motors in order to control its nick, jaw, eyes and facial expression, which have been located according to analysis of human biomechanics. Arslan’s nick mechanism allows one side axial rotation of 41° and flexion/extension of 45°, mandible rotation around the hinge axis of 6 degrees, its eyes can rotate around their axis of motion with Adduction 19°, Abduction 34°, Supraduction (elevation) 0° and Infraduction (depression) 20°. The distance between the upper and the lower eyelid of Arslan can range from 0 up to 17 mm to cover different expressions. Also, it is equipped with devices for motion, vision, audition, smelling, and speech. It also can recognize its head tilting and is capable of maintaining its eyes' direction constantly during head motion

Post-pubertal developmental trajectories of laryngeal shape and size in humans

Laryngeal morphotypes have been hypothesized related to both phonation and to laryngeal pathologies. Morphotypes have not been validated or demonstrated quantitatively and sources of shape and size variation are incompletely understood but are critical for the explanation of behavioral changes (e.g., changes of physical properties of a voice) and for therapeutic approaches to the larynx. This is the first study to take this crucial step and results are likely to have implications for surgeons and speech language pathologists. A stratified human sample was interrogated for phenotypic variation of the vocal organ. First, computed tomography image stacks were used to generate three-dimensional reconstructions of the thyroid cartilage. Then cartilage shapes were quantified using multivariate statistical analysis of high dimensional shape data from margins and surfaces of the thyroid cartilage. The effects of sex, age, body mass index (BMI) and body height on size and shape differences were analyzed. We found that sex, age, BMI and the age–sex interaction showed significant effects on the mixed sex sample. Among males, only age showed a strong effect. The thyroid cartilage increased in overall size, and the angulation between left and right lamina decreased in older males. Age, BMI and the age–height interaction were statistically significant factors within females. The angulation between left and right lamina increased in older females and was smaller in females with greater BMI. A cluster analysis confirmed the strong age effect on larynx shape in males and a complex interaction between the age, BMI and height variables in the female sample. The investigation demonstrated that age and BMI, two risk factors in a range of clinical conditions, are associated with shape and size variation of the human larynx. The effects influence shape differently in female and male larynges. The male–female shape dichotomy is partly size-dependent but predominantly size-independent.

Object-Occluded Human Shape and Pose Estimation With Probabilistic Latent Consistency.

Occlusions between human and objects, especially for the activities of human-object interactions, are very common in practical applications. However, most of the existing approaches for 3D human shape and pose estimation require that human bodies are well captured without occlusions or with minor self-occlusions. In this paper, we focus on the problem of directly estimating the object-occluded human shape and pose from single color images. Our key idea is to utilize a partial UV map to represent an object-occluded human body, and the full 3D human shape estimation is ultimately converted as an image inpainting problem. We propose a novel two-branch network architecture to train an end-to-end regressor via a latent distribution consistency, which also includes a novel visible feature sub-net to extract the human information from object-occluded color images. To supervise the network training, we further build a novel dataset named as 3DOH50K. Several experiments are conducted to reveal the effectiveness of the proposed method. Experimental results demonstrate that the proposed method achieves state-of-the-art compared with previous methods. The dataset and codes are publicly available at https://www.yangangwang.com/papers/ZHANG-OOH-2020-03.html.

Albert Camus’ eksistentsialismi jooni Priit Pärna filmides

Comparison of two works benefits the understanding of both of them. The comparison finds the overlap between the works along with the standpoints that distinguish them. The interpretation of these works is broadened and also more distinctly delimited. The article examines the films of Estonia’s most accomplished animation director Priit Pärn for signs of existentialist thought, based on Albert Camus’s The Myth of Sisyphus. Close textual analysis was used to answer the research question: which existentialist traits appear in Priit Pärn’s films, and how they resemble and differ from Albert Camus’s view of existentialism. The research found numerous parallels between Priit Pärn’s filmography and Camus’s existentialist thought. One of the leitmotifs of Pärn’s films is the prevalence of preposterous totalitarian systems, where the prevailing situation is nearly identical to one in which the absurd man would discover the absurd. Various characters from these worlds have lost their human shape in a way that causes discomforting alienation in the viewer, which is one of the absurd discoveries mentioned by Camus. Though many of Pärn’s characters are similar to the absurd man, only in one of the films are they able to free themselves from the rigid routine conclusively. However, none of the characters recognize the absurdity of the system around them: they either live under its hardship or those who free themselves resemble the absurd man only in their actions, not in their mentality, because they do not act upon a logical discussion. Distinctive to Pärn’s style, the unrestricted approach to reality aligns with Camus’s reluctance about the mindset that treats humans and everything else as if they had a definite place and purpose in the world. Priit Pärn’s films have many similarities with Albert Camus’ existentialism. Although juxtaposing them reveals that while Camus concentrates on a mentality on how to interpret and embrace the absurd, Pärn’s films focus on the absurd and the absurd systems themselves, along with describing the burdens they bring about. Additionally, often in Priit Pärn’s films, the oppressive plight of the characters remains persistent. In contrast, Camus ends his essay in faith that even in a world burdened with absurd one may be happy.

Feature Representation for High‐resolution Clothed Human Reconstruction

AbstractDetailed and accurate feature representation is essential for high‐resolution reconstruction of clothed human. Herein we introduce a unified feature representation for clothed human reconstruction, which can adapt to changeable posture and various clothing details. The whole method can be divided into two parts: the human shape feature representation and the details feature representation. Specifically, we firstly combine the voxel feature learned from semantic voxel with the pixel feature from input image as an implicit representation for human shape. Then, the details feature mixed with the clothed layer feature and the normal feature is used to guide the multi‐layer perceptron to capture geometric surface details. The key difference from existing methods is that we use the clothing semantics to infer clothed layer information, and further restore the layer details with geometric height. We qualitative and quantitative experience results demonstrate that proposed method outperforms existing methods in terms of handling limb swing and clothing details. Our method provides a new solution for clothed human reconstruction with high‐resolution details (style, wrinkles and clothed layers), and has good potential in three‐dimensional virtual try‐on and digital characters.

통계적 신체 외형 데이터베이스를 활용한 실루엣으로부터의 3차원 인체 외형 예측

In this paper, we present a method for estimating full 3D shapes from given 2D silhouette images of human bodies. Because the silhouette only consists of the partial information on the true shape, it is an ill-posed problem. To address the problem, we use the statistical human shape space obtained from the existing large 3D human shape database. The method consists of three steps. First, we extract the boundary pixels and their appropriate normal vectors from the input silhouette images. Then, we initialize the correspondences of each pixel to the vertex of the statistically-deformable 3D human model. Finally, we numerically optimize the parameters of the statistical model to fit best to the given silhouettes. The viability and the robustness of the method is demonstrated with various experiments.

Reconstructing dynamic human shapes from sparse silhouettes via latent space optimization of Parametric shape models

The problem of dynamic 3D reconstruction has gained popularity over the last few years with most approaches relying on data driven learning and optimization methods. However this is quite a challenging task because of the need for tracking different features in both space and time?that too of deformable objects-where such robust tracking may not always be possible. A common way to better ground the problem is by using some forms of regularizations primarily on the shape representations. Over the years, mesh-based linear blend skinning models have been the standard for fitting templates of humans to the observed time series data of human deformation. However, this approach suffers from optimization difficulties arising from maintaining a consistent mesh topology. In this paper, a novel algorithm for reconstructing dynamic human shapes has been proposed, which uses only sparse silhouette information. This is achieved by first creating shape models based on the signed distance neural fields which are subsequently optimized via volumetric differentiable rendering to best match the observed data. Several experiments have been carried out in this work to test the robustness of this method and the results show it to be quite robust, outperforming prior state of the art on dynamic human shape reconstruction by 45%.

A Kist o’ Whistles

A Kist o’ Whistles