3D Avatar Research Articles

3D standard avatar creation of Korean women in their twenties and thirties by body types for apparel industry

The purposes of this study are to classify body types of Korean women in their twenties and thirties for the creation of the 3D avatars and to propose the representative body size of each body type by analyzing the body size of Korean women in their twenties and thirties, to propose a 3D avatar modeling process design that reflects the body shapes of Korean women in their twenties and thirties, and to present standard 3D avatars of each body type of Korean women in their twenties and thirties which are verified with measurement suitability. The 3D anthropometric data of the Korean Anthropometric Survey (6th Size Korea) conducted in 2010 was used in this study. The collected subjects were 410 Korean women in their twenties and thirties. The 3D avatar modeling process using Maya 2013 was proposed to create the representative 3D avatars show superior measurement suitability. This process includes four steps; Analyzing body size measurements, 2D Image plane design, 3D avatar modeling, and 3D avatar evaluation. The 3D avatars created with this process showed the acceptable range of error. The factor analysis was performed on fifty-five body measurements chosen from the measurements of the 6th Size Korea anthropometric survey. Seven factors were extracted. With the seven extracted factors, body shapes of 406 Korean women in their twenties and thirties are classified into four groups by cluster analysis. The classified groups were named Full & Short, Slim & Short, Full & Tall, and Slim & Tall.

Animated 3D human avatars from a single image with GAN-based texture inference

With the development of AR/VR technologies, a reliable and straightforward way to digitize a three-dimensional human body is in high demand. Most existing methods use complex equipment and sophisticated algorithms, but this is impractical for everyday users. In this paper, we propose a pipeline that reconstructs a 3D human shape avatar from a single image. Our approach simultaneously reconstructs the three-dimensional human geometry and whole body texture map with only a single RGB image as input. We first segment the human body parts from the image and then obtain an initial body geometry by fitting the segment to a parametric model. Next, we warp the initial geometry to the final shape by utilizing a silhouette-based dense correspondence. Finally, to infer invisible back texture from a frontal image, we propose a network called InferGAN. Based on human semantic information, we also propose a method to handle partial occlusion by reconstructing the occluded body parts separately. Comprehensive experiments demonstrate that our solution is robust and effective on both public and our own datasets. Our human avatars can be easily rigged and animated using MoCap data. We have developed a mobile application that demonstrates this capability for AR applications.

Development of arabic sign language dictionary using 3D avatar technologies

<span>The arabic sign language (ArSL) is the natural language of the deaf community in Arabic countries. ArSL suffers from a lack of resources such as unified dictionaries and corpora. In this work, a dictionary of Arabic language to ArSL has been constructed as a part of a translation system. The Arabic words are converted into hamburg notation system (HamNoSys) using eSign editor Software. HamNoSys was used to create manual parameters (handshape, hand orientation, hand location, and hand movement), while non-manual parameters (facial expressions, shoulder raising, mouthing gesture, head tilting, and body movement) added by using (mouth, face, and limbs) in the eSign editor software. The sign then converted to the sign gesture markup language (SiGML) file, and later 3D avatar interprets the SiGML file scripts to the animated sign. The constructed dictionary has three thousand signs; therefore, it can be adopted for the translation system in which written text can be transformed into sign language and can be utilized for the education of deaf people. The dictionary will be available as a free resource for researchers. It is hard and time-consuming work, but it is an essential step in machine translation of whole Arabic text to ArSL with 3D animations. </span>

Experimental evidence for yawn contagion in orangutans (Pongo pygmaeus)

Yawning is highly contagious, yet both its proximate mechanism(s) and its ultimate causation remain poorly understood. Scholars have suggested a link between contagious yawning (CY) and sociality due to its appearance in mostly social species. Nevertheless, as findings are inconsistent, CY’s function and evolution remains heavily debated. One way to understand the evolution of CY is by studying it in hominids. Although CY has been found in chimpanzees and bonobos, but is absent in gorillas, data on orangutans are missing despite them being the least social hominid. Orangutans are thus interesting for understanding CY’s phylogeny. Here, we experimentally tested whether orangutans yawn contagiously in response to videos of conspecifics yawning. Furthermore, we investigated whether CY was affected by familiarity with the yawning individual (i.e. a familiar or unfamiliar conspecific and a 3D orangutan avatar). In 700 trials across 8 individuals, we found that orangutans are more likely to yawn in response to yawn videos compared to control videos of conspecifics, but not to yawn videos of the avatar. Interestingly, CY occurred regardless of whether a conspecific was familiar or unfamiliar. We conclude that CY was likely already present in the last common ancestor of humans and great apes, though more converging evidence is needed.

Automatic Feedback For Physiotherapy Exercises Based On PoseNet

This project helps patients who undergo physio- therapy exercises. It can help patients to do right poses and prevent them getting injured during the exercises. The home- based physiotherapy exercise (HPTE) dataset is used. The dataset consists of eight exercises which focus on shoulder, and knee. It is performed by five different actors with different shapes and sex. Our system uses seven exercises with two sides for more variety and better user experience. The main idea of the system is based on detecting the body key points using one of pose estimation techniques. The key points are extracted from different exercises and build our pickle file. Then, those pickle file’s key points are compared to the key points of the patient using dynamic time warping. The system is implemented as a desktop application with a simple user interface and a 3D avatar is used to guide the patients to perform the exercises correctly. The system gives them a score on the exercise and a feedback on which parts of their body have incorrectly posture. Our system offers high accuracy pose estimation and pose comparison with a minimum accuracy of 80%.

3D Visualization of Body Motion in Speed Climbing.

Speed climbing involves an optimization of the velocity of the ascent and the trajectory path during performance. Consequently, any amount of energy spent in the two other directions than vertical, namely the lateral direction and the direction perpendicular to the wall plane, is a potential loss of performance. To assess this principle, we present a study on 3D motion analysis and its 3D visualization for a subject during a speed climbing performance. The fundamentals of geometrical measurement in 3D require to integrate multiple 2D cues, at least two, in order to extract 3D information. First results with two drones following an athlete's ascent show that a 3D velocity profile can be provided from the tracking of a marker on the harness, pointing critical phases in the ascent where the vertical speed is not dominant any more. We further investigate 3D motion of full body using markerless video-based tracking. Our approach is based on a full body 3D avatar model of the climber, represented as a 3D mesh. This model and its deformation are learned in a laboratory studio. The learning needs to be done only once. Result is a manifold embedding of the 3D mesh and its deformations, which can be used afterwards to perform registration onto video of performance of speed climbing. The results of the tracking is an inference of the 3D mesh aligned onto videos of speed climbing performance. From this 3D mesh, we deduce an estimation of the center of mass (COM). We show that this estimation from 3D mesh differs from the usual approximation of the COM as a marker on the harness. In particular, the 3D mesh COM takes into account the whole body movement such as the influence of the limbs which is not detected by a marker on the harness.

4D Scanning - Dynamic View on Body Measurements

Fitting test in movement are an important issue not only in work and sports wear since garment should not restrict the range of motion. Therefore, dynamic anthropometry is a major research topic. Until now, only static position could be captured. Still, it is not known how body geometry changes in dynamic movement. In IGF project “Mobilityrestrictions” photogrammetry scanner “Little Alice” was investigated regarding 4D. It enables serial recording in seconds. The aim of the research project was beside other to identify differences between static and dynamic body measurements. 
 Scanner “Little Alice” has never been utilized for body form analysis. Therefore, a basic research was performed. Several parameters were examined by iterative tests before scan procedure was defined. Three work or sport related movements were defined and compared to standard position: Biceps curl, leg flex and squat. The changes in scan surface were investigated by a three-step analysis: body measurements, cross sections and a 3D analysis. Scan procedure was performed by six test subjects German sizes 50 and 58, age group 25 – 55 years. 
 The results show that photogrammetry can be utilized to investigate body geometry changes due to movement. Body surface deviations have been investigated. Thus, not in all cases there were differences between static and dynamic scans. Yet, body geometry alters. 4D scanning enables comprehensive analysis of body geometry changes due to movement. Body measurement and surface alterations can be visualized and quantified. Scans of motions may be used to validate 3D simulation avatars.

Fusion of Multiple Lidars and Inertial Sensors for the Real-Time Pose Tracking of Human Motion.

Today, enhancement in sensing technology enables the use of multiple sensors to track human motion/activity precisely. Tracking human motion has various applications, such as fitness training, healthcare, rehabilitation, human-computer interaction, virtual reality, and activity recognition. Therefore, the fusion of multiple sensors creates new opportunities to develop and improve an existing system. This paper proposes a pose-tracking system by fusing multiple three-dimensional (3D) light detection and ranging (lidar) and inertial measurement unit (IMU) sensors. The initial step estimates the human skeletal parameters proportional to the target user’s height by extracting the point cloud from lidars. Next, IMUs are used to capture the orientation of each skeleton segment and estimate the respective joint positions. In the final stage, the displacement drift in the position is corrected by fusing the data from both sensors in real time. The installation setup is relatively effortless, flexible for sensor locations, and delivers results comparable to the state-of-the-art pose-tracking system. We evaluated the proposed system regarding its accuracy in the user’s height estimation, full-body joint position estimation, and reconstruction of the 3D avatar. We used a publicly available dataset for the experimental evaluation wherever possible. The results reveal that the accuracy of height and the position estimation is well within an acceptable range of ±3–5 cm. The reconstruction of the motion based on the publicly available dataset and our data is precise and realistic.

Motion-Sphere: Visual Representation of the Subtle Motion of Human Joints

Understanding and differentiating subtle human motion over time as sequential data is challenging. We propose Motion-sphere, which is a novel trajectory-based visualization technique, to represent human motion on a unit sphere. Motion-sphere adopts a two-fold approach for human motion visualization, namely a three-dimensional (3D) avatar to reconstruct the target motion and an interactive 3D unit sphere, that enables users to perceive subtle human motion as swing trajectories and color-coded miniature 3D models for twist. This also allows for the simultaneous visual comparison of two motions. Therefore, the technique is applicable in a wide range of applications, including rehabilitation, choreography, and physical fitness training. The current work validates the effectiveness of the proposed work with a user study in comparison with existing motion visualization methods. Our study’s findings show that Motion-sphere is informative in terms of quantifying the swing and twist movements. The Motion-sphere is validated in threefold ways: validation of motion reconstruction on the avatar, accuracy of swing, twist, and speed visualization, and the usability and learnability of the Motion-sphere. Multiple range of motions from an online open database are selectively chosen, such that all joint segments are covered. In all fronts, Motion-sphere fares well. Visualization on the 3D unit sphere and the reconstructed 3D avatar make it intuitive to understand the nature of human motion.

The eyes have it

Interacting with people across large distances is important for remote work, interpersonal relationships, and entertainment. While such face-to-face interactions can be achieved using 2D video conferencing or, more recently, virtual reality (VR), telepresence systems currently distort the communication of eye contact and social gaze signals. Although methods have been proposed to redirect gaze in 2D teleconferencing situations to enable eye contact, 2D video conferencing lacks the 3D immersion of real life. To address these problems, we develop a system for face-to-face interaction in VR that focuses on reproducing photorealistic gaze and eye contact. To do this, we create a 3D virtual avatar model that can be animated by cameras mounted on a VR headset to accurately track and reproduce human gaze in VR. Our primary contributions in this work are a jointly-learnable 3D face and eyeball model that better represents gaze direction and upper facial expressions, a method for disentangling the gaze of the left and right eyes from each other and the rest of the face allowing the model to represent entirely unseen combinations of gaze and expression, and a gaze-aware model for precise animation from headset-mounted cameras. Our quantitative experiments show that our method results in higher reconstruction quality, and qualitative results show our method gives a greatly improved sense of presence for VR avatars.

Point-cloud avatars to improve spatial communication in immersive collaborative virtual environments

Collaborative virtual environments allow remote users to work together in a shared 3D space. To take advantage of the possibilities offered by such systems, their design must allow the users to interact and communicate efficiently. One open question in this field concerns the avatar fidelity of remote partners. This can impact communication between the remote users, more particularly when performing collaborative spatial tasks. In this paper, we present an experimental study comparing the effects of two partner’s avatars on collaboration during spatial tasks. The first avatar was based on a 2.5D streamed point-cloud and the second avatar was based on a 3D preconstructed avatar replicating the remote user movements. These avatars differ in their fidelity levels described through two components: visual and kinematic fidelity. The collaborative performance was evaluated through the efficacy of completing two spatial communication tasks, a pointing task and spatial guidance task. The results indicate that the streamed point-cloud avatar permitted a significant improvement of the collaborative performance for both tasks. The subjective evaluation suggests that these differences in performance can mainly be attributed to the higher kinematic fidelity of this representation as compared with the 3D preconstructed avatar representation. We conclude that, when designing spatial collaborative virtual environments, it is important to reach a high kinematic fidelity of the partner’s representation while a moderate visual fidelity of this representation can suffice.

The spatial effect of fearful faces in the autonomic response.

Peripersonal space (PPS) corresponds to the space around the body and it is defined by the location in space where multimodal inputs from bodily and external stimuli are integrated. Its extent varies according to the characteristics of external stimuli, e.g., the salience of an emotional facial expression. In the present study, we investigated the psycho-physiological correlates of the extension phenomenon. Specifically, we investigated whether an approaching human face showing either an emotionally negative (fearful) or positive (joyful) facial expression would differentially modulate PPS representation, compared to the same face with a neutral expression. To this aim, we continuously recorded the skin conductance response (SCR) of 27 healthy participants while they watched approaching 3D avatar faces showing fearful, joyful or neutral expressions, and then pressed a button to respond to tactile stimuli delivered on their cheeks at three possible delays (visuo-tactile trials). The results revealed that the SCR to fearful faces, but not joyful or neutral faces, was modulated by the apparent distance from the participant's body. SCR increased from very far space to far and then to near space. We propose that the proximity of the fearful face provided a cue to the presence of a threat in the environment and elicited a robust and urgent organization of defensive responses. In contrast, there would be no need to organize defensive responses to joyful or neutral faces and, as a consequence, no SCR differences were found across spatial positions. These results confirm the defensive function of PPS.

Enhancing Data-Driven Algorithms for Human Pose Estimation and Action Recognition Through Simulation

Recognizing human actions, reliably inferring their meaning and being able to potentially exchange mutual social information are core challenges for autonomous systems when they directly share the same space with humans. Intelligent transport systems in particular face this challenge, as interactions with people are often required. The development and testing of technical perception solutions is done mostly on standard vision benchmark datasets for which manual labelling of sensory ground truth has been a tedious but necessary task. Furthermore, rarely occurring human activities are underrepresented in these datasets, leading to algorithms not recognizing such activities. For this purpose, we introduce a modular simulation framework, which offers to train and validate algorithms on various human-centred scenarios. We describe the usage of simulation data to train a state-of-the-art human pose estimation algorithm to recognize unusual human activities in urban areas. Since the recognition of human actions can be an important component of intelligent transport systems, we investigated how simulations can be applied for his purpose. Laboratory experiments show that we can train a recurrent neural network with only simulated data based on motion capture data and 3D avatars, which achieves an almost perfect performance in the classification of those human actions on real data.

An investigation of the ability of wearable technology to aid physiotherapists in assessing jump-landing movement compared to video recording

Functional movement assessment is essential in screening individuals for risk of injury and planning physiotherapy. The Landing error scoring system (LESS) is a valid and reliable paper-based tool used in assessing the risk of knee injury, which requires evaluating multiple joints across two planes using two video cameras, whilst participants land from a jump. The LESS involves of 17 items concern detecting any faulty movements (errors), then counting these errors as an overall score of the LESS. This is seldom used in physiotherapy clinics and sport fields because of the space required to place cameras, lack of obtaining related clinical movement outcomes objectively, and the privacy of the subjects. Wearable sensors which quantify accurate clinical movement outcomes and generate a 3D avatar recording could be a potential tool for movement analysis in physiotherapy. This study aimed to compare avatar to video recording in assessing jump-landing movements.

Development of shirt pattern for wheelchair tennis players through 3D avatars

This study is to develop the wheelchair tennis pattern to improve comfort by reducing garment strain in a CAD program by analyzing seven body movements related to garment strain during play. The bodice was subdivided into surface zones in the present study to efficiently develop 2D patterns that can later be transferred into 3D virtual garments and draped onto 3D avatars. The 2D patterns were developed based on morphological analysis that provided different possible solutions to enhance comfort by releasing garment strain at each surface zone. The structural lines and wearing ease grade points were utilized to adjust a tennis shirt block pattern when developing this 2D pattern at morphological analysis step. In this study, 3D body scanner was used to create 3D avatars with postures based on the determined seven body movements. The drafted pattern was used to drape the virtual garments on the each movement of 3D avatars in the CAD program, and then the stain areas of the pattern design were identified using the tension map. In all of the movements, the majority of garment strain appeared on the shoulder surface and around the neck within the surface zone of bodice shoulder angle. The majority of other strain found was on the shoulder blade as the bodice bust zone during movements in the forehand swing, backhand swing, serve toss, and serve follow through postures. Based on these strain results obtained from 3D scanner, the first pattern was developed to release strains in shoulder as seen by the significant decrease in strain during both forehand movements. The second pattern was constructed to release other maximized strain on the back surface zone of the shoulder blade. Based on the results of this study, it should be noted that the shoulder and the shoulder blade areas are important during development of patterns in terms of decreasing strain for comfort during wheelchair tennis play movements.

Sign Language Generation System Based on Indian Sign Language Grammar

Sign Language (SL), also known as gesture-based language, is used by people with hearing loss to convey their messages. SL interpreters are required for people who do not have the knowledge of SL, but interpreters are not readily available. Thus, a machine-based translation system is required to translate the text into SL. In this article, a system is implemented for translating English text into Indian Sign Language (ISL). It acts as a tool for human-computer interaction and eliminates the need for an ISL human interpreter for communicating with people who have hearing loss. The system features a rich corpus of English words and commonly used sentences. It consists of components such as an ISL parser, the Hamburg Notation System, the Signing Gesture Mark-up Language, and 3D avatar animation for generating SL according to ISL grammar. The proposed system has been tested rigorously by SL users. The results proved that the proposed system is highly efficient and achieves an average score of accuracy (i.e., 4.2 for English words and 3.8 for sentences on a scale from 1 to 5). The performance of proposed system has also been evaluated using the BiLingual Evaluation Understudy score, which results in 0.95 accuracy. The proposed system and mobile application together has the potential to bring individuals with hearing loss and their entourage together.

성인여성 버추얼 패션 3D 아바타와 Size Korea 인체형상의 형태 차이 비교

성인여성 버추얼 패션 3D 아바타와 Size Korea 인체형상의 형태 차이 비교

Augmented Virtual Teleportation for High-Fidelity Telecollaboration.

Telecollaboration involves the teleportation of a remote collaborator to another real-world environment where their partner is located. The fidelity of the environment plays an important role for allowing corresponding spatial references in remote collaboration. We present a novel asymmetric platform, Augmented Virtual Teleportation (AVT), which provides high-fidelity telepresence of a remote VR user (VR-Traveler) into a real-world collaboration space to interact with a local AR user (AR-Host). AVT uses a 360° video camera (360-camera) that captures and live-streams the omni-directional scenes over a network. The remote VR-Traveler watching the video in a VR headset experiences live presence and co-presence in the real-world collaboration space. The VR-Traveler's movements are captured and transmitted to a 3D avatar overlaid onto the 360-camera which can be seen in the AR-Host's display. The visual and audio cues for each collaborator are synchronized in the Mixed Reality Collaboration space (MRC-space), where they can interactively edit virtual objects and collaborate in the real environment using the real objects as a reference. High fidelity, real-time rendering of virtual objects and seamless blending into the real scene allows for unique mixed reality use-case scenarios. Our working prototype has been tested with a user study to evaluate spatial presence, co-presence, and user satisfaction during telecollaboration. Possible applications of AVT are identified and proposed to guide future usage.

The use of a 3D avatar to determine the association between actual and perceived body mass index

Introduction: Literature surrounding body image, body composition and Body Mass Index (BMI) have shown that when participant use visual impressions, they can often lead to a false sense of weight status. Therefore, the main objectives of this investigation was to determine participants BMI and to establish the correlation between actual and perceived BMI. Method: n = 32 female participants ( ± s; body mass = 70.1 ± 13.6 kg, stretched stature = 172.4 ± 8.1 cm) were recruited. A computer generated (Unity Player) 3D Avatar rotated 3600 and permitted a visual slide from an underweight to average to obese continuum. Stretched stature (m) and body mass (kg) was taken and values used to calculate BMI (kg/m2). P value was set at (P<0.001) and a Paired t-Test was used to test for the difference and Pearson's Correlation Coefficient was used to test for the strength of the association between the actual and perceived BMI. Results: Perceived BMI ranged from 16.5 – 32.5 ( 23.5± 4.1), whereas the actual BMI ranged from 17.7 – 31.3 ( 24.3± 3.7). A Paired t-test and Pearson’s Correlation Coefficient found values ranged from -5.3 and 8.6 ( - 0.2± 2.5) and a value of t = 0.81 and r = 0.68 suggesting a significant difference between actual and perceived BMI (P < 0.001). Discussion: This investigation reports that perceived BMI was higher than the participants actual BMI and that the use of visual impressions led to a false sense of weight status. Recommendations: Further research is necessary to investigate the reasons behind these perceived versus actual differences and the creation of a 3D Avatar for male participants and younger populations would be beneficial.

Translator System for Peruvian Sign Language Texts through 3D Virtual Assistant

The population with hearing impairment in Peru is a community that doesn’t receive the necessary support from the government, nor does it have the necessary resources for the inclusion of its members as active persons in this society. Few initiatives have been launched to achieve this goal and for this reason we will create a resource that will give the deaf community to have greater access to textual information of the listener-speaking community. Our goal is to build a tool that translates general texts and also academic content such as encyclopedias into Peruvian sign language which will be represented by a 3D avatar. This translation will feature different lexical-syntactic processing modules as well as the disambiguation of terms using a small lexicon, similar to Wordnet synsets. The project is developed in collaboration with the deaf community.