3D Shape Model Research Articles

Multi-modal gesture recognition with voting-based dynamic time warping

Gesture recognition has remained a challenging problem in the fields of human robot interaction. With the development of depth sensors such as Kinect, different modalities become available for gesture recognition while its advantages have not been fully exploited. One of the critical issues for multi-modal gesture recognition is how to fuse features from different modalities. In this article, we present a unified framework for multi-modal gesture recognition based on dynamic time warping. The 3D implicit shape model is applied to characterize the space-time structure of the local features extracted from different modalities. And then, all votes from the local features are incorporated into a common probability space which is then used for building the distance matrix. Meanwhile, an upper-bounding method UB_Pro is proposed to speed up dynamic time warping. The proposed approach is evaluated on the challenging ChaLearn Isolated Gesture Dataset, showing comparable performance in comparison to the state-of-the-art approaches for multi-modal gesture recognition problem.

Weakly-Supervised Learning of Category-Specific 3D Object Shapes.

Category-specific 3D object shape models have greatly boosted the recent advances in object detection, recognition and segmentation. However, even the most advanced approach for learning 3D object shapes still requires heavy manual annotations on large-scale 2D images. Such annotations include object categories, object keypoints, and figure-ground segmentation for the instances in each image. In particular, annotating figure-ground segmentation is unbearably labor-intensive and time-consuming. To address this problem, this paper devotes to learn category-specific 3D shape models under weak supervision, where only object categories and keypoints are required to be manually annotated on the training 2D images. By exploring the underlying relationship between two tasks: object segmentation and category-specific 3D shape reconstruction, we propose a novel weakly-supervised learning framework to jointly address these two tasks and combine them to boost the final performance of the learned 3D shape models. Moreover, learning without using figure-ground segmentation leads to ambiguous solutions. To this end, we develop the confidence weighting schemes in the viewpoint estimation and 3D shape learning procedure. These schemes effectively reduce the confusion caused by the noisy data and thus increase the chances for recovering more reliable 3D object shapes. Comprehensive experiments on the challenging PASCAL VOC benchmark show that our framework achieves comparable performance with the state-of-the-art methods that use expensive manual segmentation-level annotations. In addition, our experiments also demonstrate that our 3D shape models improve object segmentation performance.

3D Shape Modeling and Analysis of Retinal Microvasculature in OCT-Angiography Images.

3D optical coherence tomography angiography (OCT-A) is a novel and non-invasive imaging modality for analyzing retinal diseases. The studies of microvasculature in 2D en face projection images have been widely implemented, but comprehensive 3D analysis of OCT-A images with rich depth-resolved microvascular information is rarely considered. In this paper, we propose a robust, effective, and automatic 3D shape modeling framework to provide a high-quality 3D vessel representation and to preserve valuable 3D geometric and topological information for vessel analysis. Effective vessel enhancement and extraction steps by means of curvelet denoising and optimally oriented flux (OOF) filtering are first designed to produce 3D microvascular networks. Afterwards, a novel 3D data representation of OCT-A microvasculature is reconstructed via advanced mesh reconstruction techniques. Based on the 3D surfaces, shape analysis is established to extract novel shape-based microvascular area distortion via the Laplace-Beltrami eigen-projection. The extracted feature is integrated into a graph-cut segmentation system to categorize large vessels and small capillaries for more precise shape analysis. The proposed framework is validated on a dedicated repeated scan dataset including 260 volume images and shows high repeatability. Statistical analysis using the surface area biomarker is performed on small capillaries to avoid the effect of tailing artifact from large vessels. It shows significant differences ( ) between DR stages on 100 subjects in a OCTA-DR dataset. The proposed shape modeling and analysis framework opens the possibility for further investigating OCT-A microvasculature in a new perspective.

Rosetta/OSIRIS observations of the 67P nucleus during the April 2016 flyby: high-resolution spectrophotometry

Context.From August 2014 to September 2016, the Rosetta spacecraft followed comet 67P/Churyumov–Gerasimenko along its orbit. After the comet passed perihelion, Rosetta performed a flyby manoeuvre over the Imhotep–Khepry transition in April 2016. The OSIRIS/Narrow-Angle-Camera (NAC) acquired 112 observations with mainly three broadband filters (centered at 480, 649, and 743 nm) at a resolution of up to 0.53 m/px and for phase angles between 0.095° and 62°.Aims.We have investigated the morphological and spectrophotometrical properties of this area using the OSIRIS/NAC high-resolution observations.Methods.We assembled the observations into coregistered color cubes. Using a 3D shape model, we produced the illumination conditions and georeference for each observation. We mapped the observations of the transition to investigate its geomorphology. Observations were photometrically corrected using the Lommel–Seeliger disk law. Spectrophotometric analyses were performed on the coregistered color cubes. These data were used to estimate the local phase reddening.Results.The Imhotep–Khepry transition hosts numerous and varied types of terrains and features. We observe an association between a feature’s nature, its reflectance, and its spectral slopes. Fine material deposits exhibit an average reflectance and spectral slope, while terrains with diamictons, consolidated material, degraded outcrops, or features such as somber boulders present a lower-than-average reflectance and higher-than-average spectral slope. Bright surfaces present here a spectral behavior consistent with terrains enriched in water-ice. We find a phase-reddening slope of 0.064 ± 0.001%/100 nm/° at 2.7 au outbound, similar to the one obtained at 2.3 au inbound during the February 2015 flyby.Conclusions.Identified as the source region of multiple jets and a host of water-ice material, the Imhotep–Khepry transition appeared in April 2016, close to the frost line, to further harbor several potential locations with exposed water-ice material among its numerous different morphological terrain units.

Surface evolution of the Anhur region on comet 67P/Churyumov-Gerasimenko from high-resolution OSIRIS images

Context. The southern hemisphere of comet 67P/Churyumov-Gerasimenko (67P) became observable by the Rosetta mission in March 2015, a few months before cometary southern vernal equinox. The Anhur region in the southern part of the comet’s larger lobe was found to be highly eroded, enriched in volatiles, and highly active. Aims. We analyze high-resolution images of the Anhur region pre- and post-perihelion acquired by the OSIRIS imaging system on board the Rosetta mission. The Narrow Angle Camera is particularly useful for studying the evolution in Anhur in terms of morphological changes and color variations. Methods. Radiance factor images processed by the OSIRIS pipeline were coregistered, reprojected onto the 3D shape model of the comet, and corrected for the illumination conditions. Results. We find a number of morphological changes in the Anhur region that are related to formation of new scarps; removal of dust coatings; localized resurfacing in some areas, including boulders displacements; and vanishing structures, which implies localized mass loss that we estimate to be higher than 50 million kg. The strongest changes took place in and nearby the Anhur canyon-like structure, where significant dust cover was removed, an entire structure vanished, and many boulders were rearranged. All such changes are potentially associated with one of the most intense outbursts registered by Rosetta during its observations, which occurred one day before perihelion passage. Moreover, in the niche at the foot of a new observed scarp, we also see evidence of water ice exposure that persisted for at least six months. The abundance of water ice, evaluated from a linear mixing model, is relatively high (>20%). Our results confirm that the Anhur region is volatile-rich and probably is the area on 67P with the most pristine exposures near perihelion.

Geometric Modeling and 3D Printing Using Recursively Generated Point Cloud

Previous studies have reported that a recursive process called the point cloud creation algorithm (PCA) that generates a point cloud is useful for reverse engineering a planner shape. This study elucidates the characteristics of the parameters used in the recursive process as well as its ability in geometric modeling and 3D printing of 3D shapes. In the recursive process, three constants (center point, initial distance, and initial angle) and two variables (instantaneous distance and instantaneous rotational angle) are employed. The shape-modeling characteristics of the constants and variables are elucidated using some commonly used shapes (straight-line, circle, ellipses, spiral, astroid, S-shape, and leaf-shape). In addition, the shape-modeling capability of the recursive process as a whole is quantified using two parameters called the radius of curvature and aesthetic value. Moreover, an illustrative example that shows the efficacy of the recursive process in virtual and physical prototyping of a relatively complex 3D object is presented. The results show that reverse engineering performed by the recursive-process-created point cloud is free from computational complexity compared to reverse engineering performed by the 3D-scanner-created point cloud. As such, the outcomes of this study enrich the field of reverse engineering.

Generating Point Cloud from Measurements and Shapes Based on Convolutional Neural Network: An Application for Building 3D Human Model.

It has been widely known that 3D shape models are comprehensively parameterized using point cloud and meshes. The point cloud particularly is much simpler to handle compared with meshes, and it also contains the shape information of a 3D model. In this paper, we would like to introduce our new method to generating the 3D point cloud from a set of crucial measurements and shapes of importance positions. In order to find the correspondence between shapes and measurements, we introduced a method of representing 3D data called slice structure. A Neural Networks-based hierarchical learning model is presented to be compatible with the data representation. Primary slices are generated by matching the measurements set before the whole point cloud tuned by Convolutional Neural Network. We conducted the experiment on a 3D human dataset which contains 1706 examples. Our results demonstrate the effectiveness of the proposed framework with the average error 7.72% and fine visualization. This study indicates that paying more attention to local features is worthwhile when dealing with 3D shapes.

Interview

Ouk Choi Professor Ouk Choi from Incheon National University, Korea, talks to Electronics Letters about the paper ‘Robust alternating optimization for extrinsic calibration of RGB-D cameras’, page 992. My research field is computer vision, and I am currently interested in the acquisition, processing, and understanding of 3D data. During my Ph.D. studies, my research topic was to solve highly ambiguous correspondence problems such as matching windows in image pairs of buildings. After receiving my Ph.D. degree, I joined the Samsung Advanced Institute of Technology, where I conducted research projects on time-of-flight cameras. I was intrigued by the fact that time-of-flight cameras can acquire 3D data without solving difficult correspondence problems. Since then, my research has been directed toward 3D computer vision. Nowadays, I am building systems that can acquire multi-view depth images of subjects, so that the acquired datasets can be used for machine learning and other research purposes. Early time-of-flight cameras had many problems such as depth noise, low resolution, and phase wrapping. Many of these problems seem to have been solved in commercially available RGB-D cameras such as the Kinect v2. In addition, they are factory calibrated, so that we can access the intrinsic parameters without calibration. However, if we want to use multiple RGB-D cameras with their 3D measurements represented in the same reference coordinate system, finding the extrinsic parameters such as the rotation and translation between the cameras needs to be done manually. In my Letter, we describe a robust and efficient bundle adjustment algorithm that is indispensable in realising a fully automatic, easy-to-use, and online extrinsic calibration method. In a previous work referenced in my Letter, Su et al. proposed an efficient bundle adjustment algorithm, which alternates between solving for better extrinsic parameters and solving for better calibration target locations. In the presence of outliers, the algorithm tends to fail but this problem was not addressed in their work. On the other hand, in our previous work, Kwon et al. proposed a robust bundle adjustment algorithm, which works in the presence of outliers. However, the algorithm is highly inefficient. In my Letter, my colleagues and I have reported a bundle adjustment algorithm that is as efficient as Su et al.’s algorithm and as robust as Kwon et al.’s algorithm. The most attractive point of our work is that we can perform online bundle adjustment with the proposed algorithm. Although the execution time is not real-time with a single processor, the computational complexity of the algorithm is linear with the number of RGB-D cameras and the number of calibration images. In addition, the algorithm is highly parallelisable, which means that it can be implemented to work in real-time with sufficient computing resources. This will enable online extrinsic calibration of multiple RGB-D cameras. Once we compute an initial solution, we can add images of the spherical calibration target and apply our bundle adjustment algorithm frame-by-frame until a certain level of accuracy is attained. We plan to capture 3D data of human subjects as such data can be used to build a learning-based generative model of 3D human shape and pose as well as establish a parametric generative model. In addition, such a multi-view capture system can be used for analysing the interaction between a person and an object or for analysing social behaviour among people. The collected data and developed algorithms will be used to realise a virtual agent who seems to act human. In the future, it will be possible to generate a movie with only a scenario, using the virtual actors and their human-like behaviours. Traditionally, 3D computer vision was a combination of theories on multiple-view geometry and optimisation-based parameter estimation algorithms. Many research projects have contributed to building 3D models of our surrounding world, and it is now easy to access the online 3D maps provided by Google or Apple. Optimisation-based estimation algorithms require good initial solutions based on theories and feature correspondence across multiple views. Recently, machine learning algorithms such as random forests and deep neural networks are directly applied to raw data, successfully replacing the expert-knowledge-based methods for finding initial solutions. In addition, the research interest is being shifted from static scenes to dynamic objects. Deep learning-based methods are now applied to reconstructing and analysing human performance capture, from which we can obtain a generative model, which can change its shape, pose, and facial expression. As generated models get more accurate and realistic, we will be able to see virtual humanoids in the 3D maps of our surrounding world. I guess that it will take no more than ten years to see such human-like dynamic agents.

18F]‐Sodium Fluoride PET/MR Imaging for Bone–Cartilage Interactions in Hip Osteoarthritis: A Feasibility Study

ABSTRACTThis study characterized the distribution of [18F]‐sodium fluoride (NaF) uptake and blood flow in the femur and acetabulum in hip osteoarthritis (OA) patients to find associations between bone remodeling and cartilage composition in the presence of morphological abnormalities using simultaneous positron emission tomography and magnetic resonance imaging (PET/MR), quantitative magnetic resonance imaging (MRI) and femur shape modeling. Ten patients underwent a [18F]‐NaF PET/MR dynamic scan of the hip simultaneously with: (i) fast spin‐echo CUBE for morphology grading and (ii) T 1ρ/T 2 magnetization‐prepared angle‐modulated partitioned k‐space spoiled gradient echo snapshots for cartilage, bone segmentation, bone shape modeling, and T 1ρ/T 2 quantification. The standardized uptake values (SUVs) and Patlak kinetic parameter (K pat) were calculated for each patient as PET outcomes, using an automated post‐processing pipeline. Shape modeling was performed to extract the variations in bone shapes in the patients. Pearson's correlation coefficients were used to study the associations between bone shapes, PET outcomes, and patient reported pain. Direct associations between quantitative MR and PET evidence of bone remodeling were established in the acetabulum and femur. Associations of shaft thickness with SUV in the femur (p = 0.07) and K pat in the acetabulum (p = 0.02), cam deformity with acetabular score (p = 0.09), osteophytic growth on the femur head with K pat (p = 0.01) were observed. Pain had increased correlations with SUV in the acetabulum (p = 0.14) and femur (p = 0.09) when shaft thickness was accounted for. This study demonstrated the ability of [18F]‐NaF PET‐MRI, 3D shape modeling, and quantitative MRI to investigate cartilage‐bone interactions and bone shape features in hip OA, providing potential investigative tools to diagnose OA. © 2019 The Authors. Journal of Orthopaedic Research ® published by Wiley Periodicals, Inc. on behalf of Orthopaedic Research Society J Orthop Res 37:2671–2680, 2019

Computer-assisted intra-operative verification of surgical outcome for the treatment of syndesmotic injuries through contralateral side comparison.

Fracture reduction and fixation of syndesmotic injuries is a common procedure in trauma surgery. An intra-operative evaluation of the surgical outcome is challenging due to high inter-individual anatomical variation. A comparison to the contralateral uninjured ankle would be highly beneficial but would also incur additional radiation and time consumption. In this work, we pioneer automatic contralateral side comparison while avoiding an additional 3D scan. We reconstruct an accurate 3D surface of the uninjured ankle joint from three low-dose 2D fluoroscopic projections. Through CNN complemented 3D shape model segmentation, we create a reference model of the injured ankle while addressing the issues of metal artifacts and initialization. Following 2D-3D multiple bone reconstruction, a final reference contour can be created and matched to the uninjured ankle for contralateral side comparison without any user interaction. The accuracy and robustness of individual workflow steps were assessed using 81 C-arm datasets, with 2D and 3D images available for injured and uninjured ankles. Furthermore, the entire workflow was tested on eleven clinical cases. These experiments showed an overall average Hausdorff distance of [Formula: see text]mm measured at clinical evaluation level. Reference contours of the contralateral side reconstructed from three projection images can assist surgeons in optimizing reduction results, reducing the duration of radiation exposure and potentially improving postoperative outcomes in the long term.

S12-09 SESSION 12: FACIOCRANIOSYNOSTOSIS – PART II A MACHINE LEARNING APPROACH FOR OUTCOME PREDICTION OF MIDFACIAL BIPARTITION DISTRACTION IN APERT PATIENTS

Introduction: Apert syndrome is a craniofacial syndrome characterized by deformities including hypertelorism with a negative canthal axis, exorbitism due to shallow orbits and maxillary hypoplasia resulting in a biconcave face in both midsagittal and axial planes. These abnormalities can be corrected surgically by a procedure such as midfacial bipartition distraction. Overall, surgical planning is done by the surgeon’s experience and expertise. Even though these operations seem successful based on this approach, little research has been done on the predicted outcomes of an operation based on statistical methods. This study uses various computer vision and machine learning tools to show a possible outcome of the operation. Methods: This study included pre-, and postoperative CT scans of Apert patients who had midfacial bipartition distraction. Each CT scan was cleaned and manually annotated using a set of 68 3-D sparse annotations. Then, it was aligned to the mean mesh of the Syndromic 3D Morphable Model, which is a powerful statistical model of 3D facial shape by means of Procrustes Analysis. Finally, the mesh was brought into dense correspondence to the mean face by exploiting Gaussian Processes and Non-Rigid iterative closest point (ICP) algorithms. At the end of this process, the mesh could be represented by a set of shape parameters. Results: CT scans of 17 Apert patients could be included. Every mesh was generated using only the shape parameters because every mesh in our training set had been registered. Thus, in order to predict an outcome of the operation, we can calculate a projection matrix from the pre-operative shape parameters to the post-operative parameters. In order to check the generalization ability of our model, we used a cross-validation scheme. Due to the small size of our dataset, we used the Leave-One-Out scheme. This means that we used 16 couples of pre- and postoperative CT scans to train our model and test it in the left-out sample. For the prediction, a linear regression to the shape parameters of the datasets was applied. A successful model for the prediction on the surgical outcome for midfacial bipartition distraction in Apert patients was created. Conclusion: We present a novel method for the prediction of the outcome of Midfacial bipartition distraction in Apert patients. Although the available dataset was small, we show that our model works. With this model applications such as treatment simulation could be created.

Retrieval of the Fluid Love Number k2 in Exoplanetary Transit Curves

Abstract We are witness to a great and increasing interest in internal structure, composition, and evolution of exoplanets. However, direct measurements of exoplanetary mass and radius cannot be uniquely interpreted in terms of interior structure, justifying the need for an additional observable. The second degree fluid Love number, k 2, is proportional to the concentration of mass toward the body’s center, hence providing valuable additional information about the internal structure. When hydrostatic equilibrium is assumed for the planetary interior, k 2 is a direct function of the planetary shape. Previous attempts were made to link the observed tidally and rotationally induced planetary oblateness in photometric light curves to k 2 using ellipsoidal shape models. Here, we construct an analytical 3D shape model function of the true planetary mean radius that properly accounts for tidal and rotational deformations. Measuring the true planetary mean radius is critical when one wishes to compare the measured k 2 to interior theoretical expectations. We illustrate the feasibility of our method and show, by applying a Differential Evolution Markov Chain to synthetic data of WASP-121b, that a precision ≤65 ppm/ is required to reliably retrieve k 2 with present understanding of stellar limb darkening (LD), therefore improving recent results based on ellipsoidal shape models. Any improvement on stellar LD would increase such performance.

Typical Shape Differences in the Subtalar Joint Bones Between Subjects with Chronic Ankle Instability and Controls.

ABSTRACTBone shapes, particularly those defining the subtalar joint (STJ), have not received much attention yet as a risk factor for developing chronic ankle instability (CAI) after sustaining a lateral ankle sprain (LAS). This study aimed to compare three‐dimensional (3D) shape variations in the STJ bones within individuals with CAI and healthy controls. 3D statistical shape models (SSMs) of the STJ bones were built to describe the bone shape variations observed within a population consisting of 26 individuals with unilateral CAI and 26 healthy controls. Using the SSMs and analysis of covariance test, age‐ and gender‐adjusted shape variations in the bones were compared within individuals with CAI and healthy controls. The mean age of the CAI patients (14 males and 12 females) and healthy controls (12 males and 14 females) was 29 (standard deviation [SD] = 11) and 36 years (SD = 11), respectively. Tali and calcanei did not significantly vary between ipsilateral CAI and their contralateral ankle. Two shape modes, one for the talus (p = 0.015, variations in the curvature of the talar lateral process and the inclination angle of the talar neck relative to the body) and one for the calcaneus (p = 0.003, variations in the medial and lateral tuberosities, and the contour of the anterior articular surface), described significant shape differences between the CAI patients and healthy controls. The CAI patients generally had flatter talar joint surfaces and a flattened calcaneal ground‐contact surface. These findings suggest that specific bone shapes may increase the risk of developing CAI after sustaining a LAS. © 2019 The Authors. Journal of Orthopaedic Research® Published by Wiley Periodicals, Inc. on behalf of Orthopaedic Research Society. J Orthop Res 37:1892–1902, 2019

Estimation of thyroid volume from scintigraphy through 2D/3D registration of a statistical shape model

Accurate measurement of thyroid volume is important for thyroid disease diagnosis and therapy. In nuclear medicine, the thyroid volume is usually estimated from scintigraphy images using empirical equations. However, due to the lack of volumetric information from the scintigraphy image, the accuracy of equation-based estimation is imperfect. To solve this problem, this paper proposes a method which registers a 3D thyroid statistical shape model (SSM) to a single-view scintigraphy image to achieve more accurate volume estimation. The SSM was constructed based on a training set of segmented 3D CT images, and the thyroid shape variations between the training subjects were modelled using the point distribution model. For thyroid volume estimation, the SSM was projected into the scintigraphy image of the target patient, and then the projected model shape was nonrigidly registered with the patient’s scintigraphy image. The resultant 2D deformation file was back-projected to 3D space to guide the deformation of the 3D SSM. This process was repeated iteratively until convergence, and the volume of the finally deformed SSM was considered as the estimation of the patient’s thyroid volume. For validation, this method was evaluated based on a test set of 20 scintigraphy images, achieving an estimation error of −2.10% ± 5.20% which was much less than the error of the conventional equation-based method (35.76% ± 15.20%) based on the same test set. The robustness of this method was further tested using a challenging case, i.e. a scintigraphy image with a large thyroid tumor. For this case, the volume estimation error was only 6.08%. Our method has significantly improved the accuracy of thyroid volume estimation from scintigraphy images, and it will enhance the value of scintigraphy imaging for thyroid disease diagnosis and radioiodine therapy.

Tangible interface for shape modeling by block assembly of wirelessly connected blocks

AbstractTangible computer-aided design (CAD) that can generate and manipulate 3D models more efficiently than conventional modeling interfaces is a new interface technology between users and computers through tangible physical devices. Because the majority of tangible CAD systems use cameras or projectors, 3D models of complex structures with visual occlusion cannot be generated. In this work, we propose and develop a system that allows users to create complex shapes by assembling wireless blocks. By allowing more freedom in the orientations of blocks, models created by the assembly operation can be used to generate true 3D shapes compared to the 2.5D shapes resulting from previous research. The physical blocks of the proposed system include infrared proximity sensors, inertial magnetic units, Zigbee modules, and Arduino microcontrollers. The status of the block assembly is transmitted to the workstation wirelessly by the Zigbee. The workstation collects the sensor data of the blocks and generates and renders 3D models of the physically assembled blocks. The concept of the proposed tangible CAD system can be used in various application areas where simple, easy-to-use shape modeling interfaces for creating models by assembling small blocks are sufficient. The education and gaming industries are good examples of where this concept can be applied easily.Highlights A 3D shape modeling system through physical block assembly with sensors is developed. Using wireless communication, a degree of freedom of block assembly is increased. The rotation of the block was calculated through the 9-axis sensor and applied to the rendered 3D model. Through efficient data processing, the problems of real-time wireless communication environment are overcome. Users who are not trained in CAD tools can easily create 3D shape models.

3D patella bone shape modeling after ACL injury and the association with cartilage health over 3-years after acl reconstruction

3D patella bone shape modeling after ACL injury and the association with cartilage health over 3-years after acl reconstruction

Markerless Multiview Motion Capture with 3D Shape Model Adaptation

AbstractIn this paper, we address simultaneous markerless motion and shape capture from 3D input meshes of partial views onto a moving subject. We exploit a computer graphics model based on kinematic skinning as template tracking model. This template model consists of vertices, joints and skinning weights learned a priori from registered full‐body scans, representing true human shape and kinematics‐based shape deformations. Two data‐driven priors are used together with a set of constraints and cues for setting up sufficient correspondences. A Gaussian mixture model‐based pose prior of successive joint configurations is learned to soft‐constrain the attainable pose space to plausible human poses. To make the shape adaptation robust to outliers and non‐visible surface regions and to guide the shape adaptation towards realistically appearing human shapes, we use a mesh‐Laplacian‐based shape prior. Both priors are learned/extracted from the training set of the template model learning phase. The output is a model adapted to the captured subject with respect to shape and kinematic skeleton as well as the animation parameters to resemble the observed movements. With example applications, we demonstrate the benefit of such footage. Experimental evaluations on publicly available datasets show the achieved natural appearance and accuracy.

Closing the gap between Earth-based and interplanetary mission observations: Vesta seen by VLT/SPHERE

Context. Over the past decades, several interplanetary missions have studied small bodies in situ, leading to major advances in our understanding of their geological and geophysical properties. These missions, however, have had a limited number of targets. Among them, the NASA Dawn mission has characterised in detail the topography and albedo variegation across the surface of asteroid (4) Vesta down to a spatial resolution of ~20 m pixel−1 scale. Aims. Here our aim was to determine how much topographic and albedo information can be retrieved from the ground with VLT/SPHERE in the case of Vesta, having a former space mission (Dawn) providing us with the ground truth that can be used as a benchmark. Methods. We observed Vesta with VLT/SPHERE/ZIMPOL as part of our ESO large programme (ID 199.C-0074) at six different epochs, and deconvolved the collected images with a parametric point spread function (PSF). We then compared our images with synthetic views of Vesta generated from the 3D shape model of the Dawn mission, on which we projected Vesta’s albedo information. Results. We show that the deconvolution of the VLT/SPHERE images with a parametric PSF allows the retrieval of the main topographic and albedo features present across the surface of Vesta down to a spatial resolution of ~20–30 km. Contour extraction shows an accuracy of ~1 pixel (3.6 mas). The present study provides the very first quantitative estimate of the accuracy of ground-based adaptive-optics imaging observations of asteroid surfaces. Conclusions. In the case of Vesta, the upcoming generation of 30–40 m telescopes (ELT, TMT, GMT) should in principle be able to resolve all of the main features present across its surface, including the troughs and the north–south crater dichotomy, provided that they operate at the diffraction limit.

Patch-based facial texture super-resolution by fitting 3D face models

Incorporating 3D models for face hallucination (FH) is an ill-posed problem in light of the low-resolution (LR) conditions. To deal with the challenges, a specific 3D shape modeling approach targeting LR face images is first proposed. Based on a few automatically detected 2D facial feature points, an adaptive fitting scheme to relax the fixed correspondence assumption on the facial contour is devised, allowing for pose-invariant shape recovery. In order to exploit the obtained 3D shape and pose for FH, a resolution-aware approach for registering the training 3D faces with the LR input is designed to avoid warping the LR face. Finally, the LR image formation process is reformulated to facilitate super-resolution (SR) of the 3D facial texture. Using this interpretation, the Lucas–Kanade algorithm is extended for 3D deformable models to rectify the imperfect landmark-based fitting on LR images in a posterior fashion. In this way, the final patch-wise SR stage is able to produce realistic facial textures and synthesize self-occluded regions for non-frontal poses. All in all, our main contribution is a novel and pragmatic “2D landmarks $$\rightarrow $$ 3D dense shape $$\rightarrow $$ LR fitting refinement $$\rightarrow $$ 3D FH” pipeline dedicated for the LR scenario. To justify its advantages, extensive evaluation is conducted on several publicly available datasets, revealing superior accuracy over state-of-the-art methods in 3D fitting and high-quality SR results for in-the-wild faces with an interocular distance of as few as five pixels. Moreover, the frontalized high-resolution texture is also verified to help boost the performance of cross-pose face recognition.

Marked and rapid change of bone shape in acutely ACL injured knees – an exploratory analysis of the Kanon trial

To investigate changes in knee 3D bone shape over the first 5 years after acute anterior cruciate ligament (ACL) injury in participants of the randomized controlled KANON-trial. Serial MR images over 5 years from 121 young (32 women, mean age 26.1 years) adults with an acute ACL tear in a previously un-injured knee were analyzed using statistical shape models for bone. A matched reference cohort of 176 individuals was selected from the Osteoarthritis Initiative (OAI). Primary endpoint was change in bone area of the medial femoral condyle; exploratory analyses compared results by treatment and examined other knee regions. Comparisons were made using repeated measures mixed model ANOVA with adjustment for age, sex and body mass index (BMI). Mean medial femur bone area increased 3.2% (78.0 [95% CI 70.2 to 86.4] mm2) over 5 years after ACL injury and most prominently in knees treated with ACL reconstruction (ACLR). A higher rate of increase occurred over the first 2years compared to the latter 3-years (66.2 [59.3 to 73.2] vs 17.6 [12.2 to 23.0] mm2) and was 6.7 times faster than in the reference cohort. The pattern and location of shape change in the extrapolated KANON data was very similar to that observed in another knee-osteoarthritis cohort. 3D shape modelling after acute ACL injury revealed rapid bone shape changes, already evident at 3 months. The bone-change pattern after ACL injury demonstrated flattening and bone growth on the outer margins of the condyles similar to that reported in established knee osteoarthritis.