Sparse Set Of Landmarks Research Articles

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Deep learning-based markerless tracking has revolutionized studies of animal behavior. Yet the generalizability of trained models tends to be limited, as new training data typically needs to be generated manually for each setup or visual environment. With each model trained from scratch, researchers track distinct landmarks and analyze the resulting kinematic data in idiosyncratic ways. Moreover, due to inherent limitations in manual annotation, only a sparse set of landmarks are typically labeled. To address these issues, we developed an approach, which we term GlowTrack, for generating orders of magnitude more training data, enabling models that generalize across experimental contexts. We describe: a) a high-throughput approach for producing hidden labels using fluorescent markers; b) a multi-camera, multi-light setup for simulating diverse visual conditions; and c) a technique for labeling many landmarks in parallel, enabling dense tracking. These advances lay a foundation for standardized behavioral pipelines and more complete scrutiny of movement.

Deep Unsupervised 3D Human Body Reconstruction from a Sparse set of Landmarks

In this paper we propose the first deep unsupervised approach in human body reconstruction to estimate body surface from a sparse set of landmarks, so called DeepMurf. We apply a denoising autoencoder to estimate missing landmarks. Then we apply an attention model to estimate body joints from landmarks. Finally, a cascading network is applied to regress parameters of a statistical generative model that reconstructs body. Our set of proposed loss functions allows us to train the network in an unsupervised way. Results on four public datasets show that our approach accurately reconstructs the human body from real world mocap data.

Single‐view procedural braided hair modeling through braid unit identification

AbstractWe propose the first approach that can generate procedural three‐dimensional (3D) hair involving braids modeled from a single‐view photograph. Existing single‐view image‐based hair modeling methods fail to handle braided hairstyles. Our approach combines image processing, deep neural networks, as well as two‐dimensional (2D) and 3D geometric algorithms. In order to train our neural network, we create a braid unit data set. Our recognition and segmentation system can successfully segment hair regions, braid and non‐braid regions, using convolutional neural networks. We further process the images to obtain the locations, sizes, and orientations of the braid units. Given these braid units, we perform braid structure analysis to obtain the braid strand curves. The procedural modeling of the 3D braids is represented using 3D helical curves where the parameters are extracted from the 2D image analysis. Furthermore, we extract 2D hair strands from the non‐braid region using the Gabor filter and orientation maps. Then, a 3D hair volume is generated with the hair region silhouette information. We project the 2D hair strands and braids on the 3D hair volume to obtain the 3D hair strands and 3D braids. The strands for the braid and non‐braid regions are used as guides to generate dense hair strands. Dense strands are emitted from the hair root triangle mesh and follow the guide strands. With a sparse set of landmarks, the hair region of the photograph is texture mapped to the 3D hair root mesh and used to color the strands. We successfully tested our approach on photographs showing variations of braid styles and hair color.

Finding complete 3D vertex correspondence for statistical shape modeling.

A statistical shape model that accurately generalizes a family of 3D shapes requires establishing correspondences across the set of shapes. However in 3D anatomical meshes, finding a sufficient number of landmarks to accurately describe the shape can be a challenge, and often only a few points are easily identifiable due to the smooth nature of the object surface. Using a sparse set of landmarks, this paper finds a dense set of vertex correspondences across a set of 3D aortic root meshes. This is achieved by non-rigidly transforming a source mesh to a target mesh. Then, for every vertex on the target, a corresponding vertex on the deformed source is found, resulting in complete correspondence. A more accurate transformation results in better correspondence establishment, and our mesh registration experiments show an average Hausdorff distance of 3.65mm, and an average point-to-mesh distance of 0.41mm, i. e. within one voxel.

3D facial landmarks: Inter-operator variability of manual annotation.

BackgroundManual annotation of landmarks is a known source of variance, which exist in all fields of medical imaging, influencing the accuracy and interpretation of the results. However, the variability of human facial landmarks is only sparsely addressed in the current literature as opposed to e.g. the research fields of orthodontics and cephalometrics. We present a full facial 3D annotation procedure and a sparse set of manually annotated landmarks, in effort to reduce operator time and minimize the variance.MethodFacial scans from 36 voluntary unrelated blood donors from the Danish Blood Donor Study was randomly chosen. Six operators twice manually annotated 73 anatomical and pseudo-landmarks, using a three-step scheme producing a dense point correspondence map. We analyzed both the intra- and inter-operator variability, using mixed-model ANOVA. We then compared four sparse sets of landmarks in order to construct a dense correspondence map of the 3D scans with a minimum point variance.ResultsThe anatomical landmarks of the eye were associated with the lowest variance, particularly the center of the pupils. Whereas points of the jaw and eyebrows have the highest variation. We see marginal variability in regards to intra-operator and portraits. Using a sparse set of landmarks (n=14), that capture the whole face, the dense point mean variance was reduced from 1.92 to 0.54 mm.ConclusionThe inter-operator variability was primarily associated with particular landmarks, where more leniently landmarks had the highest variability. The variables embedded in the portray and the reliability of a trained operator did only have marginal influence on the variability. Further, using 14 of the annotated landmarks we were able to reduced the variability and create a dense correspondences mesh to capture all facial features.

As-Rigid-AsPossible Distance Field Metamorphosis

Widely used for morphing between objects with arbitrary topology, distance field interpolation (DFI) handles topological transition naturally without the need for correspondence or remeshing, unlike surface-based interpolation approaches. However, lack of correspondence in DFI also leads to ineffective control over the morphing process. In particular, unless the user specifies a dense set of landmarks, it is not even possible to measure the distortion of intermediate shapes during interpolation, let alone control it. To remedy such issues, we introduce an approach for establishing correspondence between the interior of two arbitrary objects, formulated as an optimal mass transport problem with a sparse set of landmarks. This correspondence enables us to compute non-rigid warping functions that better align the source and target objects as well as to incorporate local rigidity constraints to perform as-rigid-aspossible DFI. We demonstrate how our approach helps achieve flexible morphing results with a small number of landmarks.

A Dense Deformable Model for Craniofacial Reconstruction

The traditional craniofacial reconstruction methods construct the face shape according to the soft tissue thickness measured at a sparse set of landmarks on the skull. But the landmarks are difficult to detect and generally need human interactive assistance. The quantity and position of the landmarks lack for uniform definition. We propose an automatic craniofacial reconstruction method based on a dense deformable model. To construct the model, hundreds of skull and face samples are acquired by CT scanner. A dense mesh registration algorithm is proposed to build the point-to-point correspondences of the samples. Based on the aligned samples, the deformable model is constructed. For a given skull, the reconstructed face is obtained by a model matching procedure. Experimental results indicate that the deformable model has good performance for craniofacial reconstruction.

Craniofacial reconstruction using a combined statistical model of face shape and soft tissue depths: Methodology and validation

Forensic facial reconstruction aims at estimating the facial outlook associated with an unidentified skull specimen. Estimation is generally based on tabulated average values of soft tissue thicknesses measured at a sparse set of landmarks on the skull. Traditional ‘plastic’ methods apply modeling clay or plasticine on a cast of the skull, approximating the estimated tissue depths at the landmarks and interpolating in between. Current computerized techniques mimic this landmark interpolation procedure using a single static facial surface template. However, the resulting reconstruction is biased by the specific choice of the template and no face-specific regularization is used during the interpolation process. We reduce the template bias by using a flexible statistical model of a dense set of facial surface points, combined with an associated sparse set of skull-based landmarks. This statistical model is constructed from a facial database of ( N = 118) individuals and limits the reconstructions to statistically plausible outlooks. The actual reconstruction is obtained by fitting the skull-based landmarks of the template model to the corresponding landmarks indicated on a digital copy of the skull to be reconstructed. The fitting process changes the face-specific statistical model parameters in a regularized way and interpolates the remaining landmark fit error using a minimal bending thin-plate spline (TPS)-based deformation. Furthermore, estimated properties of the skull specimen (BMI, age and gender, e.g.) can be incorporated as conditions on the reconstruction by removing property-related shape variation from the statistical model description before the fitting process. The proposed statistical method is validated, both in terms of accuracy and identification success rate, based on leave-one-out cross-validation tests applied on the facial database. Accuracy results are obtained by statistically analyzing the local 3D facial surface differences of the reconstructions and their corresponding ground truth. Identification success rate is obtained by comparing, based on correlation, Euclidean distance matrix (EDM) signatures of the reconstructed and the original 3D facial surfaces in the database. A subjective identification success rate is quantified based on face-pool tests. Finally a qualitative comparison is made between facial reconstructions of a real-case skull, based on two typical static face models and our statistical model, showing the shortcomings of current face models and the improved performance of the statistical model.

Statistically Deformable Face Models for Cranio-Facial Reconstruction

Forensic facial reconstruction aims at estimating the facial outlook associated to an unknown skull specimen. Estimation is based on tabulated average values of soft tissue thicknesses measured at a sparse set of landmarks on the skull. Traditional 'plastic' methods apply modeling clay or plasticine on a cast of the skull approximating the estimated tissue depths at the landmarks and interpolating in between. Current computerized techniques mimic this landmark interpolation procedure using a single facial surface template. However, the resulting reconstruction is biased by the specific choice of the template and no face specific regularization is present. We reduce the bias by using a flexible statistical model of a dense set of facial surface points combined with an associated sparse set of skull landmarks. The statistical model also provides a probability value, which can be used to regularize the reconstruction towards more plausible outlooks. The reconstruction is obtained by fitting the model skull landmarks to the corresponding landmarks indicated on a digital copy of the skull to be reconstructed. The fitting process alternates between changing the face-specific statistical model parameters and interpolating the remaining landmark fit error using a minimal bending Thin-Plate Spline (TPS) based deformation. Furthermore, estimated properties of the skull specimen (BMI, age and gender e.g.) can be incorporated as conditions on the reconstruction by removing property-related shape variation from the statistical model description before the fitting process. This iterative statistical model based reconstruction process is shown by experiment to converge to a realistic reconstruction of the face, independent of the initial template.