3D Landmarks Research Articles

Deep learning-based workflow for hip joint morphometric parameter measurement from CT images

Objective. Precise hip joint morphometry measurement from CT images is crucial for successful preoperative arthroplasty planning and biomechanical simulations. Although deep learning approaches have been applied to clinical bone surgery planning, there is still a lack of relevant research on quantifying hip joint morphometric parameters from CT images. Approach. This paper proposes a deep learning workflow for CT-based hip morphometry measurement. For the first step, a coarse-to-fine deep learning model is designed for accurate reconstruction of the hip geometry (3D bone models and key landmark points). Based on the geometric models, a robust measurement method is developed to calculate a full set of morphometric parameters, including the acetabular anteversion and inclination, the femoral neck shaft angle and the inclination, etc. Our methods were validated on two datasets with different imaging protocol parameters and further compared with the conventional 2D x-ray-based measurement method. Main results. The proposed method yields high bone segmentation accuracies (Dice coefficients of 98.18% and 97.85%, respectively) and low landmark prediction errors (1.55 mm and 1.65 mm) on both datasets. The automated measurements agree well with the radiologists’ manual measurements (Pearson correlation coefficients between 0.47 and 0.99 and intraclass correlation coefficients between 0.46 and 0.98). This method provides more accurate measurements than the conventional 2D x-ray-based measurement method, reducing the error of acetabular cup size from over 2 mm to less than 1 mm. Moreover, our morphometry measurement method is robust against the error of the previous bone segmentation step. As we tested different deep learning methods for the prerequisite bone segmentation, our method produced consistent final measurement results, with only a 0.37 mm maximum inter-method difference in the cup size. Significance. This study proposes a deep learning approach with improved robustness and accuracy for pelvis arthroplasty planning.

More than one way to eat a mouse: Skull shape variation within a monophyletic group of mammals (Marsupialia; Dasyurinae)

AbstractFunctional requirements for different diets have a strong selective pressure leading to convergent adaptations across different groups of mammals. As body size has an influence on many biomechanical and physiological characteristics, it is logical that dietary adaptations often have a strong link to body size resulting in observed ‘size guilds’ in many diverse phylogenetic lineages. However, non‐allometric variation in skull shape may provide an alternative pathway for species to adapt to diets or prey handling by breaking the mould away from the line of least evolutionary resistance, increased body size. We investigated the relationships between cranial shape and body size in the evolutionary context of the shift from insectivory to hypercarnivory in a monophyletic group of 16 dasyurine marsupial species. Analysis of 35 Procrustes‐corrected 3D skull landmarks revealed that overall, there was a correlation between mean skull shape and size that correlated with diet. However, the dominant shape variation (almost 80% irrespective of phylogeny, and more than 90% when constrained by phylogeny) was not size‐related. This non‐allometric pattern of skull shape reveals adaptation for different feeding strategies that is independent of absolute size. This highlights the potential for specific ‘thresholds’ in size and shape to exist that have the potential to mark shifts from one dietary niche to the next, that might be applied to the evolution of carnivorous taxa from other lineages.

Automatic cochlear multimodal 3D image segmentation and analysis using atlas–model-based method

Objectives To propose an automated fast cochlear segmentation, length, and volume estimation method from clinical 3D multimodal images which has a potential role in the choice of cochlear implant type, surgery planning, and robotic surgeries. Methods: Two datasets from different countries were used. These datasets include 219 clinical 3D images of cochlea from 3 modalities: CT, CBCT, and MR. The datasets include different ages, genders, and types of cochlear implants. We propose an atlas–model-based method for cochlear segmentation and measurement based on high-resolution μCT model and A-value. The method was evaluated using 3D landmarks located by two experts. Results: The average error was 0.61 ± 0.22 mm and the average time required to process an image was 5.21 ± 0.93 seconds (P<0.001). The volume of the cochlea ranged from 73.96 mm 3 to 106.97 mm 3 , the cochlear length ranged from 36.69 to 45.91 mm at the lateral wall and from 29.12 to 39.05 mm at the organ of Corti. Discussion: We propose a method that produces nine different automated measurements of the cochlea: volume of scala tympani, volume of scala vestibuli, central lengths of the two scalae, the scala tympani lateral wall length, and the organ of Corti length in addition to three measurements related to A-value. Conclusion: This automatic cochlear image segmentation and analysis method can help clinician process multimodal cochlear images in approximately 5 seconds using a simple computer. The proposed method is publicly available for free download as an extension for 3D Slicer software.

HR-Net: A Landmark Based High Realistic Face Reenactment Network

In the past, GAN-based face reenactment methods concentrated mostly on transferring the facial expressions and positions of the source. However, the generated results were susceptible to blurring in some minute details of the face, such as teeth and hair, and their backgrounds were also not guaranteed to be consistent with the manipulated images in terms of light and shadow. Because of these issues, the generated results could be distinguishable as fakes. In this paper, we proposed a landmark based method named HR-Net, which can render source facial expressions and postures on any identity and simultaneously generate realistic face details. Firstly, a lightweight landmark identity conversion module (LIC) was designed to address the identity leakage problem, and it represented facial expressions and poses with only 68 2D landmarks. On this basis, a boundary-guided face reenactment module (BFR) was presented to only learn the background of the reference images; thus, the results generated by BFR can be consistent with the reference images’ light and shadow. Moreover, a novel local perceptual loss function was implemented to support the BFR module in generating more realistic details. Extensive experiments demonstrated that our method achieved the state of the art.

Unsupervised performance analysis of 3D face alignment with a statistically robust confidence test

This paper addresses the problem of analyzing the performance of 3D face alignment (3DFA), or facial landmark localization. This task is usually supervised, based on annotated datasets. Nevertheless, in the particular case of 3DFA, the annotation process is rarely error-free, which strongly biases the results. Alternatively, unsupervised performance analysis (UPA) is investigated. The core ingredient of the proposed methodology is the robust estimation of the rigid transformation between predicted landmarks and model landmarks. It is shown that the rigid mapping thus computed is affected neither by non-rigid facial deformations, due to variabilities in expression and in identity, nor by landmark localization errors, due to various perturbations. The guiding idea is to apply the estimated rotation, translation and scale to a set of predicted landmarks in order to map them onto a mathematical home for the shape embedded in these landmarks (including possible errors). UPA proceeds as follows: (i) 3D landmarks are extracted from a 2D face using the 3DFA method under investigation; (ii) these landmarks are rigidly mapped onto a canonical (frontal) pose, and (iii) a statistically-robust confidence score is computed for each landmark. This allows to assess whether the mapped landmarks lie inside (inliers) or outside (outliers) a confidence volume. An experimental evaluation protocol, that uses publicly available datasets and several 3DFA software packages associated with published articles, is described in detail. The results show that the proposed analysis is consistent with supervised metrics and that it can be used to measure the accuracy of both predicted landmarks and of automatically annotated 3DFA datasets, to detect errors and to eliminate them. Source code and supplemental materials for this paper are publicly available at https://team.inria.fr/robotlearn/upa3dfa/.

Evaluation of age variation changes in cervical vertebrae: 2-Dimensional (2D) geometric morphometrics approach

Age estimation in adults is a complicated task because of various external factors occur concurrently with increasing age. The geometric morphometric method (GMM) is an approach that focuses on shape analysis and is widely recognized for its high reliability and reproducibility. The aim of this study was to explore the variation of cervical vertebrae among different age groups of the Malaysian population by GMM. Lateral skull radiographs of 432 subjects comprising four adult age groups; young adult age group (20–30 years old), early middle age group (31–40 years old), late middle age group (41–50 years old) and, elder adult age group (51–60 years old) were selected. Fifty-three 2-dimensional (2D) landmarks were applied to the digitalized radiographs by TPSDig2 (Version 2.31) software. Geometric morphometric analysis was performed by MorphoJ software. Results showed that the first three principal components (PC) contributed to 47.71 % of the cervical vertebrae variation and were shown in both lollipop and wireframe graphs. Procrustes ANOVA indicated that the shape was significantly different among different age groups. Canonical variate analysis revealed significant differences of both mahalanobis and procrustes distances among age groups with substantial individual overlap within groups. Discriminant function analysis (DFA) showed a correct classification rate for 61.5 % of cases respective to age groups. In conclusion, this study found significant differences in the shape of cervical vertebrae among different age groups of the Malaysian population using the GMM.

Morphological changes of the frontal sinus with age: A two-dimensional geometric morphometric study

BackgroundAge estimation of unidentified human remains is an essential for establishing the biological profile of unidentified human remains. Previous studies have investigated the value linear measurements of the frontal sinus for age estimation while the value of two-dimensional (2D) geometric morphometric measurements have not been explored for this purpose. ObjectiveThis study aims to evaluate the morphological variation of frontal sinus among adults Malaysian for age estimation using 2D geometric morphometric methods. MethodsLateral skull radiographs of 453 adult Malaysians, comprising 151 young adults (20–39 years old), 150 middle-aged adults (40–59 years old) and 152 old adults (60–79 years old), were retrieved retrospectively. Eight 2D landmarks were applied to digitalized radiographs using TPSDig2 software. 2D geometric morphometric analysis was performed by MorphoJ software. ResultsProcrustes ANOVA revealed no significant differences in shape and size (p-value > 0.05) of frontal sinus between the three age groups. However, Canonical Variate Analysis showed significant differences for Mahalanobis and Procrustes distances (p-value < 0.05) between young and old adult groups. Discriminant Function Analysis after cross-validation demonstrated an age classification accuracy of 53.5 %. ConclusionThe study suggests that 2D geometric morphometric analysis of the frontal sinus cannot be reliably used to estimate the age of an unknown Malaysian individual.

Measuring the Unilateral Cleft Lip Nasal Deformity: Lateral Deviation of Subnasale Is a Clinical and Morphologic Index of Unrepaired Severity.

Objective measurement of pre-operative severity is important to optimize evidence-based practices given that the wide spectrum of presentation likely influences outcomes. The purpose of this study was to determine the correlation of objective measures of form with a subjective standard of cleft severity. 3D images were ranked according to severity of nasal deformity by 7 cleft surgeons so that the mean rank could be used as the severity standard. 45 patients with unilateral cleft lip and 5 normal control subjects. Each image was assessed using traditional anthropometric analysis, 3D landmark displacements, and shape-based analysis to produce 81 indices for each subject. The correlation of objective measurements with the clinical severity standard. Lateral deviation of subnasale from midline was the best predictor of severity (0.86). Other strongly-correlated anthropometric measurements included columellar angle, nostril width ratio, and lateral lip height ratio (0.72, 0.80, 0.79). Almost all shape-based measurements had tight correlation with the severity standard, however, dorsum deviation and point difference nasolabial symmetry were the most predictive (0.84, 0.82). Quantitative measures of severity transcend cleft type and can be used to grade clinical severity. Lateral deviation of subnasale was the best measure of severity and may be used as a surrogate of uncoupled premaxillary growth; it should be recorded as an index of pre-operative severity with every cleft lip repair. The correlation of other measures evaluated clarify treatment priorities and could potentially be used to grade outcomes.

Characterizing Three-Dimensional Alignment of the Hip, Knee, Ankle and Foot under Physiological Upright Load. A Weight-Bearing Computed Tomography Study in Arthritic Joints and Healthy Controls

Category: Ankle Arthritis; Other Introduction/ Purpose: Lower limb alignment assessment is commonly performed using two-dimensional (2D) conventional- radiographs. Weight-Bearing Computed-Tomography (WBCT) imaging that allows concomitant 3D imaging of the hip, knee, ankle, and foot, a more complete and multidimensional assessment of the entire overall lower limb alignment is now possible. The aims of this study were: (1) to characterize the normal relative 3D alignment of the center of the Hip, Knee, and Ankle joints in relation to the weight bearing Foot Tripod in a cohort of healthy control volunteers with no lower extremity pathologies, using WBCT imaging. (2) to perform the same 3D WBCT assessment in a cohort of patients with either hip osteoarthritis (HOA), knee osteoarthritis (KOA) or ankle osteoarthritis (AOA), and to compare the results between arthritic cases and controls. Methods: Prospective comparative and controlled cohort-study contained 7 HOA limbs (4 patients), 17 KOA limbs (10 patients), 7 AOA limbs (4 patients) and 10 control limbs (5 patients) that received WBCT imaging of the full lower extremity. Using multiplanar reconstruction WBCT images, 3D landmark coordinates (on X, Y, and Z planes) were manually measured by two observers. The utilized software (CubeVue®) generated an automatic calculation of the Foot-Hip Offset (FHO), Foot-Knee Offset (FKO) and Foot and Ankle Offset (FAO). The relationship between the center of the hip, knee and ankle joints and the bisecting line of the foot tripod was assessed and compared between HOA, KOA, AOA patients and controls. Examples of measurements for arthritic patients and controls is presented in Figure 1. Continuous data was assessed for normality with the Shapiro-Wilk test, and variables were compared using ANOVA or Kruskal- Wallis Rank Sum. P-Values of less than 0.05 were considered significant. Results: The average FAO and 95%-Confidence-intervals-(CI) for respectively HOA, KOA, AOA and controls were respectively: 3.62% (0.4 to 6.8) (neutral), 2.8% (0.78 to 4.9) (neutral), -4.68% (-7.8 to -1.4) (varus), and 2.12% (-0.5 to 4.8) (neutral). The FAO- differences were found to be significant between the groups (p=0.0077), with AOA patients being significantly different than all the other groups (Figure 2). Similarly, the HFO was found to be significantly different-between the groups (p=0.002), with the following average values and 95%CI for respectively HOA, KOA, AOA and controls: 0.7% (-6.4 to 7.8), 2.3% (-2.3 to 6.8), -10.1% (-17.2 to -3.0), and 5.3% (-0.6 to 11.3). Again, the AOA patients were found to be significantly different than the other groups. No significant differences were found between the groups when assessing the KFO (p=0.37). Conclusion: The baseline 3D lower limb alignment and relative position of the hip, knee, ankle and foot was assessed and established for the first time in the literature. When comparing 3D alignment in arthritic patients with hip, knee or ankle OA and controls, we observed that AOA was found to be the one affecting more the overall 3D alignment of the lower extremity, with no complete compensation of the deformity through the other joints, resulting in significantly different values of HFO, KFO and FAO in patients with ankle OA. Additional prospective studies with longer cohorts of patients are needed.

Region-Aware Deep Feature-Fused Network for Robust Facial Landmark Localization

In facial landmark localization, facial region initialization usually plays an important role in guiding the model to learn critical face features. Most facial landmark detectors assume a well-cropped face as input and may underperform in real applications if the input is unexpected. To alleviate this problem, we present a region-aware deep feature-fused network (RDFN). The RDFN consists of a region detection subnetwork and a region-wise landmark localization subnetwork to explicitly solve the input initialization problem and derive the landmark score maps, respectively. To exploit the association between tasks, we develop a cross-task feature fusion scheme to extract multi-semantic region features while trading off their importance in different dimensions via global channel attention and global spatial attention. Furthermore, we design a within-task feature fusion scheme to capture the multi-scale context and improve the gradient flow for the landmark localization subnetwork. At the inference stage, a location reweighting strategy is employed to transform the score maps into 2D landmark coordinates. Extensive experimental results demonstrate that our method has competitive performance compared to recent state-of-the-art methods, achieving NMEs of 3.28%, 1.48%, and 3.43% on the 300W, AFLW, and COFW datasets, respectively.

Using a structured light scanner to evaluate 3-dimensional soft-tissue changes after extracting 4 premolars in young adult female patients

Facial esthetics have always received much attention in orthodontic treatment, especially in young adult female patients. Three-dimensional (3D) soft-tissue changes after orthodontic extraction have not been fully explained. This study evaluated the 3D morphologic changes after orthodontic extraction in young female patients using a structured light scanner. Forty-five adult female patients aged 20-25 years were enrolled in our study. The treatment group consisted of patients who received orthodontic treatment with 4 premolar extractions, and the control group was composed of young female volunteers who had not undergone any orthodontic treatment. To monitor the soft-tissue changes, 9 morphologic regions and 12 landmarks were identified for the 3D deviation analyses. The spatial deviations of landmarks and regions in the x, y, and z directions were constructed for quantitative analysis. Color map images were constructed to visualize soft-tissue displacement as a qualitative evaluation. The paired sample test was used to compare differences at the beginning of the experiment (T0) and after 24 months (T1) in both groups. An independent t test with Bonferroni correction was performed to compare differences between the treatment and control groups. A linear regression test was performed between incisor retraction and changes in the perioral tissues. Subtracting the effect of aging from the lip changes in the control group, the treatment group showed a statistically significant difference in the displacement of labrale superius (-1.37 mm), labrale inferius (-1.89 mm), the upper lip region (-0.98 mm), and the lower lip region (-1.36 mm) along the z-axis. No significant differences were found between the treatment and control groups in the temporal, parotideomasseteric, and buccal regions. Pearson correlation tests indicated a positive correlation between incisor tip retraction and changes in soft tissues (two-dimensional cephalometric analysis, 3D landmark measurements, and 3D regional measurements). The correlation coefficient ranged between 0.45 and 0.55. Three-dimensional soft-tissue changes were mainly concentrated in the upper and lower lip regions in adult female patients after the 4 premolars were extracted. For female patients aged 20-25 years with 4 extracted premolars, soft-tissue changes in the temporal, parotideomasseteric, and buccal regions were not clinically significant.

Three-Dimensional Cephalometric Landmark Annotation Demonstration on Human Cone Beam Computed Tomography Scans.

Craniofacial cephalometric analysis is a diagnostic tool used for the assessment of the relationship of various bones and soft tissues in the head and face. Cephalometric analysis has been traditionally conducted with the use of 2D radiographs and landmark sets and restricted to size, linear and angular measurements, and 2D relationships. The increasing use of 3D cone beam computed tomography (CBCT) scans in the dental field dictates the need for the evolution to 3D cephalometric analysis, which incorporates shape and a more realistic analysis of longitudinal development in all three planes. This study is a demonstration of 3D cephalometric analysis with the use of a validated set of skeletal tissue landmarks on human CBCT scans. Detailed instructions for the annotation of each landmark on a 3D volume are provided as part of a step-by-step protocol. The generated measurements and 3D coordinates of the landmarks can be exported and used both for clinical and research purposes. The introduction of 3D cephalometric analysis in basic and clinical craniofacial studies will lead to future advancements in the field of craniofacial growth and development.

Sequential Monte Carlo localization in topometric appearance maps

Representing the scene appearance by a global image descriptor (BoW, NetVLAD, etc.) is a widely adopted choice to address Visual Place Recognition (VPR). The main reasons are that appearance descriptors can be effectively provided with radiometric and perspective invariances as well as they can deal with large environments because of their compactness. However, addressing metric localization with such descriptors (a problem called Appearance-based Localization or AbL) achieves much poorer accuracy than those techniques exploiting the observation of 3D landmarks, which represent the standard for visual localization. In this paper, we propose ALLOM (Appearance-based Localization with Local Observation Models) which addresses AbL by leveraging the topological location of a robot within a map to achieve accurate metric estimations. This topology-assisted metric localization is implemented with a sequential Monte Carlo Bayesian filter that applies a specific observation model for each different place of the environment, thus taking advantage of the local correlation between the pose and the appearance descriptor within each region. ALLOM also benefits from the topological structure of the map to detect eventual robot loss-of-tracking and to effectively cope with its relocalization by applying VPR. Our proposal demonstrates superior metric localization capability compared to different state-of-the-art AbL methods under a wide range of situations.

In Cabin Driver Monitoring and Alerting System For Passenger cars using Machine Learning.

The primary causes of road accidents are driver errors. Such as excessive speeding, driver distraction, and driver sleepiness. Diver sleepiness and distraction can be controlled by implementing advanced technology such as an in-cabin driver monitoring system. This is a technological solution that uses a machine learning algorithm to improve road safety. This paper describes a new real-time driver monitoring and alerting system that solely monitors driver Sleepiness, diver distraction, and seatbelt wearing status in order to prevent road accidents. It employs the YOLOv5 deep learning algorithm to detect various types of distraction and For seatbelt wearing status and computer vision’s cvzone framework is used to detect sleepiness using 3D facial landmarking. In this system, The camera is mounted in the centre of the vehicle’s dashboard for data collection and real-time implementation. The Raspberry Pi 3b module is used for the Realtime implementation. According to the findings of this study, the system is capable of real-time implementation, and the machine learning model used in the study achieves an accuracy of more than 90%.

Neurocranial growth in the OIM mouse model of osteogenesis imperfecta.

Osteogenesis imperfecta (OI) is a disorder of type I collagen characterized by abnormal bone formation. The OI craniofacial phenotype includes midfacial underdevelopment, as well as neurocranial changes (e.g., macrocephaly and platybasia) that may also affect underlying nervous tissues. This study aims to better understand how OI affects the integrated development of the neurocranium and the brain. Juvenile and adult mice with OI (OIM) and unaffected wild type (WT) littermates were imaged using in vivo micro-computed tomography (microCT). Virtual endocast models were used to measure brain volume, and 3D landmarks were collected from the cranium and brain endocasts. Geometric morphometric analyses were used to compare brain shape and integration between the genotypes. OIM mice had increased brain volumes (relative to cranial centroid size) only at the juvenile stage. No significant difference was seen in cranial base angle (CBA) between OIM and WT mice. However, CBA was higher in juvenile than in adult OIM mice. Brain shape was significantly different between OIM and WT mice at both stages, with OIM mice having more globular brains than WT mice. Neurocranial and brain morphology were strongly integrated within both genotypes, while adult OIM mice tended to have lower levels of skull-brain integration than WT mice. These results suggest that neurocranial dysmorphologies in OI may be more severe at earlier stages of postnatal development. Decreased skull-brain integration in adult mice suggests that compensatory mechanisms may exist during postnatal growth to maintain neurological function despite significant changes in neurocranial morphology.

ImplicitPCA: Implicitly-proxied parametric encoding for collision-aware garment reconstruction

The emerging remote collaboration in a virtual environment calls for quickly generating high-fidelity 3D humans with cloth from a single image. To estimate clothing geometry and topology, parametric models are widely used but often lack details. Alternative approaches based on implicit functions can generate accurate details but are limited to closed surfaces and may not produce physically correct reconstructions, such as collision-free human avatars. To solve these problems, we present ImplicitPCA, a framework for high-fidelity single-view garment reconstruction that bridges the good ends of explicit and implicit representations. The key is a parametric SDF network that closely couples parametric encoding with implicit functions and thus enjoys the fine details brought by implicit reconstruction while maintaining correct topology with open surfaces. We further introduce a collision-aware regression network to ensure the physical correctness of cloth and human. During inference, an iterative routine is applied to an input image with 2D garment landmarks to obtain optimal parameters by aligning the cloth mesh projection with the 2D landmarks and fitting the parametric implicit fields with the reconstructed cloth SDF. The experiments on the public dataset and in-the-wild images demonstrate that our result outperforms the prior works, reconstructing detailed, topology-correct 3D garments while avoiding garment-body collisions.

The effect of sex and age on facial shape directional asymmetry in adults: A 3D landmarks-based method study.

Facial directional asymmetry research, including age-related changes, is crucial for the evaluation of treatment of craniofacial malformations/trauma in orthodontics, facial surgery and forensic sciences. The aim was to describe facial directional asymmetry (DA) in different age categories of adults using 3D methods. According to our hypothesis, facial shape DA (1) depends on sex; (2) differs among age groups; and (3) has wider variability in older age. A cross-sectional sample of healthy Czech adults without craniofacial trauma or anomalies consisted of 300 3D facial models (151 females). The age-range in the study was between 20-80 years. The shape asymmetry of 28 3D landmarks was evaluated using geometric morphometrics and multivariate statistics. The manifestation of DA was similar in both sexes and in each age category; however, there were some statistical differences. In contrast to the ideal symmetrical face, the mean asymmetrical faces tended to create a slightly bent "C" shape of the midline. Therefore, the upper face was rotated slightly clockwise and the lower face counter-clockwise. The right eye was located slightly higher, with the nasal tip and mandibular region tilting to the left. Sex differences in facial DA were significant before the age of 40. DA was more significant in the youngest males than in the oldest, while the women's DA did not change. The DA patterns were similar in both sexes and in all age categories (a slightly bent C shape of the midline); however, some significant local differences between male age groups were found. A significantly more pronounced asymmetry compared to other age groups was found only in the youngest males from 20 to 40 years. Moreover, significant sexual dimorphism of DA rapidly decreased after middle age, likely caused by the same age-related changes of the face during aging.

SDMT: Spatial Dependence Multi-Task Transformer Network for 3D Knee MRI Segmentation and Landmark Localization

Knee segmentation and landmark localization from 3D MRI are two significant tasks for diagnosis and treatment of knee diseases. With the development of deep learning, Convolutional Neural Network (CNN) based methods have become the mainstream. However, the existing CNN methods are mostly single-task methods. Due to the complex structure of bone, cartilage and ligament in the knee, it is challenging to complete the segmentation or landmark localization alone. And establishing independent models for all tasks will bring difficulties for surgeon's clinical using. In this paper, a Spatial Dependence Multi-task Transformer (SDMT) network is proposed for 3D knee MRI segmentation and landmark localization. We use a shared encoder for feature extraction, then SDMT utilizes the spatial dependence of segmentation results and landmark position to mutually promote the two tasks. Specifically, SDMT adds spatial encoding to the features, and a task hybrided multi-head attention mechanism is designed, in which the attention heads are divided into the inter-task attention head and the intra-task attention head. The two attention head deal with the spatial dependence between two tasks and correlation within the single task, respectively. Finally, we design a dynamic weight multi-task loss function to balance the training process of two task. The proposed method is validated on our 3D knee MRI multi-task datasets. Dice can reach 83.91% in the segmentation task, and MRE can reach 2.12 mm in the landmark localization task, it is competitive and superior over other state-of-the-art single-task methods.

Free-HeadGAN: Neural Talking Head Synthesis With Explicit Gaze Control

We present Free-HeadGAN, a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks is sufficient for achieving state-of-the-art generative performance, without relying on strong statistical priors of the face, such as 3D Morphable Models. Apart from 3D pose and facial expressions, our method is capable of fully transferring the eye gaze, from a driving actor to a source identity. Our complete pipeline consists of three components: a canonical 3D key-point estimator that regresses 3D pose and expression-related deformations, a gaze estimation network and a generator that is built upon the architecture of HeadGAN. We further experiment with an extension of our generator to accommodate few-shot learning using an attention mechanism, in case multiple source images are available. Compared to recent methods for reenactment and motion transfer, our system achieves higher photo-realism combined with superior identity preservation, while offering explicit gaze control.

3D occlusal changes of upper first molars after rapid maxillary expansion on permanent versus deciduous teeth: a retrospective multicenter CBCT study

BackgroundThe purpose of this study was to compare the three-dimensional dental changes for the maxillary first molars and the overall skeletal effects achieved after expansion between the rapid maxillary expansion (RME) appliance attached to two different anchor units, the maxillary deciduous molars and the maxillary permanent first molars.MethodsPatients were retrospectively selected according to the anchorage unit used for RME: deciduous upper second molars (RME-E group; 10 M, 10 F; mean age 8.4 ± 1.1 years) and first upper permanent molars (RME-6 group; 10 M, 10 F; mean age 12.6 ± 1.8 years). CBCT scans were obtained before treatment start (T1) and after retention and removal of the expander (T2). Images were analyzed using a new three-dimensional intra-hemi-maxillary reference system. 3D landmarks were marked to calculate all changes on maxillary first permanent molars; mesio-distal and buccal-lingual inclination and rotation, as well as intermolar and interforaminal distances were calculated. The Wilcoxon test was used to compare within-group changes, whereas the Mann–Whitney test was used to compare between-group differences, with the significance level set at 0.05.ResultsIn the RME-E group, significant distorotation and lingual inclination of the first permanent molars at T2 were observed (p < 0.01); in the RME-6 group, only the buccolingual inclination of the crossbite side after RME was resulted statistically significant (p < 0.01). In both groups, intermolar and interforaminal values increased significantly (p < 0.01). Intergroup analysis showed a significantly higher distorotation and reduced buccal inclination of maxillary first permanent molars in the RME-E group after RME (p < 0.01).ConclusionsRME is effective in treating maxillary transverse hypoplasia; RME anchored too deciduous teeth spontaneously reduces buccal inclination and increases distorotation of maxillary first permanent molars, whereas anchorage to permanent molars is associated with increased buccal inclination, albeit with little clinical significance.