Deformable Template Research Articles

Estimation of footprints of the canine stifle ligaments using deformable shape templates of bones

Knowledge regarding the ligament footprints in the canine stifle is essential for biomechanical modeling of the joint and patient-specific surgical planning for anatomical ligament reconstruction. The present study aimed to establish and evaluate deformable shape templates (DSTs) of the femur and tibia with footprints of the cruciate and collateral ligaments embedded for the noninvasive estimation of ligament footprint positions. To this end, a data set of computed tomography (CT)-derived surface models of the femur and tibia were established and used to build statistical shape models (SSMs). The contours of the stifle ligaments were obtained from CT scans of 27 hindlimb specimens with radio-opaque markings on the ligament footprints. The DST, constructed by embedding averaged footprint contours into the SSM, was used to estimate subject-specific ligament footprints in a leave-one-out cross-validation framework. The DST predictions were compared with those derived from radio-opaque-marked footprints. The results showed that the averaged Euclidean distances between the estimated and reference footprint centroids were less than 1.2 mm for the cruciate ligaments and 2.0 mm for the collateral ligaments. The DST appeared to provide a feasible alternative approach for noninvasively estimating the footprints of the stifle ligaments in vivo.

Tomato flower pollination features recognition based on binocular gray value-deformation coupled template matching

It is necessary to recognize the tomato pollination features for the designing demand of intelligent and precise tomato supplementary pollination equipment. Mentioned pollination features include flower opening state and the three-dimensional position and pose of flower anther. Tomato flower pollination features recognition method is designed in this paper based on the deep learning full opened flower recognition model and binocular template matching three-dimensional information recognition method. First of all, a binocular stereo vision system is built to acquire the tomato flower images in greenhouse with natural lighting. Which can help vision system avoid the impact of inconsistent light intensity on image recognition. The acquired images were equalized with three groups parameters to labeled images. And the improved MC-AlexNet deep learning model is established to recognize the full opened tomato flowers in the left image of image pair acquired with binocular vision system. Then, the template is created with the full opened flower recognition result of deep learning model in left image based on gray value and deformation template matching method. And the template matching is established to recognize the corresponding full opened flowers in the right image of image pair. Finally, with the template matching result, the anther segmentation is conducted to calculate three-dimensional position and pose of anther. The experimental results show that the accuracy of full opened tomato flowers recognition model is 96.23%. The average position recognition deviation of anther is 5.94 mm. And the average anther pose recognition deviation in three plane is 6.24° for the images that anthers can be observed. And the average time consuming is about 143.18 ms per image pair. It can be concluded that the method of binocular template matching established in this paper can fulfill the demand of supplementary pollination equipment design, and the research result lays a foundation for designing and improvement of tomato supplementary pollination equipment in greenhouse.

A Review of Face Detection Methods Based on Feature Approach

Face detection technology is the first and essential step for facial-based analysis algorithms such as face recognition, face feature extraction, face alignment, face enhancement, and face parsing. That is besides serving other applications related to human intention and act analysis such as facial expression recognition, gender recognition, and age classification. Face detection is used to detect faces in digital images. It is a special case of object detection and can be used in many areas such as biometrics, security, law enforcement, entertainment, and personal safety. There are various methods proposed in the field of face detection, and they all compete to make it more advanced and accurate. These methods belong to two main approaches; feature-based approach and image-based approach. This paper reviews the face detection methods that belong to the feature-based approach. Moreover, their work concepts, strengths, and limitations are mentioned. This paper concentrates on the feature-based approach due to its simplicity and high applicability in real-time detection compared to the image-based approach. Index Terms— Face Detection Methods, Feature-Based Approach, Viola-Jones Algorithm, Deformable Template Matching.

Unique contributions of brain amyloid load and APOE4 Status to Regional Brain Volume

AbstractBackgroundBrain amyloid and tau, and volumetric atrophy on MRI, are defining features of AD (Jack et al., 2018). Despite growing interest in these component processes, their relationship is still poorly understood.We evaluated the unique contributions of APOE4 allele (E4+/E4−) and brain amyloid (Aβ) status (Amy+/Amy−) to brain volume in the elderly to identify associations with neurodegeneration in regions vulnerable to Alzheimer’s disease (AD).MethodAccelerated 3T T1‐weighted brain MRI scans (N=1,353) obtained at screening for three distinct phases of ADNI with available information on APOE status and/or Amyloid positive/negative status were identified for the following subjects: 561 healthy controls (CN) (age: 71.8±6.3 y, 330F/231M), 599 MCI (mild cognitive impairment; age: 71.9 ± 7.5 y, 264 F/335 M), and 193 AD (age: 74.4±8.2 y, 79 F/114 M). Amyloid status was based on standardized cut‐off values in amyloid sensitive PET scans (Royse et al., 2021).We applied unbiased tensor‐based morphometry (TBM; Leow et al., 2009; Hua et al., 2011) to each scan to measure relative volume differences between each subject and a common minimal deformation template (MDT), resulting in 3D Jacobian maps to represent relative expansion or contraction at each brain voxel.At each voxel, a linear regression model was fit to relate regional brain volumes to amyloid status, and APOE4, after adjusting for covariates. P‐value maps were generated to visualize the pattern of voxel‐wise model contributions and statistical significance while enforcing a standard 5% false discovery rate (FDR) correction for multiple statistical comparisons across the brain (Benjamini et al., 1995).ResultAmyloid positive status was associated with significantly lower regional volume in the temporal lobes and higher volume in the ventricles, i.e., ventricular expansion. APOE4 positive status was associated with lower volume in the temporal lobes (see Fig. 1). ConclusionThe highly localized effects of APOE4 status and amyloid load on brain volumes are consistent with greatest effects of AD pathologic processes in the temporal lobes and similar associations found for Amy+ and E4+ status with cortical thinning (Li et al. (2017).Continued analysis of longitudinal ADNI data will offer further insight into effects of amyloid and APOE4 allele status on atrophy estimates.

An intelligent system for craniomaxillofacial defecting reconstruction

Craniomaxillofacial defects caused by congenital or acquired reasons seriously affect patients' physical and mental health. How to accurately and objectively repair the morphology of craniomaxillofacial tissues and organs through surgery is a difficult problem, and the preoperative virtual design is crucial. Traditional preoperative virtual design methods include mirror technology, statistical shape model, and deformable template. However, these methods are complex, time-consuming, and only applicable to some types of defects. Therefore, a general, intelligent, and personalized craniomaxillofacial defect virtual reconstruction system is desired. To solve this problem, a novel deep learning method, RecGAN, is proposed in this paper. RecGAN can learn the bone morphology of normal people, repair the defect intelligently based on the patient's remaining bone, and fully adapt to the special conditions of different patients. Currently, there are no open-source maxillofacial data sets available. Thus, a new maxillofacial computed tomography image data set with 500 simulated cases and 100 clinical cases is constructed to train and validate the method. The experimental results show that RecGAN can effectively restore the normal bone and tissue morphology of the patient's craniomaxillofacial defect area, solve the problem that there is no objective repair method for craniomaxillofacial defect, and achieve the best effect in the same type of research. The proposed intelligent craniomaxillofacial defect virtual reconstruction system based on RecGAN is expected to be applied in future clinical practice.

The Construction of Biologically Relevant Fiber-Reinforced Hydrogel Geometries Using Air-Assisted Dual-Polarity Electrospinning.

Fiber-reinforced hydrogels are a class of soft composite materials that have seen increased use across a wide variety of biomedical applications. However, existing fabrication techniques for these hydrogels are unable to realize biologically relevant macro/mesoscale geometries. To address this limitation, this paper presents a novel air-assisted, dual-polarity electrospinning printhead that converges high-strength electric fields, with low velocity air flow to remove the collector dependency seen with traditional far-field electrospinning setups. The use of this printhead in conjunction with different configurations of deformable collection templates has resulted in the production of three classes of fiber-reinforced hydrogel prototype geometries, viz., (i) tubular geometries with bifurcations and mesoscale texturing; (ii) hollow, nontubular geometries with single and dual-entrances; and (iii) three-dimensional (3D) printed flat geometries with varying fiber density. All three classes of prototype geometries were mechanically characterized to have properties that were in line with those observed in living soft tissues. With the realization of this printhead, biologically relevant macro/mesoscale geometries can be realized using fiber-reinforced hydrogels to aid a wide array of biomedical applications.

Fully Automated 3D Segmentation and Diffeomorphic Medial Modeling of the Left Ventricle Mitral Valve Complex in Ischemic Mitral Regurgitation.

There is an urgent unmet need to develop a fully-automated image-based left ventricle mitral valve analysis tool to support surgical decision making for ischemic mitral regurgitation patients. This requires an automated tool for segmentation and modeling of the left ventricle and mitral valve from immediate pre-operative 3D transesophageal echocardiography. Previous works have presented methods for semi-automatically segmenting and modeling the mitral valve, but do not include the left ventricle and do not avoid self-intersection of the mitral valve leaflets during shape modeling. In this study, we develop and validate a fully automated algorithm for segmentation and shape modeling of the left ventricular mitral valve complex from pre-operative 3D transesophageal echocardiography. We performed a 3-fold nested cross validation study on two datasets from separate institutions to evaluate automated segmentations generated by nnU-net with the expert manual segmentation which yielded average overall Dice scores of 0.82±0.03 (set A), 0.87±0.08 (set B) respectively. A deformable medial template was subsequently fitted to the segmentation to generate shape models. Comparison of shape models to the manual and automatically generated segmentations resulted in an average Dice score of 0.93-0.94 and 0.75-0.81 for the left ventricle and mitral valve, respectively. This is a substantial step towards automatically analyzing the left ventricle mitral valve complex in the operating room.

Morphology library of nanosilica based on a thermally induced deformable template.

We report a versatile platform for the synthesis of shape-tunable nanosilica based on a thermally induced deformable template with diverse morphologies ranging from spheres, horns, ultrathin nanosheets, and rings to belts. This was realized by creating soft templates from a pair of anionic/cationic surfactants.

Deformable Template Network (DTN) for Object Detection

Objects often have different appearances because of viewpoint changes or part deformation. How to reasonably model these variations is still a big challenge for object detection. In this paper, we propose a novel Deformable Template Network (DTN), which exploits the pictorial structure to model possible variations of an object. DTN represents an object by virtue of a generated template in a deformable way. It has two key modules: the template generating module and the part matching module. The template generating module produces a template for a given object which defines the anchor positions of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k{\times }k$</tex-math></inline-formula> parts. Based on such a template, the part matching module aims to perform part alignment around the anchor positions. In terms of each part, the matching process makes a trade-off between maximizing the detection score and minimizing the deformation cost relative to the anchor position. Moreover, DTN is a fully convolutional network which means it is competitive in terms of detection efficiency. We evaluate DTN on both the PASCAL VOC and MSCOCO datasets, achieving the state-of-the-art results, an accuracy of 82.7% for PASCAL VOC and of 44.9% for MSCOCO.

Automatic landmark annotation in 3D surface scans of skulls: Methodological proposal and reliability study

Background and Objectives: Craniometric landmarks are essential in many biomedical applications, such as morphometric analysis or forensic identification. The process of locating landmarks is usually a manual and slow task, highly influenced by fatigue, skills and the experience of the practitioner. Localization errors are propagated and magnified in subsequent steps, which can result in incorrect measurements or assumptions. Thereby, standardization, reliability and reproducibility lay the foundations for the necessary accuracy in subsequent measurements or anatomical analysis. In this paper, we present an automatic method to annotate 3D surface skull models taking into account anatomical and geometrical features.Methods: The proposed method follows a hybrid structure where a deformable template is used to initialize the landmark positions. Then, a refinement stage is applied using prior anatomical knowledge to ensure a correct placement. Our proposal is validated over thirty 3D skull scans of male Caucasians, acquired by hand-held surface scanning, and a set of 58 craniometric landmarks. A statistical analysis was carried out to analyze the inter- and intra-observer variability of manual annotations and the automatic results, along with a visual assessment of the final results.Results: Inter-observer errors show significant differences, which are reflected in the expert consensus used as reference. The average localization error was 2.19±1.5 mm when comparing the automatic landmarks to the reference location. The subsequent visual analysis confirmed the reliability of the refinement method for most landmarks.Conclusions: Repeated manual annotations show a high variability depending on both skills and expertise of the observer, and landmarks’ location and characteristics. In contrast, the automatic method provides an accurate, robust and reproducible alternative to the tedious and error-prone task of manual landmarking.

Controllable Image Generation with Semi-supervised Deep Learning and Deformable-Mean-Template Based Geometry-Appearance Disentanglement

Typical deep-neural-network (DNN) based generative image models often (i) show limited ability to learn a disentangled latent representation, (ii) show limited controllability leading to undesirable side effects when manipulating selected attributes during image generation, and (iii) require large attribute-annotated training sets. We propose a generative DNN model for face images by explicitly disentangling geometry and appearance modeling to achieve selective controllability of the desired attributes with less side effects. To learn geometric variability, we leverage grayscale sketch representations to learn (i) a deformable mean template representing the population-mean face geometry and (ii) a generative model of deformations to model individual face-geometry variations, using dense image registration. We learn the appearance variability in a (color-image) space that we explicitly design by factoring out the geometric variability. We propose a variational formulation to enable semi-supervised learning when manually-annotated attributes are severely limited in the training set. Results on large datasets show that, compared to schemes using deformation models or variational learning, our method significantly improves face-image model fits and facial-feature controllability even with semi-supervised learning.

Robot visual measurement and grasping strategy for roughcastings

To overcome the challenging problem of visual measurement and grasping of roughcasts, a visual grasping strategy for an industrial robot is designed and implemented on the basis of deep learning and a deformable template matching algorithm. The strategy helps realize the positioning recognition and grasping guidance for a metal blank cast in complex backgrounds under the interference of external light. The proposed strategy has two phases: target detection and target localization. In the target detection stage, a deep learning algorithm is used to recognize the combined features of the surface of an object for a stable recognition of the object in nonstructured environments. In the target localization stage, high-precision positioning of metal casts with an unclear contour is realized by combining the deformable template matching and LINE-MOD algorithms. The experimental results show that the system can accurately provide visual grasping guidance for robots.

Modality-Constrained Density Estimation via Deformable Templates

Estimation of a probability density function (pdf) from its samples, while satisfying certain shape constraints, is an important problem that lacks coverage in the literature. This article introduces a novel geometric, deformable template constrained density estimator (dtcode) for estimating pdfs constrained to have a given number of modes. Our approach explores the space of thus-constrained pdfs using the set of shape-preserving transformations: an arbitrary template from the given shape class is transformed via a shape-preserving transformation to obtain the final optimal estimate. The search for this optimal transformation, under the maximum-likelihood criterion, is performed by mapping transformations to the tangent space of a Hilbert sphere, where they are effectively linearized, and can be expressed using an orthogonal basis. This framework is first applied to (univariate) unconditional densities and then extended to conditional densities. We provide asymptotic convergence rates for dtcode, and an application of the framework to the speed distributions for different traffic flows on Californian highways.

Statistical shape model-based planning organ-at-risk volume: application to pancreatic cancer patients

Purpose. To introduce the concept of statistical shape model (SSM)-based planning organ-at-risk volume (sPRV) for pancreatic cancer patients. Methods. A total of 120 pancreatic cancer patients were enrolled in this study. After correcting inter-patient variations in the centroid position of the planning target volume (PTV), four different SSMs were constructed by registering a deformable template model to an individual model for the stomach and duodenum. The sPRV, which focused on the following different components of the inter-patient variations, was then created: Scenario A: shape, rotational angle, volume, and centroid position; Scenario B: shape, rotational angle, and volume; Scenario C: shape and rotational angle; and Scenario D: shape. The conventional PRV (cPRV) was created by adding an isotropic margin R (3–15 mm) to the mean shape model. The corresponding sPRV was created from the SSM until the volume difference between the cPRV and sPRV was less than 1%. Thereafter, we computed the overlapping volume between the PTV and cPRV (OLc) or sPRV (OLs) in each patient. OLs being larger than OLc implies that the local shape variations in the corresponding OAR close to the PTV were large. Therefore, OLs/OLc was calculated in each patient for each R-value, and the median value of OLs/OLc was regarded as a surrogate for plan quality for each R-value. Results. For R = 3 and 5 mm, OLs/OLc exceeded 1 for the stomach and duodenum in all scenarios, with a maximum OLs/OLc of 1.21. This indicates that smaller isotropic margins did not sufficiently account for the local shape changes close to the PTV. Conclusions. Our results indicated that, in contrast to conventional PRV, SSM-based PRVs, which account for local shape changes, would result in better dose sparing for the stomach and duodenum in pancreatic cancer patients.

Advance Techniques in Medical Imaging under Big Data Analysis: Covid-19 Images

Quantitative analysis of digital images requires detection and segmentation of the borders of the object of interest. Accurate segmentation is required for volume determination, 3D rendering, radiation therapy, and surgery planning. In medical images, segmentation has traditionally been done by human experts. Substantial computational and storage requirements become especially acute when object orientation and scale have to be considered. Therefore, automated or semi-automated segmentation techniques are essential if these software applications are ever to gain widespread clinical use. Many methods have been proposed to detect and segment 2D shapes, most of which involve template matching. Advanced segmentation techniques called Snakes or active contours have been used, considering deformable models or templates. The main purpose of this work is to apply segmentation techniques for the definition of 3D organs (anatomical structures) when big data information has been stored and must be organized by the doctors for medical diagnosis. The processes would be implemented in the CT images from patients with COVID-19.

Learning Spatiotemporal Probabilistic Atlas of Fetal Brains with Anatomically Constrained Registration Network.

Brain atlases are of fundamental importance for analyzing the dynamic neurodevelopment in fetal brain studies. Since the brain size, shape, and anatomical structures change rapidly during the prenatal period, it is essential to construct a spatiotemporal (4D) atlas equipped with tissue probability maps, which can preserve sharper early brain folding patterns for accurately characterizing dynamic changes in fetal brains and provide tissue prior informations for related tasks, e.g., segmentation, registration, and parcellation. In this work, we propose a novel unsupervised age-conditional learning framework to build temporally continuous fetal brain atlases by incorporating tissue segmentation maps, which outperforms previous traditional atlas construction methods in three aspects. First, our framework enables learning age-conditional deformable templates by leveraging the entire collection. Second, we leverage reliable brain tissue segmentation maps in addition to the low-contrast noisy intensity images to enhance the alignment of individual images. Third, a novel loss function is designed to enforce the similarity between the learned tissue probability map on the atlas and each subject tissue segmentation map after registration, thereby providing extra anatomical consistency supervision for atlas building. Our 4D temporally-continuous fetal brain atlases are constructed based on 82 healthy fetuses from 22 to 32 gestational weeks. Compared with the atlases built by the state-of-the-art algorithms, our atlases preserve more structural details and sharper folding patterns. Together with the learned tissue probability maps, our 4D fetal atlases provide a valuable reference for spatial normalization and analysis of fetal brain development.

Pupil detection and gaze tracking using a deformable template

This paper suggests a method for tracking gaze of a person at a distance around 2 m, using a single pan-tilt-zoom (PTZ) camera. In the suggested method, images are acquired for gaze tracking by turning the camera to the wide angle mode, or the narrow angle mode, depending on the location of the person. The face that is present in the field of view (FOV) of a camera, is detected in the wide angle mode. Once the location of the face is calculated, the camera turns to the narrow angle mode. The images, which have been acquired in the narrow angle mode, contain information on the direction of gaze of the person, who is at a distance. The method for calculating the direction of gaze is comprised of the head pose estimation and gaze direction calculation steps. The head pose estimation is performed using the location information on the eyes and nose in the face. The direction of gaze is generated using the process of partitioning the pupil through a deformable template, and extracting the center of an eye using the end points of both eyes and head pose information. This paper shows that the proposed gaze tracking algorithm can effectively track the direction of a person’s gaze, at varying distances.

Timely-Automatic Procedure for Estimating the Endocardial Limits of the Left Ventricle Assessed Echocardiographically in Clinical Practice

In this paper, we propose an analytical rapid method to estimate the endocardial borders of the left ventricular walls on echocardiographic images for prospective clinical integration. The procedure was created as a diagnostic support tool for the clinician and it is based on the use of the anisotropic generalized Hough transform. Its application is guided by a Gabor-like filtering for the approximate delimitation of the region of interest without the need for computing further anatomical characteristics. The algorithm is applying directly a deformable template on the predetermined filtered region and therefore it is responsive and straightforward implementable. For accuracy considerations, we have employed a support vector machine classifier to determine the confidence level of the automated marking. The clinical tests were performed at the Cardiology Clinic of the County Emergency Hospital Timisoara and they improved the physicians perception in more than 50% of the cases. The report is concluded with medical discussions.

Multilevel principal components analysis of three-dimensional facial growth in adolescents

Background and objectivesThe study of age-related facial shape changes across different populations and sexes requires new multivariate tools to disentangle different sources of variations present in 3D facial images. Here we wish to use a multivariate technique called multilevel principal components analysis (mPCA) to study three-dimensional facial growth in adolescents. MethodsThese facial shapes were captured for Welsh and Finnish subjects (both male and female) at multiple ages from 12 to 17 years old (i.e., repeated-measures data). 1000 “dense” 3D points were defined regularly for each shape by using a deformable template via “meshmonk” software. A three-level model was used here, namely: level 1 (sex/ethnicity); level 2, all “subject” variations excluding sex, ethnicity, and age; and level 3, age. The technicalities underpinning the mPCA method are presented in Appendices. ResultsEigenvalues via mPCA predicted that: level 1 (ethnicity/sex) contained 7.9% of variation; level 2 contained 71.5%; and level 3 (age) contained 20.6%. The results for the eigenvalues via mPCA followed a similar pattern to those results of single-level PCA. Results for modes of variation made sense, where effects due to ethnicity, sex, and age were reflected in modes at appropriate levels of the model. Standardised scores at level 1 via mPCA showed much stronger differentiation between sex and ethnicity groups than results of single-level PCA. Results for standardised scores from both single-level PCA and mPCA at level 3 indicated that females had different average “trajectories” with respect to these scores than males, which suggests that facial shape matures in different ways for males and females. No strong evidence of differences in growth patterns between Finnish and Welsh subjects was observed. ConclusionsmPCA results agree with existing research relating to the general process of facial changes in adolescents with respect to age quoted in the literature. They support previous evidence that suggests that males demonstrate larger changes and for a longer period of time compared to females, especially in the lower third of the face. These calculations are therefore an excellent initial test that multivariate multilevel methods such as mPCA can be used to describe such age-related changes for “dense” 3D point data.

P01.091 A robust method for rapidly simulating TTFields distributions within patient-specific computational head models

Tumor Treating Fields (TTFields) are delivered to the brain through two pairs of transducer arrays placed on the patient’s scalp. TTFields distribution in the brain depends on the array’s position, patient anatomy and the electric properties of the tissues and tumor. We aim to create realistic computational patient models in order to understand how field distribution influences disease progression. However, preparation of such models is a highly time consuming task. In addition, in clinical scenarios, MRI acquisition time is often reduced by increased slice spacing, limited field of view, or increased scan speed, leading to difficulties in creating automated models. Our method is designed to enable model creation under those restrictions. It does so by using a realistic head model of a healthy individual used as a deformable template with which the patient model is derived. A highly detailed healthy head model serves as a deformable template from which patient models are created. The first step is pre-processing, involving denoising and background noise reduction, as well as super-resolution algorithms when needed. To create the patient model, the tumor is first segmented manually and masked, leaving only healthy tissues in the MRI, which is then registered to the template space to yield the transformation from patient space to template space. The template is then deformed into the patient space using the inverse transformation, and the tumor is placed back creating a full patient model. Next, automatic identification of landmarks on the patient’s head is used to position the transducer arrays on the head, which are then introduced into the model. Finally, boundary conditions are set, and field distribution is simulated using Finite Differences Time Domain (FDTD) method (Sim4Life V3.0, ZMT-Zurich). We have simulated TTFields distribution of 317 patients treated with TTFields as part of the EF-14 trial. Our method is optimized for accurate contouring of tissues highly influencing the distribution of electric field (Scalp, skull, CSF, ventricles), and it is robust, having the ability to give sufficient results even when MRI data quality is low. Thus enabling a study correlating the spatial distribution of TTFields and patient outcome. Our process for rapidly creating patient models presents a breakthrough that enables the first study in which the spatial distribution of therapeutic electric fields correlates with patient outcome. In the future this method can be used for clinical studies investigating other clinical indications of large datasets.