Non-rigid Registration Algorithm Research Articles

A model that predicts a real-time tumour surface using intra-treatment skin surface and end-of-expiration and end-of-inhalation planning CT images.

To develop a mapping model between skin surface motion and internal tumour motion and deformation using end-of-exhalation (EOE) and end-of-inhalation (EOI) 3D CT images for tracking lung tumours during respiration. Before treatment, skin and tumour surfaces were segmented and reconstructed from the EOE and the EOI 3D CT images. A non-rigid registration algorithm was used to register the EOE skin and tumour surfaces to the EOI, resulting in a displacement vector field that was then used to construct a mapping model. During treatment, the EOE skin surface was registered to the real-time, yielding a real-time skin surface displacement vector field. Using the mapping model generated, the input of a real-time skin surface can be used to calculate the real-time tumour surface. The proposed method was validated with and without simulated noise on 4D CT images from 15 patients at Léon Bérard Cancer Center and the 4D-lung dataset. The average centre position error, dice similarity coefficient (DSC), 95%-Hausdorff distance and mean distance to agreement of the tumour surfaces were 1.29 mm, 0.924, 2.76 mm, and 1.13 mm without simulated noise, respectively. With simulated noise, these values were 1.33 mm, 0.920, 2.79 mm, and 1.15 mm, respectively. A patient-specific model was proposed and validated that was constructed using only EOE and EOI 3D CT images and real-time skin surface images to predict internal tumour motion and deformation during respiratory motion. The proposed method achieves comparable accuracy to state-of-the-art methods with fewer pre-treatment planning CT images, which holds potential for application in precise image-guided radiation therapy.

Three-Dimensional Virtual Reconstruction of External Nasal Defects Based on Facial Mesh Generation Network.

(1) Background: In digital-technology-assisted nasal defect reconstruction methods, a crucial step involves utilizing computer-aided design to virtually reconstruct the nasal defect's complete morphology. However, current digital methods for virtual nasal defect reconstruction have yet to achieve efficient, precise, and personalized outcomes. In this research paper, we propose a novel approach for reconstructing external nasal defects based on the Facial Mesh Generation Network (FMGen-Net), aiming to enhance the levels of automation and personalization in virtual reconstruction. (2) Methods: We collected data from 400 3D scans of faces with normal morphology and combined the structured 3D face template and the Meshmonk non-rigid registration algorithm to construct a structured 3D facial dataset for training FMGen-Net. Guided by defective facial data, the trained FMGen-Net automatically generated an intact 3D face that was similar to the defective face, and maintained a consistent spatial position. This intact 3D face served as the 3D target reference face (3D-TRF) for nasal defect reconstruction. The reconstructed nasal data were extracted from the 3D-TRF based on the defective area using reverse engineering software. The '3D surface deviation' between the reconstructed nose and the original nose was calculated to evaluate the effect of 3D morphological restoration of the nasal defects. (3) Results: In the simulation experiment of 20 cases involving full nasal defect reconstruction, the '3D surface deviation' between the reconstructed nasal data and the original nasal data was 1.45 ± 0.24 mm. The reconstructed nasal data, constructed from the personalized 3D-TRF, accurately reconstructed the anatomical morphology of nasal defects. (4) Conclusions: This paper proposes a novel method for the virtual reconstruction of external nasal defects based on the FMGen-Net model, achieving the automated and personalized construction of the 3D-TRF and preliminarily demonstrating promising clinical application potential.

Prostate volume analysis in image registration for prostate cancer care: a verification study

Combined magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) may enhance diagnosis, aid surgical planning and intra-operative orientation for prostate biopsy and radical prostatectomy. Although PET-MRI may provide these benefits, PET-MRI machines are not widely available. Image fusion of Prostate specific membrane antigen PET/CT and MRI acquired separately may be a suitable clinical alternative. This study compares CT-MR registration algorithms for urological prostate cancer care. Paired whole-pelvis MR and CT scan data were used (n = 20). A manual prostate CTV contour was performed independently on each patients MR and CT image. A semi-automated rigid-, automated rigid- and automated non-rigid registration technique was applied to align the MR and CT data. Dice Similarity Index (DSI), 95% Hausdorff distance (95%HD) and average surface distance (ASD) measures were used to assess the closeness of the manual and registered contours. The automated non-rigid approach had a significantly improved performance compared to the automated rigid- and semi-automated rigid-registration, having better average scores and decreased spread for the DSI, 95%HD and ASD (all p < 0.001). Additionally, the automated rigid approach had similar significantly improved performance compared to the semi-automated rigid registration across all accuracy metrics observed (all p < 0.001). Overall, all registration techniques studied here demonstrated sufficient accuracy for exploring their clinical use. While the fully automated non-rigid registration algorithm in the present study provided the most accurate registration, the semi-automated rigid registration is a quick, feasible, and accessible method to perform image registration for prostate cancer care by urologists and radiation oncologists now.

Enabling supra-aortic vessels inclusion in statistical shape models of the aorta: a novel non-rigid registration method.

Statistical Shape Models (SSMs) are well-established tools for assessing the variability of 3D geometry and for broadening a limited set of shapes. They are widely used in medical imaging due to their ability to model complex geometries and their high efficiency as generative models. The principal step behind these techniques is a registration phase, which, in the case of complex geometries, can be a critical issue due to the correspondence problem, as it necessitates the development of correspondence mapping between shapes. The thoracic aorta, with its high level of morphological complexity, poses a multi-scale deformation problem due to the presence of several branch vessels with varying diameters. Moreover, branch vessels exhibit significant variability in shape, making the correspondence optimization even more challenging. Consequently, existing studies have focused on developing SSMs based only on the main body of the aorta, excluding the supra-aortic vessels from the analysis. In this work, we present a novel non-rigid registration algorithm based on optimizing a differentiable distance function through a modified gradient descent approach. This strategy enables the inclusion of custom, domain-specific constraints in the objective function, which act as landmarks during the registration phase. The algorithm's registration performance was tested and compared to an alternative Statistical Shape modeling framework, and subsequently used for the development of a comprehensive SSM of the thoracic aorta, including the supra-aortic vessels. The developed SSM was further evaluated against the alternative framework in terms of generalisation, specificity, and compactness to assess its effectiveness.

Hybrid Eulerian–Lagrangian framework for structural full-field vibration quantification and modal shape visualization

Vibration measurement and modal analysis under a high spatial and temporal resolution have high significance for revealing inner structural characteristics. This work develops a hybrid Eulerian–Lagrangian framework for structural full-field dynamic vibration quantification and modal shape visualization and validates its performance through vision-based vibration experiments. By assuming a linear vibration system, modal responses that are estimated from the vibration signals on a structural surface provide a reliable basis for manipulating temporal and spatial vibrations in Eulerian motion processing. Then, dynamic spatial motion can be quantified by using the improved non-rigid image registration algorithm. Meanwhile, from the Lagrangian perspective, high-quality visualization of modal shapes is implemented by compensating spatial motion according to magnified dense displacement fields. The experiment results showed that the proposed hybrid framework has superiority in controlled modal tests and has potential for more practical applications.

Non-rigid registration for medical images based on deformable convolution and multi-scale feature focusing modules

Non-rigid registration plays an important role in medical image analysis. U-Net has been proven to be a hot research topic in medical image analysis and is widely used in medical image registration. However, existing registration models based on U-Net and its variants lack sufficient learning ability when dealing with complex deformations, and do not fully utilize multi-scale contextual information, resulting insufficient registration accuracy. To address this issue, a non-rigid registration algorithm for X-ray images based on deformable convolution and multi-scale feature focusing module was proposed. First, it used residual deformable convolution to replace the standard convolution of the original U-Net to enhance the expression ability of registration network for image geometric deformations. Then, stride convolution was used to replace the pooling operation of the downsampling operation to alleviate feature loss caused by continuous pooling. In addition, a multi-scale feature focusing module was introduced to the bridging layer in the encoding and decoding structure to improve the network model's ability of integrating global contextual information. Theoretical analysis and experimental results both showed that the proposed registration algorithm could focus on multi-scale contextual information, handle medical images with complex deformations, and improve the registration accuracy. It is suitable for non-rigid registration of chest X-ray images.

Study on the method of automatically determining maxillary complex landmarks based on non-rigid registration algorithms

Objective: To explore an automatic landmarking method for anatomical landmarks in the three-dimensional (3D) data of the maxillary complex and preliminarily evaluate its reproducibility and accuracy. Methods: From June 2021 to December 2022, spiral CT data of 31 patients with relatively normal craniofacial morphology were selected from those who visited the Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology. The sample included 15 males and 16 females, with the age of (33.3±8.3) years. The maxillary complex was reconstructed in 3D using Mimics software, and the resulting 3D data of the maxillary complex was mesh-refined using Geomagic software. Two attending physicians and one associate chief physician manually landmarked the 31 maxillary complex datasets, determining 24 anatomical landmarks. The average values of the three expert landmarking results were used as the expert-defined landmarks. One case that conformed to the average 3D morphological characteristics of healthy individuals' craniofacial bones was selected as the template data, while the remaining 30 cases were used as target data. The open-source MeshMonk program (a non-rigid registration algorithm) was used to perform an initial alignment of the template and target data based on 4 landmarks (nasion, left and right zygomatic arch prominence, and anterior nasal spine). The template data was then deformed to the shape of the target data using a non-rigid registration algorithm, resulting in the deformed template data. Based on the unchanged index property of homonymous landmarks before and after deformation of the template data, the coordinates of each landmark in the deformed template data were automatically retrieved as the automatic landmarking coordinates of the homonymous landmarks in the target data, thus completing the automatic landmarking process. The automatic landmarking process for the 30 target data was repeated three times. The root-mean-square distance (RMSD) of the dense corresponding point pairs (approximately 25 000 pairs) between the deformed template data and the target data was calculated as the deformation error of the non-rigid registration algorithm, and the intra-class correlation efficient (ICC) of the deformation error in the three repetitions was analyzed. The linear distances between the automatic landmarking results and the expert-defined landmarks for the 24 anatomical landmarks were calculated as the automatic landmarking errors, and the ICC values of the 3D coordinates in the three automatic landmarking repetitions were analyzed. Results: The average three-dimensional deviation (RMSD) between the deformed template data and the corresponding target data for the 30 cases was (0.70±0.09) mm, with an ICC value of 1.00 for the deformation error in the three repetitions of the non-rigid registration algorithm. The average automatic landmarking error for the 24 anatomical landmarks was (1.86±0.30) mm, with the smallest error at the anterior nasal spine (0.65±0.24) mm and the largest error at the left oribital (3.27±2.28) mm. The ICC values for the 3D coordinates in the three automatic landmarking repetitions were all 1.00. Conclusions: This study established an automatic landmarking method for three-dimensional data of the maxillary complex based on a non-rigid registration algorithm. The accuracy and repeatability of this method for landmarking normal maxillary complex 3D data were relatively good.

Preliminary study on three-dimensional morphological reconstruction method for external nose defect based on three-dimensional face template

Objective: To provide a new solution for the digital design of nasal prostheses, this study explores the three-dimensional (3D) facial morphology completion method for external nasal defects based on the non-rigid registration process of 3D face template. Methods: A total of 20 male patients with tooth defect and dentition defect who visited the Department of Prosthodontics, Peking University School and Hospital of Stomatology from June to December 2022 were selected, age 18-45 years old. The original 3D facial data of patients were collected, and the 3D facial data of the external nose defect was constructed in Geomagic Wrap 2021 software. Using the structured 3D face template data constructed in the previous research of the research group, the 3D face template was deformed and registered to the 3D facial data of external nose defect (based on the morphology of non-defective area) by non-rigid registration algorithm (MeshMonk program), and the personalized deformed data of the 3D face template was obtained, as the complemented facial 3D data. Based on the defect boundary of the 3D facial data of the external nose defect, the complemented external nose 3D data can be cut out from the complemented facial 3D data. Then the nasofacial angle and nasolabial angle of the complemented facial 3D data and the original 3D facial data was compared and analyzed, the ratio between the nose length and mid-face height, nose width and medial canthal distance of the complemented facial 3D data was measured, the edge fit between the edge curve of the complemented external nose 3D data and the defect edge curve of the 3D facial data of external nose defect was evaluated, and the morphological difference of the nose between the complemented external nose 3D data and the original 3D facial data was analyzed. Results: There was no significant statistically difference (t=-0.23, P=0.823; Z=-1.72, P=0.086) in the nasofacial angle (28.2°±2.9°, 28.4°±3.5° respectively) and nasolabial angle [95.4°(19.2°), 99.9°(9.5°) respectively] between the 20 original 3D facial data and the complemented facial 3D data. The value of the ratio of nose length to mid-face height in the complemented facial 3D data was 0.63±0.03, and the value of the ratio of nose width to medial canthal distance was 1.07±0.08. The curve deviation (root mean square value) between the edge curve of the complemented external nose 3D data and the defect edge curve of the 3D facial data of external nose defect was (0.37±0.09) mm, the maximum deviation was (1.14±0.32) mm, and the proportion of the curve deviation value within±1 mm was (97±3)%. The distance of corresponding nose landmarks between the complemented facial 3D data and the original 3D facial data were respectively, Nasion: [1.52(1.92)] mm; Pronasale: (3.27±1.21) mm; Subnasale: (1.99±1.09) mm; Right Alare: (2.64±1.34) mm; Left Alare: (2.42± 1.38) mm. Conclusions: The method of 3D facial morphology completion of external nose defect proposed in this study has good feasibility. The constructed complemented external nose 3D data has good facial coordination and edge fit, and the morphology is close to the nose morphology of the original 3D facial data.

Comparison Study of Extraction Accuracy of 3D Facial Anatomical Landmarks Based on Non-Rigid Registration of Face Template

(1) Background: Three-dimensional (3D) facial anatomical landmarks are the premise and foundation of facial morphology analysis. At present, there is no ideal automatic determination method for 3D facial anatomical landmarks. This research aims to realize the automatic determination of 3D facial anatomical landmarks based on the non-rigid registration algorithm developed by our research team and to evaluate its landmark localization accuracy. (2) Methods: A 3D facial scanner, Face Scan, was used to collect 3D facial data of 20 adult males without significant facial deformities. Using the radial basis function optimized non-rigid registration algorithm, TH-OCR, developed by our research team (experimental group: TH group) and the non-rigid registration algorithm, MeshMonk (control group: MM group), a 3D face template constructed in our previous research was deformed and registered to each participant's data. The automatic determination of 3D facial anatomical landmarks was realized according to the index of 32 facial anatomical landmarks determined on the 3D face template. Considering these 32 facial anatomical landmarks manually selected by experts on the 3D facial data as the gold standard, the distance between the automatically determined and the corresponding manually selected facial anatomical landmarks was calculated as the "landmark localization error" to evaluate the effect and feasibility of the automatic determination method (template method). (3) Results: The mean landmark localization error of all facial anatomical landmarks in the TH and MM groups was 2.34 ± 1.76 mm and 2.16 ± 1.97 mm, respectively. The automatic determination of the anatomical landmarks in the middle face was better than that in the upper and lower face in both groups. Further, the automatic determination of anatomical landmarks in the center of the face was better than in the marginal part. (4) Conclusions: In this study, the automatic determination of 3D facial anatomical landmarks was realized based on non-rigid registration algorithms. There is no significant difference in the automatic landmark localization accuracy between the TH-OCR algorithm and the MeshMonk algorithm, and both can meet the needs of oral clinical applications to a certain extent.

Attenuation Correction for Dedicated Cardiac SPECT Imaging Without Using Transmission Data.

Attenuation correction (AC) using transmission scanning-like computed tomography (CT) is the standard method to increase the accuracy of cardiac single-photon emission computed tomography (SPECT) images. Recently developed dedicated cardiac SPECT do not support CT, and thus, scans on these systems are vulnerable to attenuation artifacts. This study presented a new method for generating an attenuation map directly from emission data by segmentation of precisely non-rigid registration extended cardiac-torso (XCAT)-digital phantom with cardiac SPECT images. In-house developed non-rigid registration algorithm automatically aligns the XCAT- phantom with cardiac SPECT image to precisely segment the contour of organs. Pre-defined attenuation coefficients for given photon energies were assigned to generate attenuation maps. The CT-based attenuation maps were used for validation with which cardiac SPECT/CT data of 38 patients were included. Segmental myocardial counts of a 17-segment model from these databases were compared based on the basis of the paired t-test. The mean, and standard deviation of the mean square error and structural similarity index measure of the female stress phase between the proposed attenuation maps and the CT attenuation maps were 6.99±1.23% and 92±2.0%, of the male stress were 6.87±3.8% and 96±1.0%. Proposed attenuation correction and computed tomography based attenuation correction average myocardial perfusion count was significantly higher than that in non-AC in the mid-inferior, mid-lateral, basal-inferior, and lateral regions (p<0.001). The proposed attenuation maps showed good agreement with the CT-based attenuation map. Therefore, it is feasible to enable AC for a dedicated cardiac SPECT or SPECT standalone scanners.

Modeling of Respiratory Motion to Support the Minimally Invasive Destruction of Liver Tumors.

Objective: Respiratory movements are a significant factor that may hinder the use of image navigation systems during minimally invasive procedures used to destroy focal lesions in the liver. This article aims to present a method of estimating the displacement of the target point due to respiratory movements during the procedure, working in real time. Method: The real-time method using skin markers and non-rigid registration algorithms has been implemented and tested for various classes of transformation. The method was validated using clinical data from 21 patients diagnosed with liver tumors. For each patient, each marker was treated as a target and the remaining markers as target position predictors, resulting in 162 configurations and 1095 respiratory cycles analyzed. In addition, the possibility of estimating the respiratory phase signal directly from intraoperative US images and the possibility of synchronization with the 4D CT respiratory sequence are also presented, based on ten patients. Results: The median value of the target registration error (TRE) was 3.47 for the non-rigid registration method using the combination of rigid transformation and elastic body spline curves, and an adaptation of the assessing quality using image registration circuits (AQUIRC) method. The average maximum distance was 3.4 (minimum: 1.6, maximum 6.8) mm. Conclusions: The proposed method obtained promising real-time TRE values. It also allowed for the estimation of the TRE at a given geometric margin level to determine the estimated target position. Directions for further quantitative research and the practical possibility of combining both methods are also presented.

Construction of damage-free digital twin of damaged aero-engine blades for repair volume generation in remanufacturing

Accurate repair volume generation from 3D scan data of damaged aero-engine blades is of great importance in additive or hybrid remanufacturing for restoring the blades to a like-new condition. In addition to material-missing damages, the blade's surface also deforms due to working in harsh environments, which makes it deviate from the nominal design geometry. Therefore, the Boolean operation between the nominal CAD model and the scanned point cloud of the damaged blade does not yield an accurate repair volume. This paper presents a new methodology to construct an accurate damage-free digital twin model of the defective blades that contains the deformations of the blade's undamaged regions. The Boolean difference between the scan data of the damaged blade and its damage-free digital twin yields the repair volume with a smooth geometric transition at the interface of the repair patch and unrepaired regions. At first, the data points of damaged regions of the blade surface are detected and eliminated from the scan through a region growing segmentation. Then, a CAD-to-scan non-rigid registration algorithm deforms the nominal CAD model of the blade to best match it to the scanned point cloud in the undamaged regions. The non-rigid registration algorithm iteratively minimizes the distance between two datasets under the local rigidity constraint to avoid shrinkage and expansion of the deformed CAD model. A constrained point-to-surface weighted correspondence search method is proposed to reduce the influence of noise and unreliable correspondences on the non-rigid registration. The results of numerical and experimental case studies have demonstrated that the proposed method is accurate and robust to noise, and it can be effectively applied to construct a damage-free digital twin model for repair volume generation.

Preliminary study on the method of automatically determining facial landmarks based on three-dimensional face template

Objective: To explore the establishment of an efficient and automatic method to determine anatomical landmarks in three-dimensional (3D) facial data, and to evaluate the effectiveness of this method in determining landmarks. Methods: A total of 30 male patients with tooth defect or dentition defect (with good facial symmetry) who visited the Department of Prosthodontics, Peking University School and Hospital of Stomatology from June to August 2021 were selected, and these participants' age was between 18-45 years. 3D facial data of patients was collected and the size normalization and overlap alignment were performed based on the Procrustes analysis algorithm. A 3D face average model was built in Geomagic Studio 2013 software, and a 3D face template was built through parametric processing. MeshLab 2020 software was used to determine the serial number information of 32 facial anatomical landmarks (10 midline landmarks and 22 bilateral landmarks). Five male patients with no mandibular deviation and 5 with mild mandibular deviation were selected from the Department of Orthodontics or Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology from June to August 2021. 3D facial data of patients was collected as test data. Based on the 3D face template and the serial number information of the facial anatomical landmarks, the coordinates of 32 facial anatomical landmarks on the test data were automatically determined with the help of the MeshMonk non-rigid registration algorithm program, as the data for the template method to determine the landmarks. The positions of 32 facial anatomical landmarks on the test data were manually determined by the same attending physician, and the coordinates of the landmarks were recorded as the data for determining landmarks by the expert method. Calculated the distance value of the coordinates of facial anatomical landmarks between the template method and the expert method, as the landmark localization error, and evaluated the effect of the template method in determining the landmarks. Results: For 5 patients with no mandibular deviation, the landmark localization error of all facial anatomical landmarks by template method was (1.65±1.19) mm, the landmark localization error of the midline facial anatomical landmarks was (1.19±0.45) mm, the landmark localization error of bilateral facial anatomical landmarks was (1.85±1.33) mm. For 5 patients with mild mandibular deviation, the landmark localization error of all facial anatomical landmarks by template method was (2.55±2.22) mm, the landmark localization error of the midline facial anatomical landmarks was (1.85±1.13) mm, the landmark localization error of bilateral facial anatomical landmarks was (2.87±2.45) mm. Conclusions: The automatic determination method of facial anatomical landmarks proposed in this study has certain feasibility, and the determination effect of midline facial anatomical landmarks is better than that of bilateral facial anatomical landmarks. The effect of determining facial anatomical landmarks in patients without mandibular deviation is better than that in patients with mild mandibular deviation.

Geodesic-Based Bayesian Coherent Point Drift.

Coherent point drift is a well-known algorithm for non-rigid registration, i.e., a procedure for deforming a shape to match another shape. Despite its prevalence, the algorithm has a major drawback that remains unsolved: It unnaturally deforms the different parts of a shape, e.g., human legs, when they are neighboring each other. The inappropriate deformations originate from a proximity-based deformation constraint, called motion coherence. This study proposes a non-rigid registration method that addresses the drawback. The key to solving the problem is to redefine the motion coherence using a geodesic, i.e., the shortest route between points on a shape's surface. We also propose the accelerated variant of the registration method. In numerical studies, we demonstrate that the algorithms can circumvent the drawback of coherent point drift. We also show that the accelerated algorithm can be applied to shapes comprising several millions of points.

Non-Rigid Registration Algorithm of Lung Computed Tomography Image Based on Multi-Scale Parallel Fully Convolutional Neural Network

Non-Rigid Registration Algorithm of Lung Computed Tomography Image Based on Multi-Scale Parallel Fully Convolutional Neural Network

Ultrasound strain mapping of the mouse Achilles tendon during passive dorsiflexion

Immediately prior to inserting into bone, many healthy tendons experience impingement from nearby bony structures. However, super-physiological levels of impingement are implicated in insertional tendinopathies. Unfortunately, the mechanisms underlying the connection between impingement and tendon pathology remain poorly understood, in part due to the shortage of well-characterized animal models of impingement at clinically relevant sites. As a first step towards developing a model of excessive tendon impingement, the objective of this study was to characterize the mechanical strain environment in the mouse Achilles tendon insertion under passive dorsiflexion and confirm that – like humans – mice experience impingement of the tendon insertion from the calcaneus (heel bone) in dorsiflexed ankle positions. Based on previous work in humans, we hypothesized that during dorsiflexion, the mouse Achilles tendon insertion would experience high levels of transverse compressive strain due to calcaneal impingement. A custom-built loading platform was used to apply passive dorsiflexion, while an ultrasound transducer positioned over the Achilles tendon captured radiofrequency images. A non-rigid image registration algorithm was then used to map the transverse compressive strain based on the acquired ultrasound image sequences. Our results demonstrate that during passive dorsiflexion, transverse compressive strains were produced throughout the Achilles tendon, with significantly larger strain magnitudes at the tendon insertion than at the midsubstance. Furthermore, there was increasing transverse compressive strain observed within the Achilles tendon as a function of increasing dorsiflexion angle. This study enhances our understanding of the unique mechanical loading environment of the Achilles tendon under physiologically relevant conditions.

Nonrigid Registration Algorithm in the Face Recognition Problem

Nonrigid Registration Algorithm in the Face Recognition Problem

Real-time multimodal image registration with partial intraoperative point-set data.

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial.

Robust and Accurate Nonrigid Point Set Registration Algorithm to Accommodate Anisotropic Positional Localization Error Based on Coherent Point Drift

Nonrigid point set (PS) registration is an outstanding and fundamental problem in the fields of robotics, computer vision, medical image analysis, and image-guided surgery (IGS). The aim of a nonrigid registration problem is to align together two point sets where one has been deformed. The assumption of isotropic localization error is shared in the previous nonrigid registration algorithms. In this article, we have derived and presented a novel nonrigid registration algorithm, where the position localization error (PLE) is generalized to be anisotropic, which means that the error distribution is not the same in different spatial directions. The motivation of considering the anisotropic characteristic is that the PLE is actually different in three spatial directions in real applications of registrations, such as IGS. Mathematically, the difficulty in dealing with the anisotropic error case comes from the change from a standard deviation that is a scalar to a covariance matrix. The formulas for updating the parameters in both expectation and maximization steps are derived. More specifically, in the expectation step, we compute the posterior probabilities that represent the correspondences between points in two PSs. In the maximization step, given the current posteriors, the covariance matrix of the PLE and the nonrigid transformation are updated. To further speed up the proposed algorithm, the low-rank approximation variation of our method is also presented. We have demonstrated through experiments on both general and medical data sets (corrupted with noise) that the proposed algorithm outperforms the state-of-the-art ones in terms of registration accuracy and robustness to noise. More specifically, all the experimental results have passed the statistical tests at the 5% significance level. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by solving the problem of nonrigidly registering two point sets where one has been deformed and corrupted with anisotropic noise. Most existing registration methods generally assume the positional error to be the same in all directions, which in fact is not the case in real scenarios. This article presents a new robust method that assumes the positional error to be anisotropic, which is the case in point sets coming from the stereo reconstruction. The nonrigid registration problem is formulated as a maximum-likelihood (ML) problem and solved with the expectation–maximization (EM) technique. We have demonstrated through extensive experiments on both general and medical data sets that the proposed registration algorithm achieves significantly improved accuracy, robustness to noise, and outliers compared with state-of-the-art algorithms. The algorithm is particularly suitable for biomedical applications involving the registration, such as medical imaging and image-guided surgery (IGS).

Dynamic Myocardial Ultrasound Localization Angiography.

Dynamic Myocardial Ultrasound Localization Angiography (MULA) is an ultrasound-based imaging modality destined to enhance the diagnosis and treatment monitoring of coronary pathologies. Current diagnosis methods of coronary artery disease focus on the observation of vessel narrowing in the coronary vasculature to assess the organ's condition. However, we would strongly benefit from mapping and measuring flow from intramyocardial arterioles and capillaries as they are the direct vehicle of the myocardium blood income. With the advent of ultrafast ultrasound scanners, imaging modalities based on the localization and tracking of injected microbubbles allow for the subwavelength resolution imaging of an organ's vasculature. Yet, the application of these vascular imaging modalities relies on an accumulation of cine loops of a region of interest undergoing no or minimal tissue motion. This work introduces the MULA framework that combines 1) the mapping of the dynamics of the microvascular flow using an ultrasound sequence triggered by the electrocardiogram with a 2) novel Lagrangian beamformer based on non-rigid motion registration algorithm to form images directly in the myocardium's material coordinates and thus correcting for the large myocardial motion and deformation. Specifically, we show that this framework enables the non-invasive imaging of the angioarchitecture and dynamics of intramyocardial flow in vessels as small as a few tens of microns in the rat's beating heart in vivo.