Traditional Registration Methods Research Articles

Spatial-aware contrastive learning for cross-domain medical image registration.

With the rapid advancement of medical imaging technologies, precise image analysis and diagnosis play a crucial role in enhancing treatment outcomes and patient care. Computed tomography (CT) and magnetic resonance imaging (MRI), as pivotal technologies in medical imaging, exhibit unique advantages in bone imaging and soft tissue contrast, respectively. However, cross-domain medical image registration confronts significant challenges due to the substantial differences in contrast, texture, and noise levels between different imaging modalities. The purpose of this study is to address the major challenges encountered in the field of cross-domain medical image registration by proposing a spatial-aware contrastive learning approach that effectively integrates shared information from CT and MRI images. Our objective is to optimize the feature space representation by employing advanced reconstruction and contrastive loss functions, overcoming the limitations of traditional registration methods when dealing with different imaging modalities. Through this approach, we aim to enhance the model's ability to learn structural similarities across domain images, improve registration accuracy, and provide more precise imaging analysis tools for clinical diagnosis and treatmentplanning. With prior knowledge that different domains of images (CT and MRI) share same content-style information, we extract equivalent feature spaces from both images, enabling accurate cross-domain point matching. We employ a structure resembling that of an autoencoder, augmented with designed reconstruction and contrastive losses to fulfill our objectives. We also propose region mask to solve the conflict between spatial correlation and distinctiveness, to obtain a better representation space. Our research results demonstrate the significant superiority of the proposed spatial-aware contrastive learning approach in the domain of cross-domain medical image registration. Quantitatively, our method achieved an average Dice similarity coefficient (DSC) of 85.68%, target registration error (TRE) of 1.92 mm, and mean Hausdorff distance (MHD) of 1.26 mm, surpassing current state-of-the-art methods. Additionally, the registration processing time was significantly reduced to 2.67 s on a GPU, highlighting the efficiency of our approach. The experimental outcomes not only validate the effectiveness of our method in improving the accuracy of cross-domain image registration but also prove its adaptability across different medical image analysis scenarios, offering robust support for enhancing diagnostic precision and patient treatmentoutcomes. The spatial-aware contrastive learning approach proposed in this paper introduces a new perspective and solution to the domain of cross-domain medical image registration. By effectively optimizing the feature space representation through carefully designed reconstruction and contrastive loss functions, our method significantly improves the accuracy and stability of registration between CT and MRI images. The experimental results demonstrate the clear advantages of our approach in enhancing the accuracy of cross-domain image registration, offering significant application value in promoting precise diagnosis and personalized treatment planning. In the future, we look forward to further exploring the application of this method in a broader range of medical imaging datasets and its potential integration with other advanced technologies, contributing more innovations to the field of medical image analysis andprocessing.

SGRTmreg: A Learning-Based Optimization Framework for Multiple Pairwise Registrations.

Point cloud registration is a fundamental task in computer vision and graphics, which is widely used in 3D reconstruction, object tracking, and atlas reconstruction. Learning-based optimization and deep learning methods have been widely developed in pairwise registration due to their own distinctive advantages. Deep learning methods offer greater flexibility and enable registering unseen point clouds that are not trained. Learning-based optimization methods exhibit enhanced robustness and stability when handling registration under various perturbations, such as noise, outliers, and occlusions. To leverage the strengths of both approaches to achieve a less time-consuming, robust, and stable registration for multiple instances, we propose a novel computational framework called SGRTmreg for multiple pairwise registrations in this paper. The SGRTmreg framework utilizes three components-a Searching scheme, a learning-based optimization method called Graph-based Reweighted discriminative optimization (GRDO), and a Transfer module to achieve multi-instance point cloud registration.Given a collection of instances to be matched, a template as a target point cloud, and an instance as a source point cloud, the searching scheme selects one point cloud from the collection that closely resembles the source. GRDO then learns a sequence of regressors by aligning the source to the target, while the transfer module stores and applies the learned regressors to align the selected point cloud to the target and estimate the transformation of the selected point cloud. In short, SGRTmreg harnesses a shared sequence of regressors to register multiple point clouds to a target point cloud. We conduct extensive registration experiments on various datasets to evaluate the proposed framework. The experimental results demonstrate that SGRTmreg achieves multiple pairwise registrations with higher accuracy, robustness, and stability than the state-of-the-art deep learning and traditional registration methods.

Event Security Using Face Identification

Abstract: One of the latest trends in event planning is incorporating facial recognition technology into the registration experience. Event planners and hosts have realized how convenient and efficient this method compares to traditional registration methods. The purpose of this system is to automate the existing manual system with the help of computerized equipment’s fulfilling their requirements, so that their valuable data/information can be stored for a longer period with easy accessing and manipulation of the same. The required software and hardware are easily available and easy to work with. Event management system, can lead to secure and fast management system. It can assist the user to concentrate on their other activities rather to concentrate on the record keeping. The organization can maintain computerized records without redundant entries.

Points of interest linear attention network for real-time non-rigid liver volume to surface registration.

In laparoscopic liver surgery, accurately predicting the displacement of key intrahepatic anatomical structures is crucial for informing the doctor's intraoperative decision-making. However, due to the constrained surgical perspective, only a partial surface of the liver is typically visible. Consequently, the utilization of non-rigid volume to surface registration methods becomes essential. But traditional registration methods lack the necessary accuracy and cannot meet real-timerequirements. To achieve high-precision liver registration with only partial surface information and estimate the displacement of internal liver tissues inreal-time. We propose a novel neural network architecture tailored for real-time non-rigid liver volume to surface registration. The network utilizes a voxel-based method, integrating sparse convolution with the newly proposed points of interest (POI) linear attention module. POI linear attention module specifically calculates attention on the previously extracted POI. Additionally, we identified the most suitable normalization methodRMSINorm. We evaluated our proposed network and other networks on a dataset generated from real liver models and two real datasets. Our method achieves an average error of 4.23mm and a mean frame rate of 65.4fps in the generation dataset. It also achieves an average error of 8.29mm in the human breathing motiondataset. Our network outperforms CNN-based networks and other attention networks in terms of accuracy and inferencespeed.

Test-Time Adaptation via Style and Structure Guidance for Histological Image Registration

Image registration plays a crucial role in histological image analysis, encompassing tasks like multi-modality fusion and disease grading. Traditional registration methods optimize objective functions for each image pair, yielding reliable accuracy but demanding heavy inference burdens. Recently, learning-based registration methods utilize networks to learn the optimization process during training and apply a one-step forward process during testing. While these methods offer promising registration performance with reduced inference time, they remain sensitive to appearance variances and local structure changes commonly encountered in histological image registration scenarios. In this paper, for the first time, we propose a novel test-time adaptation method for histological image registration, aiming to improve the generalization ability of learning-based methods. Specifically, we design two operations, style guidance and shape guidance, for the test-time adaptation process. The former leverages style representations encoded by feature statistics to address the issue of appearance variances, while the latter incorporates shape representations encoded by HOG features to improve registration accuracy in regions with structural changes. Furthermore, we consider the continuity of the model during the test-time adaptation process. Different from the previous methods initialized by a given trained model, we introduce a smoothing strategy to leverage historical models for better generalization. We conduct experiments with several representative learning-based backbones on the public histological dataset, demonstrating the superior registration performance of our test-time adaptation method.

Point Cloud Registration Method Based on Geometric Constraint and Transformation Evaluation.

Existing point-to-point registration methods often suffer from inaccuracies caused by erroneous matches and noisy correspondences, leading to significant decreases in registration accuracy and efficiency. To address these challenges, this paper presents a new coarse registration method based on a geometric constraint and a matrix evaluation. Compared to traditional registration methods that require a minimum of three correspondences to complete the registration, the proposed method only requires two correspondences to generate a transformation matrix. Additionally, by using geometric constraints to select out high-quality correspondences and evaluating the matrix, we greatly increase the likelihood of finding the optimal result. In the proposed method, we first employ a combination of descriptors and keypoint detection techniques to generate initial correspondences. Next, we utilize the nearest neighbor similarity ratio (NNSR) to select high-quality correspondences. Subsequently, we evaluate the quality of these correspondences using rigidity constraints and salient points' distance constraints, favoring higher-scoring correspondences. For each selected correspondence pair, we compute the rotation and translation matrix based on their centroids and local reference frames. With the transformation matrices of the source and target point clouds known, we deduce the transformation matrix of the source point cloud in reverse. To identify the best-transformed point cloud, we propose an evaluation method based on the overlap ratio and inliers points. Through parameter experiments, we investigate the performance of the proposed method under various parameter settings. By conducting comparative experiments, we verified that the proposed method's geometric constraints, evaluation methods, and transformation matrix computation consistently outperformed other methods in terms of root mean square error (RMSE) values. Additionally, we validated that our chosen combination for generating initial correspondences outperforms other descriptor and keypoint detection combinations in terms of the registration result accuracy. Furthermore, we compared our method with several feature-matching registration methods, and the results demonstrate the superior accuracy of our approach. Ultimately, by testing the proposed method on various types of point cloud datasets, we convincingly established its effectiveness. Based on the evaluation and selection of correspondences and the registration result's quality, our proposed method offers a solution with fewer iterations and higher accuracy.

A Novel Multimodal Fusion Framework Based on Point Cloud Registration for Near-Field 3D SAR Perception

This study introduces a pioneering multimodal fusion framework to enhance near-field 3D Synthetic Aperture Radar (SAR) imaging, crucial for applications like radar cross-section measurement and concealed object detection. Traditional near-field 3D SAR imaging struggles with issues like target–background confusion due to clutter and multipath interference, shape distortion from high sidelobes, and lack of color and texture information, all of which impede effective target recognition and scattering diagnosis. The proposed approach presents the first known application of multimodal fusion in near-field 3D SAR imaging, integrating LiDAR and optical camera data to overcome its inherent limitations. The framework comprises data preprocessing, point cloud registration, and data fusion, where registration between multi-sensor data is the core of effective integration. Recognizing the inadequacy of traditional registration methods in handling varying data formats, noise, and resolution differences, particularly between near-field 3D SAR and other sensors, this work introduces a novel three-stage registration process to effectively address these challenges. First, the approach designs a structure–intensity-constrained centroid distance detector, enabling key point extraction that reduces heterogeneity and accelerates the process. Second, a sample consensus initial alignment algorithm with SHOT features and geometric relationship constraints is proposed for enhanced coarse registration. Finally, the fine registration phase employs adaptive thresholding in the iterative closest point algorithm for precise and efficient data alignment. Both visual and quantitative analyses of measured data demonstrate the effectiveness of our method. The experimental results show significant improvements in registration accuracy and efficiency, laying the groundwork for future multimodal fusion advancements in near-field 3D SAR imaging.

Local Contractive Registration with Biomechanical Model: Assessing Microwave Ablation after Compensation for Tissue Shrinkage.

Microwave ablation (MWA) is a minimally invasive procedure for the treatment of liver tumor. Accumulating clinical evidence has considered the minimal ablative margin (MAM) as a significant predictor of local tumor progression (LTP). In clinical practice, MAM assessment is typically carried out through image registration of pre- and post-MWA images. However, this process faces two main challenges: non-homologous match between tumor and coagulation with inconsistent image appearance, and tissue shrinkage caused by thermal dehydration. These challenges result in low precision when using traditional registration methods for MAM assessment. In this paper, we present a local contractive nonrigid registration method using a biomechanical model (LC-BM) to address these challenges and precisely assess the MAM. The LC-BM contains two consecutive parts: (1) local contractive decomposition (LC-part), which reduces the incorrect match between the tumor and coagulation and quantifies the shrinkage in the external coagulation region, and (2) biomechanical model constraint (BM-part), which compensates for the shrinkage in the internal coagulation region. After quantifying and compensating for tissue shrinkage, the warped tumor is overlaid on the coagulation, and then the MAM is assessed. We evaluated the method using prospectively collected data from 36 patients with 47 liver tumors, comparing LC-BM with 11 state-of-the-art methods. LTP was diagnosed through contrast-enhanced MR follow-up images, serving as the ground truth for tumor recurrence. LC-BM achieved the highest accuracy (97.9%) in predicting LTP, outperforming other methods. Therefore, our proposed method holds significant potential to improve MAM assessment in MWA surgeries.

NIMG-84. COMPARISON OF TRADITIONAL AND ADVANCED DEEP-LEARNING PATIENT SPECIFIC REGISTRATION METHODS IN PRESERVING TUMOR HABITAT IN GLIOBLASTOMA TUMORS: A PRELIMINARY STUDY

Abstract PURPOSE Registration of patient specific pre and post operative MRI scans in Glioblastoma patients is crucial to assess tumor progression, effect of radiation, and chemotherapy and survival prediction. However, in brain tumors longitudinal registration is challenging as tumor resection and mass effects causes tissue deformations and missing intensity correspondence across longitudinal scans. In this study, we compared effectiveness of traditional registration algorithms with state-of-the-art computationally extensive deep learning (DL) models in preserving intensity of tumor regions. METHODS Brain MRI scans (T1w,T1c,T2w,FLAIR) of 14 glioblastoma patients with first (tp1), intermediate (tp2) and follow up post operative scans (tp3) were obtained from two publicly available dataset (LUMIERE, XCURE). Preprocessing involved co-registration of all modalities within a timepoint, skull stripping, and uniformity in image dimensions, orientation, and origin across all timepoints. Tumor scans at tp2 and tp3 were registered to tp1 for all modalities independently, using different registration algorithms such as the traditional affine and symmetric diffeomorphic (SyN) registration with and without tumor masks using the ANTs software, along with the MICCAI-registration challenge winner DL algorithm (DIRAC). Intensity distribution of registered and unregistered scans for tp2 and tp3 were compared using the earth mover’s distance (EMD). RESULTS Traditional affine (tp2-0.074±0.06, tp3:0.058±0.051) and SyN (tp2-0.048±0.031 ,tp3:0.075±0.074) registration algorithm using T2w modality provided lower average EMD than DL method (tp2-0.624±0.69,tp3-0.595±0.67). Intensity distribution was better preserved for scans from intermediate timepoints compared to the post operative scans. CONCLUSION Our preliminary assessment suggests that traditional registration methods when implemented on pre-aligned MRI scans may better preserve image intensities in brain tumors scans compared to advanced and computationally expensive DL methods.

A transformer-based hierarchical registration framework for multimodality deformable image registration.

Deformable image registration (DIR) between daily and reference images is fundamentally important for adaptive radiotherapy. In the last decade, deep learning-based image registration methods have been developed with faster computation time and improved robustness compared to traditional methods. However, the registration performance is often degraded in extra-cranial sites with large volume containing multiple anatomic regions, such as Computed Tomography (CT)/Magnetic Resonance (MR) images used in head and neck (HN) radiotherapy. In this study, we developed a hierarchical deformable image registration (DIR) framework, Patch-based Registration Network (Patch-RegNet), to improve the accuracy and speed of CT-MR and MR-MR registration for head-and-neck MR-Linac treatments. Patch-RegNet includes three steps: a whole volume global registration, a patch-based local registration, and a patch-based deformable registration. Following a whole-volume rigid registration, the input images were divided into overlapping patches. Then a patch-based rigid registration was applied to achieve accurate local alignment for subsequent DIR. We developed a ViT-Morph model, a combination of a convolutional neural network (CNN) and the Vision Transformer (ViT), for the patch-based DIR. A modality independent neighborhood descriptor was adopted in our model as the similarity metric to account for both inter-modality and intra-modality registration. The CT-MR and MR-MR DIR models were trained with 242 CT-MR and 213 MR-MR image pairs from 36 patients, respectively, and both tested with 24 image pairs (CT-MR and MR-MR) from 6 other patients. The registration performance was evaluated with 7 manually contoured organs (brainstem, spinal cord, mandible, left/right parotids, left/right submandibular glands) by comparing with the traditional registration methods in Monaco treatment planning system and the popular deep learning-based DIR framework, Voxelmorph. Evaluation results show that our method outperformed VoxelMorph by 6 % for CT-MR registration, and 4 % for MR-MR registration based on DSC measurements. Our hierarchical registration framework has been demonstrated achieving significantly improved DIR accuracy of both CT-MR and MR-MR registration for head-and-neck MR-guided adaptive radiotherapy.

Boundary-aware registration network for 4D-CT lung image with sliding motion

Background and ObjectiveFast and accurate registration of 4D-CT lung images is significant for respiratory motion modeling and radiotherapy planning. However, since the complex respiratory motion involves sliding motion at lung boundary, the traditional registration methods and regularization terms perform poorly in recovering both sliding and smooth deformation in 4D-CT lung image registration. MethodsIn order to overcome these limitations of the traditional registration methods and regularization terms, we propose a boundary-aware registration model with a spatially adaptive regularization term. We incorporate a lung segmentation network into the registration model. With the lung boundary-aware information from the segmentation network, we construct a spatially adaptive regularization term, which integrates the smooth regularization and the non-smooth regularization, to accommodate both smooth and sliding motion. ResultsWe evaluate the proposed registration model on clinical 4D-CT data and the public DIR-Lab dataset. Our model provides a minimum Target Registration Error (1.59 ± 0.57 mm) of landmarks compared with the other lung registration methods. The ablation studies show that the proposed spatially adaptive regularization term provides superior performance in HD (13.75 ± 3.36 mm) and MHD (1.63 ± 0.32 mm) to the smooth regularization term and non-smooth regularization term. ConclusionsThe proposed boundary-aware registration model enables adaptive regularization term, which can flexibly regulate both the sliding motion at the lung boundary and the smooth motion inside the lung simultaneously. Therefore, our model can perform fast and accurate registration for 4D-CT lung images with sliding motion, which is beneficial to respiratory motion modeling and lung cancer radiotherapy.

A novel method for spine ultrasound and X-ray radiograph registration

Ultrasound is a promising imaging method for scoliosis evaluation because it is radiation free and provide real-time images. However, it cannot provide bony details because ultrasound cannot penetrate the bony structure. Therefore, registration of real-time ultrasound images with the previous X-ray radiograph can help physicians understand the spinal deformity of patients. In this study, an improved free-from deformation registration method based on mutual registration and hierarchical adaptive grid (MRHA-FFD) was developed. The method first performed registration grid preprocessing and then optimized control points and conducted mutual registration. Finally, a Blur-aware Attention Network was adopted for image deblurring. The performance of each step was verified by ablation experiments. Comparison experiment between the proposed method and traditional registration methods was also conducted. The qualitative and quantitative results suggested that MRHA-FFD is a promising approach for registering spine ultrasound image and X-ray radiograph.

A hybrid deformable registration method to generate motion-compensated 3D virtual MRI for fusion with interventional real-time 3D ultrasound.

Ultrasound is often the preferred modality for image-guided therapy or treatment in organs such as liver due to real-time imaging capabilities. However, the reduced conspicuity of tumors in ultrasound images adversely impacts the precision and accuracy of treatment delivery. This problem is compounded by deformable motion due to breathing and other physiological activity. This creates the need for a fusion method to align interventional US with pre-interventional modalities that provide superior soft-tissue contrast (e.g., MRI) to accurately target a structure-of-interest and compensate for liver motion. In this work, we developed a hybrid deformable fusion method to align 3D pre-interventional MRI and 3D interventional US volumes to target the structures-of-interest in liver accurately in real-time. The deformable multimodal fusion method involved an offline alignment of a pre-intervention MRI with a pre-intervention US volume using a traditional registration method, followed by real-time prediction of deformation using a trained deep-learning model between interventional US volumes across different respiratory states. This framework enables motion-compensated MRI-US image fusion in real-time for image-guided treatment. The proposed hybrid deformable registration method was evaluated on three healthy volunteers across the pre-intervention MRI and 20 US volume pairs in the free-breathing respiratory cycle. The mean Euclidean landmark distance of three homologous targets in all three volunteers was less than 3 mm for percutaneous liver procedures. Preliminary results show that clinically acceptable registration accuracies for near real-time, deformable MRI-US fusion can be achieved by our proposed hybrid approach. The proposed combination of traditional and deep-learning deformable registration techniques is thus a promising approach for motion-compensated MRI-US fusion to improve targeting in image-guided liver interventions.

Anisotropic Generalized Bayesian Coherent Point Drift for Point Set Registration

Registration is highly demanded in many real-world scenarios such as robotics and automation. Registration is challenging partly due to the fact that the acquired data is usually noisy and has many outliers. In addition, in many practical applications, one point set (PS) usually only covers a partial region of the other PS. Thus, most existing registration algorithms cannot guarantee theoretical convergence. This article presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets (PSs) are aligned and the anisotropic positional noise is considered. The high-dimensional point sets typically consist of the positional vectors and normal vectors. On one hand, with the normal vectors, the proposed method is more robust to noise and outliers, and the point correspondences can be found more accurately. On the other hand, incorporating the registration into the BCPD framework will guarantee the algorithm’s theoretical convergence. Our contributions in this article are three folds. First, the problem of rigidly aligning two general PSs with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm’s iterations are given in closed-form or with iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD. Note to Practitioners—This paper was motivated by the problem of 3D rigid PSR for computer-assisted surgery (CAS), especially in orthopedic applications. The proposed algorithm is also suitable for other scenarios where the initial coarse registration is conducted. The traditional registration methods are susceptible to noise (especially anisotropic noise), outliers, and incomplete partial data. This paper generalizes the recently proposed BCPD method to the six-dimensional scenario where anisotropic positional noise is considered and normal vectors are incorporated. The proposed noise model is decomposed into three parts to be solved alternately: the membership probability of mixture distributions, the soft correspondence estimation, and the model parameters (i.e., the rotation matrix, translation vector, the covariance matrix with the anisotropic positional error, and the concentration parameter with the estimation of the normal vectors). Especially, the convergence is guaranteed at the theoretical level using the variational inference theory. The experimental results demonstrate the superiority of our algorithm on registration accuracy, convergence speed, and robustness to noise, outliers, and partial data.

STORM: Structure-Based Overlap Matching for Partial Point Cloud Registration.

Partial point cloud registration aims to transform partial scans into a common coordinate system. It is an important preprocessing step to generate complete 3D shapes. Although previous registration methods have made great progress in recent decades, traditional registration methods, such as Iterative Closest Point (ICP) and its variants, all these methods highly depend on the sufficient overlaps between two point clouds, because they cannot distinguish outlier correspondences. Note that the overlap between point clouds could always be small, which limits the application of these methods. To tackle this problem, we present a StrucTure-based OveRlap Matching (STORM) method for partial point cloud registration. In our method, an overlap prediction module with differentiable sampling is designed to detect points in overlap utilizing structure information, and facilitates exact partial correspondence generation, which is based on discriminative pointwise feature similarity. The pointwise features which contain effective structural information are extracted by graph-based methods. Experimental results and comparison with state-of-the-art methods demonstrate that STORM can achieve better performance. Moreover, most registration methods perform worse when the overlap ratio decreases, while STORM can still achieve satisfactory performance when the overlap ratio is small.

Multi‐scale homography estimation based on dual feature aggregation transformer

AbstractThe accuracy of registration in image stitching task directly affects the performance of subsequent stages. Traditional registration methods rely heavily on the quality of the features when calculating the homography matrix, resulting in alignment failures in low‐texture or low‐overlap scenes due to extracting insufficient features. On the other hand, existing DNN‐based methods for homography estimation are more robust in multiple scenes but previous work usually employs an overly simple convolutional network structure to directly regress the homography, ignoring the redundant information contained in the feature maps so that their prediction accuracy is inferior to the traditional methods in simple scenarios. To overcome the disadvantages of the two methods, a Multi‐scale structure is proposed to extract feature maps at three scales and design two modules to handle the matrix prediction respectively. The DFA‐T module analyzes semantic information on the high‐level features to accomplish coarse‐grained alignment while the Contextual Correlation module on the bottom level to accomplish more accurate alignment. Experiments demonstrate that this method provides more accurate alignment results than the existing state‐of‐the‐art DNN‐based methods and outperforms traditional algorithms with more stable results in some extreme scenarios.

Novel optical-markers-assisted point clouds registration for panoramic 3D shape measurement

Panoramic 3D shape measurement for larger-scale and complex objects inevitably requires multi-view point clouds registration. However, traditional registration methods often use expensive devices or inflexible physical markers. In this paper, a novel optical-markers-assisted registration (OMAR) method is proposed, which substitutes traditional physical markers with optical markers for the registration. Specifically, a markers design strategy is first proposed to generate the desired optical markers according to different requirements. The designed markers can be projected by using a low-cost projector or a laser with grating. The camera captures these projected markers from the object surface, and the 3D sensor collaboratively acquires the corresponding 3D point clouds. Finally, a markers-guided algorithm is specially designed to suit optical markers, which can make full use of the optical markers and recover the 360-degree 3D shape. The provided experimental results demonstrate both the accuracy and efficiency of the proposed OMAR in measuring large-scale, complex and texture-less objects. The proposed OMAR also enables flexible and non-destructive registration without losing accuracy compared with traditional registration methods.

Stochastic Planner-Actor-Critic for Unsupervised Deformable Image Registration

Large deformations of organs, caused by diverse shapes and nonlinear shape changes, pose a significant challenge for medical image registration. Traditional registration methods need to iteratively optimize an objective function via a specific deformation model along with meticulous parameter tuning, but which have limited capabilities in registering images with large deformations. While deep learning-based methods can learn the complex mapping from input images to their respective deformation field, it is regression-based and is prone to be stuck at local minima, particularly when large deformations are involved. To this end, we present Stochastic Planner-Actor-Critic (spac), a novel reinforcement learning-based framework that performs step-wise registration. The key notion is warping a moving image successively by each time step to finally align to a fixed image. Considering that it is challenging to handle high dimensional continuous action and state spaces in the conventional reinforcement learning (RL) framework, we introduce a new concept `Plan' to the standard Actor-Critic model, which is of low dimension and can facilitate the actor to generate a tractable high dimensional action. The entire framework is based on unsupervised training and operates in an end-to-end manner. We evaluate our method on several 2D and 3D medical image datasets, some of which contain large deformations. Our empirical results highlight that our work achieves consistent, significant gains and outperforms state-of-the-art methods.

Luojia-1 Nightlight Image Registration Based on Sparse Lights

When mosaicking adjacent nightlight images of a large area that lacks human activities, traditional registration methods have difficulty realizing the tie point registrations due to the lack of structural information. In order to address this issue, this study devises an easy-to-implement engineering solution that allows for the registration of sparse light areas with high efficiency while guaranteeing accuracy in non-sparse light areas. The proposed method first extracts the sparsely distributed light point positions through use of roundness detection and the centroid method. Then, geometric positioning forward and backward algorithms and the random consistency sampling detection algorithm (RANSAC) are used in order to achieve a rough registration of the nightlight images and the remaining tie points are expanded through the affine model. Through experimentation it was found that, compared with traditional registration methods, the proposed method is more reliable and has a wider distribution in sparse light areas. Finally, through the registration test of 275 scenes of nightlight images of China from Luojia-1, the coverage ratio of the tie points was increased from 59.3% from the traditional method to 95.3% in this study and the accuracy of the block adjustment was 0.63 pixels, which verifies the effectiveness of the method. The proposed method provides a basis for the registration, block adjustment, and mosaicking of nightlight images.

A particle filter approach to dynamic kidney pose estimation in robotic surgical exposure.

Traditional soft tissue registration methods require direct intraoperative visualization of a significant portion of the target anatomy in order to produce acceptable surface alignment. Image guidance is therefore generally not available during the robotic exposure of structures like the kidneys which are not immediately visualized upon entry into the abdomen. This paper proposes guiding surgical exposure with an iterative state estimator that assimilates small visual cues into an a priori anatomical model as exposure progresses, thereby evolving pose estimates for the occluded structures of interest. Intraoperative surface observations of a right kidney are simulated using endoscope tracking and preoperative tomography from a representative robotic partial nephrectomy case. Clinically relevant random perturbations of the true kidney pose are corrected using this sequence of observations in a particle filter framework to estimate an optimal similarity transform for fitting a patient-specific kidney model at each step. The temporal response of registration error is compared against that of serial rigid coherent point drift (CPD) in both static and simulated dynamic surgical fields, and for varying levels of observation persistence. In the static case, both particle filtering and persistent CPD achieved sub-5mm accuracy, with CPD processing observations 75% faster. Particle filtering outperformed CPD in the dynamic case under equivalent computation times due to the former requiring only minimal persistence. This proof-of-concept simulation study suggests that Bayesian state estimation may provide a viable pathway to image guidance for surgical exposure in the abdomen, especially in the presence of dynamic intraoperative tissue displacement and deformation.