Image Registration Research Articles

This study aimed to employ RadioTherapy extension of the Particle and Heavy Ion Transport code System (RT-PHITS) Monte Carlo (MC) simulation for estimating absorbed doses in target organs and tumors in patients administered with 177Lu-DOTATATE, using single-photon emission computed tomography/computed tomography (SPECT/CT) imaging. Quantitative SPECT/CT images were obtained from 17 patients across the abdominal region at four time points: approximately 4, 24, 72, and 120h following the administration of 177Lu-DOTATATE. The liver, spleen, left and right kidneys, and total kidneys were automatically segmented on the CT images using the TotalSegmentator tool. Tumors were manually delineated based on SPECT/CT images. Image registration was performed using an Elastix-based method, with the first SPECT/CT time point serving as the reference. Voxel-level time-integrated activity (TIA) maps were created by fitting mono-exponential functions. These TIA maps, together with the reference CT images, were input into RT-PHITS to calculate dose distributions. The absorbed doses calculated by RT-PHITS were compared with those from IDAC-Dose 2.1 through two approaches: first, by using independently derived time-integrated activity coefficients (TIACs) from each method to assess the combined effects of kinetic modeling and dose calculation technique; second, by applying the same TIACs-obtained from time-integrated activity data-to both methods to isolate the influence of the dose calculation approach. RT-PHITS yielded higher mean absorbed doses per unit of administered activity compared to IDAC-Dose. The relative differences ranged between 0.63% and 15.35%, with the right kidney showing the largest discrepancy. When the same time-integrated data were used for both RT-PHITS and IDAC-Dose, relative differences remained below 10.40%. RT-PHITS is a capable tool for calculating absorbed doses in 177Lu-DOTATATE therapy. It consistently produced higher dose estimates than the organ-based method, emphasizing the benefits of patient-specific dosimetry, especially in organs that contain or are near tumors.

Deformable image registration estimates voxel-wise correspondences between images through spatial transformations, and plays a key role in medical imaging. While deep learning methods have significantly reduced runtime, efficiently handling large deformations remains a challenging task. Convolutional networks aggregate local features but lack direct modeling of voxel correspondences, promoting recent works to explore explicit feature matching. Among them, voxel-to-region matching is more efficient for direct correspondence modeling by computing local correlation features within neighbourhoods, while region-to-region matching incurs higher redundancy due to excessive correlation pairs across large regions. However, the inherent locality of voxel-to-region matching hinders the capture of long-range correspondences required for large deformations. To address this, we propose a Recurrent Correlation-based framework that dynamically relocates the matching region toward more promising positions. At each step, local matching is performed with low cost, and the estimated offset guides the next search region, supporting efficient convergence toward large deformations. In addition, we uses a lightweight recurrent update module with memory capacity and decouples motion-related and texture features to suppress semantic redundancy. We conduct extensive experiments on brain MRI and abdominal CT datasets under two settings: with and without affine pre-registration. Results show our method exhibits a strong accuracy-computation trade-off, surpassing or matching the state-of-the-art performance. For example, it achieves comparable performance on the non-affine OASIS dataset, while using only 9.5% of the FLOPs and running 96% faster than RDP, a representative high-performing method.

Objective.To correlate numerical simulations of the electric dose distribution with early post-operative MRI following irreversible electroporation (IRE) treatment of hepatocellular carcinoma (HCC).Approach.A standard linear electrostatic model was employed to simulate the three-dimensional electric field (EF) distribution using real electric pulses and geometrical characteristics and intraoperative cone-beam CT (CBCT) data. Spatial registration between intraoperative CBCT and post-operative MRI was performed using both rigid and deformable methods, ranging from simple global translations based on anatomical landmarks to advanced deformable image registration (DIR) techniques accounting for elastic tissue deformations.Main results.The proposed approach was retrospectively evaluated using data from 22 patients who underwent IRE liver ablation (one patient underwent two distinct IRE procedures), resulting in a total of 23 procedures. The most accurate correspondence between predicted and observed ablation zones was achieved using a dose threshold of approximately 350 V cm-1, yielding a median dice similarity coefficient around 0.74, indicative of substantial spatial overlap. Although elastic DIR approaches applied to segmented liver regions provided the highest registration accuracy, the simpler translational registration based on manually selected landmarks demonstrated surprisingly robust performance in localizing the simulated EF within the actual ablation zone.Significance.These findings contribute to the standardization of IRE efficacy assessment on MRI and highlight the significant potential EF simulations to predict the extent of tissue ablation in IRE procedures for HCC. This approach may offer a valuable tool for improving intraoperative decision-making and post-operative assessment.

Automatic digitization of applicator and catheters for MRI-guided cervical cancer brachytherapy.

Prostate cancer is a prevalent malignant tumor in men, and accurate diagnosis and personalized treatment rely on multimodal imaging, such as MRI and TRUS. However, differences in imaging mechanisms and prostate deformation due to ultrasound probe compression pose significant challenges for high-quality registration between the two modalities. In this study, we propose a label-aware weakly supervised diffusion model for MRI-TRUS multimodal image registration. First, we align label centroid positions by maximizing the Dice coefficient to correct initial biases. Second, we combine label supervision with a diffusion model to generate high-quality deformation fields. Finally, we incorporate a feature-guided module to better preserve edge structures and improve registration smoothness. Experiments conducted on the µ-RegPro dataset demonstrate that our method outperforms current state-of-the-art (SOTA) approaches across multiple evaluation metrics. Specifically, it achieves a Dice coefficient of 0.880 and reduces the target registration error (TRE) to 0.940, significantly surpassing unsupervised methods such as VoxelMorph, FSDiffReg, and supervised methods like LocalNet and AutoFuse. The results show that preliminary label centroid alignment effectively enhances the performance of the diffusion-based deformation registration model, reducing the TRE from 3.084 to 0.940. The ablation study demonstrates that the feature-guided diffusion module effectively suppresses deformation field folding, while the label-aware module enhances label alignment. When combined, the proposed framework achieves a favorable balance, substantially improving registration accuracy (Dice = 0.880, TRE = 0.940) with reduced folding (|J|≤0 = 0.134). This method exhibits strong robustness and generalizability in handling large deformations in target regions while preserving details in nontarget regions. The proposed label-aware weakly supervised diffusion model enables accurate and efficient MRI-TRUS multimodal image registration, offering strong potential for clinical applications such as prostate cancer diagnosis, targeted biopsy, and image-guided navigation.

Background: Accurate tracking of the aortic valve in 4D transesophageal echocardiography (TEE) and subsequent leaflet strain measurement remain a challenge due to limited imaging temporal resolution. Conventional intensity-based image registration techniques often fail to capture the rapid nonlinear deformation of valve leaflets across cardiac phases. The objective of this work is to present an image analysis framework that temporally augments 4D TEE with finite element modeling (FEM) to reconstruct patient-specific valve motion and quantify leaflet strain in both trileaflet and bicuspid aortic valves. Methods: We propose a hybrid framework that integrates FEM with deformable image registration to achieve leaflet tracking in 4D TEE sequences (Fig 1). First, the aortic valve is manually segmented in a mid-systolic “reference” (open) frame. A shell representation of the segmented leaflets is created and integrated with FEM to simulate valve closure. A final mid-diastolic segmentation of the leaflets is obtained by applying the following transformations to the mid-systolic reference segmentation: (1) the FEM-recovered transformations of valve closure and (2) the registration-derived transformation between an FEM-derived synthetic mid-diastolic image and the real mid-diastolic image. The proposed method was tested on six patients with varying aortic valve abnormalities (Table 1), and leaflet strains were computed. Results: The proposed method significantly improved segmentation tracking accuracy compared to conventional registration. The mean distance between the tracked closed-state segmentation and manual ground truth for six patients was 1.67 ± 0.49 mm using our hybrid approach, versus 3.19 ± 1.17 mm with conventional registration (no FEM). Strain maps showed physiologically consistent patterns with elevated strain near coaptation lines. Notably, patient 4, with severe calcification, exhibited the lowest strain. Representative results are shown in Figure 2. Conclusions: These findings confirm that biomechanically generated intermediate frames enhance registration accuracy and patient-specific geometric fidelity of the aortic valve, enabling strain analysis. The approach has the potential to inform our understanding of diverse valve mechanics and facilitate translational application to the assessment of structural heart disease.

To develop a framework (ACROBATIC) for correcting motion in 3D radial fetal MRI. Data were simulated (N = 200) and acquired in utero (N = 11, gestational age: 32 ± 2 weeks). Motion due to maternal respiration was estimated by extracting a self-gating signal and applying focused navigation. Bulk motion was estimated by splitting the acquisition into sequential bins, reconstructing 3D volumes and applying rigid image registration. These combined motion estimates were used to correct k-space. Self-gating signals were compared to ground truth in simulations and an external sensor in utero. The cumulative position error (CPE) measured the accuracy of motion estimations relative to ground truth in simulations and relative sharpness measured the corresponding impact on image quality for both simulations and in utero data. An expert reviewer performed a blinded ranking of in utero images including uncorrected and corrected data. Self-gating signals correlated strongly with ground truth for simulations (R = 0.97 ± 0.01) and a external sensor for in utero data (R = 0.75 ± 0.23). CPE decreased significantly using ACROBATIC (uncorrected: 13.33[12.73-14.05], corrected: 2.20[1.80-2.61]). Relative image sharpness increased with ACROBATIC for both simulated (4.51[2.89-5.79]) and in utero data (1.12[0.77-1.44]) consistent with expert ranking where ACROBATIC images were given the best rank in the majority of cases. ACROBATIC enables motion correction in 3D radial fetal MRI. Correction of displacement due to maternal respiration and bulk motion results in improved image quality in simulations and in utero. Comparison to 2D frameworks are now warranted to establish the added diagnostic value of this approach.

Augmented reality in the operating room enhances surgical precision. Merging information from preoperative 3D imaging into the real-time video during surgery allows for better visualization of critical structures. Displaying an augmented real-time video on a screen requires a 2D/3D to 2D image registration as a preliminary step. An accurate error estimation is crucial for evaluating the system's performance. A reprojection error estimation has been commonly calculated based on a pinhole camera model, which does not accurately represent the optical properties of an endoscope or an operative microscope. In our work, a system for registering a CT-scan to a 2D-otoendoscopic video was evaluated. This system was designed for use in an augmented reality setup for ear surgery. Evaluation was performed using the target registration reprojection error measurement. A comparison between the pinhole camera model and the thin-lens model was conducted. Additionally, a quantification formula for the target registration reprojection error was established and the parameters affecting it were analyzed. Specifically, the target's distance from the camera lens and its angle of alignment with the optical axis were assessed. Five human ear resin models bearing spherical Markers with their corresponding CT-scans were used. The estimated error with the thin-lens model was 0.65 ± 0.52 mm versus 0.79 ± 0.60 mm with the pinhole model (18% of difference). Target registration reprojection error tended to be correlated to the distance from the camera lens. In conclusion, both the target-lens distance and the target alignment influence the target registration reprojection error measurement. The choice of the optical model affects significantly the error estimation, and the thin-lens model should be preferred in endoscopic applications. Based on the results, the tested augmented reality system is compatible with ear surgery.

Abstract. Feature matching is a necessary step for many computer vision and photogrammetry applications such as image registration, structure-from-motion, and visual localization. Classical handcrafted methods such as SIFT feature detection and description combined with nearest neighbour matching and RANSAC outlier removal have been state-of-the-art for mobile mapping cameras. With recent advances in deep learning, learnable methods have been introduced and proven to have better robustness and performance under complex conditions. Despite their growing adoption, a comprehensive comparison between classical and learnable feature matching methods for the specific task of semantic 3D building camera-to-model matching is still missing. This submission systematically evaluates the effectiveness of different feature-matching techniques in visual localization using textured CityGML LoD2 models. We use standard benchmark datasets (HPatches, MegaDepth-1500) and custom datasets consisting of facade textures and corresponding camera images (terrestrial and drone). For the latter, we evaluate the achievable accuracy of the absolute pose estimated using a Perspective-n-Point (PnP) algorithm, with geometric ground truth derived from geo-referenced trajectory data. The results indicate that the learnable feature matching methods vastly outperform traditional approaches regarding accuracy and robustness on our challenging custom datasets with zero to 12 RANSAC-inliers and zero to 0.16 area under the curve. We believe that this work will foster the development of model-based visual localization methods. Link to the code: https://github.com/simBauer/To_Glue_or_not_to_Glue

ABSTRACT Historical map georeferencing, especially when dealing with maps that exhibit high levels of local distortion, remains a time-consuming process. This paper introduces a pipeline that automates much of this process by transferring georeferencing information from georeferenced maps (Anchor) to maps lacking georeferencing (Target). At its core, the method employs deep-learning algorithms for image registration (SuperPoint and SuperGlue) alongside tailored modules to exclude outliers and enhance match density. Specifically, RANSAC is combined with a Delaunay-based procedure to discard erroneous matches and preserve consistent spatial relationships. To address the reduction in keypoints following outlier emoval, we incorporate a patch-based local image registration, enabling multiscale matching. After a final outlier-removal step, the resulting high-quality matches are used to assign real-world coordinates to the Target map. We evaluated the pipeline on 86 georeferenced historical maps of Jerusalem and obtained a root mean square error (RMSE) below 1% of the map diagonal for 71 of them. Moreover, the final georeferencing accuracy was closely tied to the number of matching keypoints, with a threshold of 100 serving as a strong indicator of reliable results. Extending the pipeline to an additional 113 non-georeferenced maps, we found that 86 were successfully georeferenced based on this keypoint threshold.

OXRAY, a state-of-the-art radiation therapy system commercialized by Hitachi High-Tech Ltd. in 2023, integrates unique beam delivery and image-guided radiation therapy (IGRT) technologies as the successor to Vero4DRT. This study evaluated the performance of this second-generation O-ring-shaped linear accelerator. The percentage depth dose (PDD) and off-center ratio (OCR) were calculated using the RayStation 2023B treatment planning system with multileaf collimator-shaped square fields. PDDs were evaluated up to a depth of 250mm and OCRs at depths of 15, 100, and 200mm, compared with measurements. Patient-specific quality assurance (PSQA) was conducted for 28 volumetric-modulated arc therapy plans and evaluated using gamma pass rates (GPRs) based on a 3%/2mm criterion. The biaxial rotational dynamic radiation therapy (BROAD-RT) performance was validated with 25 trajectories. A tracking experiment under rotational irradiation was performed to assess the tracking accuracy. Additionally, image-guidance systems (kV X-ray and kV cone-beam computed tomography) were evaluated using anthropomorphic phantoms. The localization accuracy (LA) was determined by comparing the known offsets with the noted differences between the initial and corrected positions. Differences between the calculated and measured data were within the tolerance limits defined in European Society for Radiotherapy and Oncology Booklet 7 and American Association of Physicists in Medicine (AAPM) Medical Physics Practice Guideline 5.b. The median PSQA GPRs exceeded 95%, satisfying AAPM Task Group-218 criteria. BROAD-RT demonstrated submillimeter accuracy (within 0.4mm), even for complex trajectories. The tracking accuracy remained within 1mm even during rotational delivery. LA was within 0.5mm for translational shifts and 0.5° for rotational adjustments. OXRAY demonstrated clinically acceptable beam quality and high-precision dose delivery outcomes. The tracking accuracy was maintained under rotational irradiation. Automatic image registration enabled accurate, reproducible patient positioning, supporting reliable IGRT implementation. These findings offer practical guidance and technical benchmarks for institutions adopting OXRAY.

Multi-scale dual-attention frequency fusion for joint segmentation and deformable medical image registration

From Model Based to Learned Regularization in Medical Image Registration: A Comprehensive Review

DHR-Net: Dynamic Harmonized registration network for multimodal medical images.

TSMR-Net: a two-stage multimodal medical image registration method via pseudo-image generation and deformable registration

Deformable image registration (DIR) is a critical component for planning and quality assurance in CBCT-guided adaptive radiotherapy treatment. Conventional DIR methods apply a uniform regularization over the image domain, failing to account for the local biomechanical properties of different tissues. Recent advancements emphasize the need for spatially varying models to improve anatomicalplausibility. This study aims to introduce a novel, lightweight framework that incorporates local rigidity constraints in DIR to improve anatomical consistency and motion estimation accuracy. The approach is designed to be general, computationally efficient, and compatible with existing DIRpipelines. The proposed framework jointly optimizes the rigid-body transformation and the surrounding tissue deformations. The proposed framework was evaluated on pelvic CT-CBCT image pairs with annotated landmarks. Local rigidity constraints were imposed on automatically segmented sacrum, hip, and femur bones. Four registration algorithms were tested, combining different regularization types, with and without rigidity constraints. Accuracy was measured using the target registration error (TRE) and biomechanical plausibility was assessed via the Jacobian of the estimated motion. Secondary verification was performed on a thoracic CT database. With automatic segmentation and rigidity constraints placed upon the ribs and thoracic vertebrae, accuracy and plausibility were analyzed in these structures and inside thelungs. Across cases, the constrained methods improved landmark alignment compared to baseline models, increasing the proportion of cases with mean TRE below 3mm. Local rigidity constraints significantly reduced unphysical deformations in rigid bones as indicated by the Jacobian determinant mappings and analysis on the residual energy of the Jacobian orthogonality. Analysis of thoracic CT images showed improved alignment of the ribs and vertebrae with marginal increase in TRE of the landmarks in the lungs. The added constraints increased the runtime to a maximum of s. The proposed framework enforces local rigidity constraints during DIR, increasing accuracy without significantly compromising speed. It removes anatomically implausible deformations in rigid structures. Its efficiency and anatomical reliability make it well-suited for CBCT-guided adaptiveradiotherapy.

FocusMorph: A novel multi-scale fusion network for 3D brain MR image registration

Early-stage breast cancer is often asymptomatic, highlighting the critical role of computer-aided diagnostic (CAD) systems in mammography screening. While radiologists often refer to bilateral symmetry to identify abnormalities, most existing CAD methods analyze unilateral views or require image registration, which limits their ability to model structural heterogeneity and often introduces distortion. To address this, we propose a registration-free, structure-aware diagnostic framework that integrates bilateral mammography with soft spatial prompting via Vision Transformers (ViT). By directly concatenating bilateral images and introducing a soft attention mask generated from a lightweight segmentation network, our approach enables end-to-end modeling of cross-breast structural differences without the need for region-of-interest extraction. Extensive evaluations on both public and clinical datasets demonstrate that our method consistently outperforms CNN and lightweight Transformer baselines, achieving up to 0.930 accuracy and 0.972 AUC. To our knowledge, this is the first framework to combine bilateral structural modeling and soft guidance in a unified, interpretable, and scalable ViT-based pipeline for breast cancer diagnosis.

Image Quality Assessment (IQA) with references plays an important role in optimizing and evaluating computer vision tasks. Traditional methods assume that all pixels of the reference and test images are fully aligned. Such Aligned-Reference IQA (AR-IQA) approaches fail to address many real-world problems with various geometric deformations between the two images. Although significant effort has been made to attack Geometrically-Disparate-Reference IQA (GDR-IQA) problem, it has been addressed in a task-dependent fashion, for example, by dedicated designs for image super-resolution and retargeting, or by assuming the geometric distortions to be small that can be countered by translation-robust filters or by explicit image registrations. Here we rethink this problem and propose a unified, non-training-based Deep Structural Similarity (DeepSSIM) approach to address the above problems in a single framework, which assesses structural similarity of deep features in a simple but efficient way and uses an attention calibration strategy to alleviate attention deviation. The proposed method, without application-specific design, achieves state-of-the-art performance on AR-IQA datasets and meanwhile shows strong robustness to various GDR-IQA test cases. Interestingly, our test also shows the effectiveness of DeepSSIM as an optimization tool for training image super-resolution, enhancement and restoration, implying an even wider generalizability.

Isolated cardiac trabeculae are small heart muscle tissue preparations, which have been widely used in in vitro studies of mechanics and energetics function of cardiac muscle. Current instruments for such experimentation often (1) involve delicate mounting of the muscle, (2) constrain investigations to one muscle at a time, and thus (3) limit experimental throughput. Here, we present a novel device that allows trabeculae to be secured by a visible-light photo-crosslinked hydrogel, manipulated via a robust motor-driven stainless steel cantilever, and their shortening and force production to be measured and controlled using feedback from real-time imaging. The device has multiple wells, making it amenable to high-throughput testing of muscle. We use our robust, accurate image registration techniques to measure cantilever and gel deformation during trabecula contraction and thereby provide a measure of trabecula shortening and force production during twitches. We apply methods to allow the trabecula to contract either isometrically or isotonically. The methods used in this device can be widely applied to the study of the mechanics of cardiac muscle samples in laboratories with available light microscopic systems.