Registration Of Magnetic Resonance Images Research Articles

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

PurposeAccurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma.MethodsThis study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms.ResultsThe ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU.ConclusionsThe developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

Brain magnetic resonance image registration based on parallel lightweight convolution and multi-scale fusion

Medical image registration plays an important role in medical diagnosis and treatment planning. However, the current registration methods based on deep learning still face some challenges, such as insufficient ability to extract global information, large number of network model parameters, slow reasoning speed and so on. Therefore, this paper proposed a new model LCU-Net, which used parallel lightweight convolution to improve the ability of global information extraction. The problem of large number of network parameters and slow inference speed was solved by multi-scale fusion. The experimental results showed that the Dice coefficient of LCU-Net reached 0.823, the Hausdorff distance was 1.258, and the number of network parameters was reduced by about one quarter compared with that before multi-scale fusion. The proposed algorithm shows remarkable advantages in medical image registration tasks, and it not only surpasses the existing comparison algorithms in performance, but also has excellent generalization performance and wide application prospects.

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Acute toxicity outcomes from salvage high-dose-rate brachytherapy for locally recurrent prostate cancer after prior radiotherapy.

Isolated intra-prostatic recurrence of prostate adenocarcinoma after definitive radiotherapy presents a challenging clinical scenario. Salvage options require specialized expertise and pose risks of harm. This study aimed to present the acute toxicity results from using salvage high-dose-rate brachytherapy (sHDR-BT) as treatment in locally recurrent prostate cancer cases. Seventeen consecutive patients treated with sHDR-BT between 2019 and 2022 were evaluated retrospectively. Eligible patients had to have received curative intent prostate radiotherapy previously, and showed evidence of new biochemical failure. Evaluation with American Urological Association (AUA) and Common Terminology Criteria for Adverse Events (CTCAE) symptom assessments were performed for each case. The median (inter-quartile range) age prior to salvage treatment was 68 (66-74) years. The median post-sHDR-BT follow-up time was 20 (13-24) months. At baseline prior to sHDR-BT, 8 (47%) patients had significant lower urinary tract symptoms. The median AUA score prior to sHDR-BT was 7 (3-18). Three (18%) patients reported irregular bowel function and 2 (12%) reported hematochezia prior to sHDR-BT. One-month post-treatment, the median AUA score was 13 (8-21, p = 0.21). Using CTCAE scoring, there were no cases of grade 2+ bowel or rectal toxicity, and no cases of grade 3+ urinary toxicity. Reported grade 2 urinary toxicities included 10 (59%) cases of bladder spasms, 2 (12%) cases of incontinence, 1 (6%) urinary obstruction, and 4 (24%) reports of urinary urgency. All these adverse events were temporary. This study adds to the existing literature by demonstrating that the acute toxicity profile of sHDR-BT is acceptable even without intra-operative magnetic resonance (MR) guidance or image registration. Further study is ongoing to determine long-term efficacy and toxicity of treatment.

HCS-Net: Multi-level deformation strategy combined with quadruple attention for image registration

Background and objectiveNon-rigid image registration plays a significant role in computer-aided diagnosis and surgical navigation for brain diseases. Registration methods that utilize convolutional neural networks (CNNs) have shown excellent accuracy when applied to brain magnetic resonance images (MRI). However, CNNs have limitations in understanding long-range spatial relationships in images, which makes it challenging to incorporate contextual information. And in intricate image registration tasks, it is difficult to achieve a satisfactory dense prediction field, resulting in poor registration performance. MethodsThis paper proposes a multi-level deformable unsupervised registration model that combines Transformer and CNN to achieve non-rigid registration of brain MRI. Firstly, utilizing a dual encoder structure to establish the dependency relationship between the global features of two images and to merge features of varying scales, as well as to preserve the relative spatial position information of feature maps at different scales. Then the proposed multi-level deformation strategy utilizes different deformable fields of varying resolutions generated by the decoding structure to progressively deform the moving image. Ultimately, the proposed quadruple attention module is incorporated into the decoding structure to merge feature information from various directions and emphasize the spatial features in the dominant channels. ResultsThe experimental results on multiple brain MR datasets demonstrate that the promising network could provide accurate registration and is comparable to state-of-the-art methods. ConclusionThe proposed registration model can generate superior deformable fields and achieve more precise registration effects, enhancing the auxiliary role of medical image registration in various fields and advancing the development of computer-aided diagnosis, surgical navigation, and related domains.

A multi-view assisted registration network for MRI registration pre- and post-therapy.

Image registration of magnetic resonance imaging (MRI) pre- and post-therapy is an important part of evaluating the effect of therapy in tumor patients. The accuracy of evaluation results heavily relies on the alignment of the MRI image after registration. Although recent advancements have been made in medical image registration, applying these methods to MRI registration pre- and post-therapy remains challenging. Existing methods typically utilize single-view data for registration. However, when applied to MRI data where some slices are clear while others are blurred, these methods can be misled by erroneous spatial information in the blurred regions, leading to poor registration outcomes. To mitigate the interference caused by erroneous spatial information in single-view data, this paper proposes a multi-stream fusion-assisted registration network that incorporates different-view MRIs of the same patient at the same site. Additionally, a cross-attention guided fusion module is designed within the network to effectively utilize accurate spatial information from multi-view data. The proposed approach was evaluated on clinical data, and the experimental results demonstrated that incorporating multiple view data as auxiliary information significantly enhances the accuracy of MRI image registration before and after radiotherapy.

Definition and evaluation of a finite element model of the human heel for diabetic foot ulcer prevention under shearing loads.

Diabetic foot ulcers are triggered by mechanical loadings applied to the surface of the plantar skin. Strain is considered to play a crucial role in relation to ulcer etiology and can be assessed by Finite Element (FE) modeling. A difficulty in the generation of these models is the choice of the soft tissue material properties. In the literature, many studies attempt to model the behavior of the heel soft tissues by implementing constitutive laws that can differ significantly in terms of mechanical response. Moreover, current FE models lack of proper evaluation techniques that could estimate their ability to simulate realistic strains. In this article, we propose and evaluate a FE model of the human heel for diabetic foot ulcer prevention. Soft tissue constitutive laws are defined through the fitting of experimental stretch-stress curves published in the literature. The model is then evaluated through Digital Volume Correlation (DVC) based on non-rigid 3D Magnetic Resonance Image Registration. The results from FE analysis and DVC show similar strain locations in the fat pad and strain intensities according to the type of applied loads. For additional comparisons, different sets of constitutive models published in the literature are applied into the proposed FE mesh and simulated with the same boundary conditions. In this case, the results in terms of strains show great diversity in locations and intensities, suggesting that more research should be developed to gain insight into the mechanical properties of these tissues.

Multiresolution discrete optimization registration method of ultrasound and magnetic resonance images based on key points

The registration of preoperative magnetic resonance (MR) images and intraoperative ultrasound (US) images is very important in the planning of brain tumor surgery and during surgery. Considering that the two-modality images have different intensity range and resolution, and the US images are degraded by lots of speckle noises, a self-similarity context (SSC) descriptor based on local neighborhood information was adopted to define the similarity measure. The ultrasound images were considered as the reference, the corners were extracted as the key points using three-dimensional differential operators, and the dense displacement sampling discrete optimization algorithm was adopted for registration. The whole registration process was divided into two stages including the affine registration and the elastic registration. In the affine registration stage, the image was decomposed using multi-resolution scheme, and in the elastic registration stage, the displacement vectors of key points were regularized using the minimum convolution and mean field reasoning strategies. The registration experiment was performed on the preoperative MR images and intraoperative US images of 22 patients. The overall error after affine registration was (1.57 ± 0.30) mm, and the average computation time of each pair of images was only 1.36 s; while the overall error after elastic registration was further reduced to (1.40 ± 0.28) mm, and the average registration time was 1.53 s. The experimental results show that the proposed method has prominent registration accuracy and high computational efficiency.

Decreased gray matter volume and dynamic functional alterations in medicine-free obsessive-compulsive disorder

BackgroundPrevious studies discovered the presence of abnormal structures and functions in the brain regions of patients with obsessive-compulsive disorder (OCD). Nevertheless, whether structural changes in brain regions are coupled with alterations in dynamic functional connectivity (dFC) at rest in medicine-free patients with OCD remains vague.MethodsThree-dimensional T1-weighed magnetic resonance imaging (MRI) and resting-state functional MRI were performed on 50 medicine-free OCD and 50 healthy controls (HCs). Firstly, the differences in gray matter volume (GMV) between OCD and HCs were compared. Then, brain regions with aberrant GMV were used as seeds for dFC analysis. The relationship of altered GMV and dFC with clinical parameters in OCD was explored using partial correlation analysis. Finally, support vector machine was applied to examine whether altered multimodal imaging data might be adopted to distinguish OCD from HCs.ResultsOur findings indicated that GMV in the left superior temporal gyrus (STG) and right supplementary motor area (SMA) was reduced in OCD, and the dFC between the left STG and the left cerebellum Crus I and left thalamus, and between the right SMA and right dorsolateral prefrontal cortex (DLPFC) and left precuneus was decreased at rest in OCD. The brain regions both with altered GMV and dFC values could discriminate OCD from HCs with the accuracy of 0.85, sensitivity of 0.90 and specificity of 0.80.ConclusionThe decreased gray matter structure coupling with dynamic function in the left STG and right SMA at rest may be crucial in the pathophysiology of OCD.Trial registrationStudy on the mechanism of brain network in obsessive-compulsive disorder with multi-model magnetic resonance imaging (registration date: 08/11/2017; registration number: ChiCTR-COC-17,013,301).

Craniofacial Soft-Tissue Anthropomorphic Database with Magnetic Resonance Imaging and Unbiased Diffeomorphic Registration.

Objective assessment of craniofacial surgery outcomes in a pediatric population is challenging because of the complexity of patient presentations, diversity of procedures performed, and rapid craniofacial growth. There is a paucity of robust methods to quantify anatomical measurements by age and objectively compare craniofacial dysmorphology and postoperative outcomes. Here, the authors present data in developing a racially and ethnically sensitive anthropomorphic database, providing plastic and craniofacial surgeons with "normal" three-dimensional anatomical parameters with which to appraise and optimize aesthetic and reconstructive outcomes. Patients with normal craniofacial anatomy undergoing head magnetic resonance imaging (MRI) scans from 2008 to 2021 were included in this retrospective study. Images were used to construct composite (template) images with diffeomorphic image registration method using the Advanced Normalization Tools package. Composites were thresholded to generate binary three-dimensional segmentations used for anatomical measurements in Materalise Mimics. High-resolution MRI scans from 130 patients generated 12 composites from an average of 10 MRI sequences each: four 3-year-olds, four 4-year-olds, and four 5-year-olds (two male, two female, two Black, and two White). The average head circumference of 3-, 4-, and 5-year-old composites was 50.3, 51.5, and 51.7 cm, respectively, comparable to normative data published by the World Health Organization. Application of diffeomorphic registration-based image template algorithm to MRI is effective in creating composite templates to represent "normal" three-dimensional craniofacial and soft-tissue anatomy. Future research will focus on development of automated computational tools to characterize anatomical normality, generation of indices to grade preoperative severity, and quantification of postoperative results to reduce subjectivity bias.

Multiscale feature fusion network for 3D head MRI image registration.

Image registration technology has become an important medical image preprocessing step with the wide application of computer-aided diagnosis technology in various medical image analysis tasks. We propose a multiscale feature fusion registration based on deep learning to achieve the accurate registration and fusion of head magnetic resonance imaging (MRI) and solve the problem that general registration methods cannot handle the complex spatial information and position information of head MRI. Our proposed multiscale feature fusion registration network consists of three sequentially trained modules. The first is an affine registration module that implements affine transformation; the second is to realize non-rigid transformation, a deformable registration module composed of top-down and bottom-up feature fusion subnetworks in parallel; and the third is a deformable registration module that also realizes non-rigid transformation and is composed of two feature fusion subnetworks in series. The network decomposes the deformation field of large displacement into multiple deformation fields of small displacement by multiscale registration and registration, which reduces the difficulty of registration. Moreover, multiscale information in head MRI is learned in a targeted manner, which improves the registration accuracy, by connecting the two feature fusion subnetworks. We used 29 3D head MRIs for training and seven volumes for testing and calculated the values of the registration evaluation metrics for the new algorithm to register anterior and posterior lateral pterygoid muscles. The Dice similarity coefficient was 0.745±0.021, the Hausdorff distance was 3.441±0.935mm, the Average surface distance was 0.738±0.098mm, and the Standard deviation of the Jacobian matrix was 0.425±0.043. Our new algorithm achieved a higher registration accuracy compared with state-of-the-art registration methods. Our proposed multiscale feature fusion registration network can realize end-to-end deformable registration of 3D head MRI, which can effectively cope with the characteristics of large deformation displacement and the rich details of head images and provide reliable technical support for the diagnosis and analysis of head diseases.

Meta-Learning Initializations for Interactive Medical Image Registration.

We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ultrasound (TRUS) images. Our approach obtains comparable registration error (4.26 mm) to the best-performing non-interactive learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the data, and occurring in real-time during acquisition. Applying sparsely sampled data to non-interactive methods yields higher registration errors (6.26 mm), demonstrating the effectiveness of interactive MR-TRUS registration, which may be applied intraoperatively given the real-time nature of the adaptation process.

Multimodal registration of ultrasound and MR images using weighted self-similarity structure vector

ProposeMultimodal registration of 2D Ultrasound (US) and 3D Magnetic Resonance (MR) for fusion navigation can improve the intraoperative detection accuracy of lesion. However, multimodal registration remains a challenge because of the poor US image quality. In the study, a weighted self-similarity structure vector (WSSV) is proposed to registrate multimodal images. MethodThe self-similarity structure vector utilizes the normalized distance of symmetrically located patches in the neighborhood to describe the local structure information. The texture weights are extracted using the local standard deviation to reduce the speckle interference in the US images. The multimodal similarity metric is constructed by combining a self-similarity structure vector with a texture weight map. ResultsExperiments were performed on US and MR images of the liver from 88 groups of data including 8 patients and 80 simulated samples. The average target registration error was reduced from 14.91 ± 3.86 mm to 4.95 ± 2.23 mm using the WSSV-based method. ConclusionsThe experimental results show that the WSSV-based registration method could robustly align the US and MR images of the liver. With further acceleration, the registration framework can be potentially applied in time-sensitive clinical settings, such as US-MR image registration in image-guided surgery.

Geometric Deep Learning for Unsupervised Registration of Diffusion Magnetic Resonance Images.

Deep learning based models for registration predict a transformation directly from moving and fixed image appearances. These models have revolutionized the field of medical image registration, achieving accuracy on-par with classical registration methods at a fraction of the computation time. Unfortunately, most deep learning based registration methods have focused on scalar imaging modalities such as T1/T2 MRI and CT, with less attention given to more complex modalities such as diffusion MRI. In this paper, to the best of our knowledge, we present the first end-to-end geometric deep learning based model for the non-rigid registration of fiber orientation distribution fields (fODF) derived from diffusion MRI (dMRI). Our method can be trained in a fully-unsupervised fashion using only input fODF image pairs, i.e. without ground truth deformation fields. Our model introduces several novel differentiable layers for local Jacobian estimation and reorientation that can be seamlessly integrated into the recently introduced manifold-valued convolutional network in literature. The results of this work are accurate deformable registration algorithms for dMRI data that can execute in the order of seconds, as opposed to dozens of minutes to hours consumed by their classical counterparts.

Quality assurance assessment of intra-acquisition diffusion-weighted and T2-weighted magnetic resonance imaging registration and contour propagation for head and neck cancer radiotherapy.

Adequate image registration of anatomical and functional magnetic resonance imaging (MRI) scans is necessary for MR-guided head and neck cancer (HNC) adaptive radiotherapy planning. Despite the quantitative capabilities of diffusion-weighted imaging (DWI) MRI for treatment plan adaptation, geometric distortion remains a considerable limitation. Therefore, we systematically investigated various deformable image registration (DIR) methods to co-register DWI and T2-weighted (T2W) images. We compared three commercial (ADMIRE, Velocity, Raystation) and three open-source (Elastix with default settings [Elastix Default], Elastix with parameter set 23 [Elastix 23], Demons) post-acquisition DIR methods applied to T2W and DWI MRI images acquired during the same imaging session in twenty immobilized HNC patients. In addition, we used the non-registered images (None) as a control comparator. Ground-truth segmentations of radiotherapy structures (tumour and organs at risk) were generated by a physician expert on both image sequences. For each registration approach, structures were propagated from T2W to DWI images. These propagated structures were then compared with ground-truth DWI structures using the Dice similarity coefficient and mean surface distance. 19 left submandibular glands, 18 right submandibular glands, 20left parotid glands, 20 right parotid glands, 20 spinal cords, and 12 tumours were delineated. Most DIR methods took <30s to execute per case, with the exception of Elastix 23 which took ∼458s to execute per case. ADMIRE and Elastix 23 demonstrated improved performance over None for all metrics and structures (Bonferroni-corrected p<0.05), while the other methods did not. Moreover, ADMIRE and Elastix 23 significantly improved performance in individual and pooled analysis compared to all other methods. The ADMIRE DIR method offers improved geometric performance with reasonable execution time so should be favoured for registering T2W and DWI images acquired during the same scan session in HNC patients. These results are important to ensure the appropriate selection of registration strategies for MR-guided radiotherapy.

Towards 2D/3D Registration of the Preoperative MRI to Intraoperative Fluoroscopic Images for Visualisation of Bone Defects

ABSTRACT Magnetic Resonance Imaging (MRI) is a medical imaging modality that allows for the evaluation of soft-tissue diseases and the assessment of bone quality. Preoperative MRI volumes are used by surgeons to identify defected bones, perform the segmentation of lesions, and generate surgical plans before the surgery. Nevertheless, conventional intraoperative imaging modalities such as fluoroscopy are less sensitive in detecting potential lesions. In this work, we propose a 2D/3D registration pipeline that aims to register preoperative MRI with intraoperative 2D fluoroscopic images. To showcase the feasibility of our approach, we use the core decompression procedure as a surgical example to perform 2D/3D femur registration. The proposed registration pipeline is evaluated using digitally reconstructed radiographs (DRRs) to simulate the intraoperative fluoroscopic images. The resulting transformation from the registration is later used to create overlays of preoperative MRI annotations and planning data to provide intraoperative visual guidance to surgeons. Our results suggest that the proposed registration pipeline is capable of achieving reasonable transformation between MRI and digitally reconstructed fluoroscopic images for intraoperative visualisation applications.

Supine magnetic resonance image registration for breast surgery: insights on material mechanics.

Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, is being explored since it more closely represents the surgical presentation compared to conventional diagnostic imaging positions. Despite this preoperative imaging position, deformation is still present between the supine imaging and surgical state. As a result, a fast and accurate image-to-physical registration approach is needed to realize image-guided breast surgery. In this study, three registration methods were investigated on healthy volunteers' breasts ( ) with the supine arm-down position simulating preoperative imaging and supine arm-up position simulating intraoperative presentation. The registration methods included (1)point-based rigid registration using synthetic fiducials, (2)nonrigid biomechanical model-based registration using sparse data, and (3)a data-dense three-dimensional diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. Additionally, deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field to better understand breast material mechanics. The average target registration errors (TRE) were , , and (mean ± standard deviation) and the average fiducial registration errors (FRE) were , , and for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. The mechanics-based deformation metrics revealed an overall anisotropic tissue behavior and a statistically significant difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy. Overall, registration accuracy significantly improved with increasingly flexible and data-dense registration methods. Analysis of these outcomes may inform the future development of image guidance systems for lumpectomy procedures.

Latent shape image learning via disentangled representation for cross-sequence image registration and segmentation.

Cross-sequence magnetic resonance image (MRI) registration and segmentation are two essential steps in a variety of medical image analysis tasks. And have attracted considerable research interest. However, they remain challenging due to domain shifts between different sequences. This study is aiming at proposing a novel method via disentangled representations, latent shape image learning (LSIL), for cross-sequence image registration and segmentation. Images from different sequences were firstly decomposed into a shared domain-invariant shape space and a domain-specific appearance space via an unsupervised image-to-image translation approach. A latent shape image learning model is then built on the disentangled shape representations to generate latent shape images. A series of experiments including cross-sequence image registration and segmentation were performed to qualitatively and quantitatively verify the validity of our method. Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95) were adopted as our evaluation metrics. The performance of our method was evaluated based on 2 datasets total of 50 MRIs. The experimental results showed the superiority of the proposed framework over the state-of-the-art cross-sequence registration and segmentation approaches. The proposed method shows the mean DSCs of 0.711 and 0.867, respectively, in cross-sequence registration and segmentation. We proposed a novel method based on representation disentangling to solve the cross-sequence registration and segmentation problem. Experimental results prove the feasibility and generalization of the generated latent shape images. The proposed method demonstrates significant potential for use in clinical environments of missing sequences. The source code is available at https://github.com/wujiong-hub/LSIL .

Quantitative evaluation of the influence of multiple MRI sequences and of pathological tissues on the registration of longitudinal data acquired during brain tumor treatment.

Registration methods facilitate the comparison of multiparametric magnetic resonance images acquired at different stages of brain tumor treatments. Image-based registration solutions are influenced by the sequences chosen to compute the distance measure, and the lack of image correspondences due to the resection cavities and pathological tissues. Nonetheless, an evaluation of the impact of these input parameters on the registration of longitudinal data is still missing. This work evaluates the influence of multiple sequences, namely T1-weighted (T1), T2-weighted (T2), contrast enhanced T1-weighted (T1-CE), and T2 Fluid Attenuated Inversion Recovery (FLAIR), and the exclusion of the pathological tissues on the non-rigid registration of pre- and post-operative images. We here investigate two types of registration methods, an iterative approach and a convolutional neural network solution based on a 3D U-Net. We employ two test sets to compute the mean target registration error (mTRE) based on corresponding landmarks. In the first set, markers are positioned exclusively in the surroundings of the pathology. The methods employing T1-CE achieves the lowest mTREs, with a improvement up to 0.8 mm for the iterative solution. The results are higher than the baseline when using the FLAIR sequence. When excluding the pathology, lower mTREs are observable for most of the methods. In the second test set, corresponding landmarks are located in the entire brain volumes. Both solutions employing T1-CE obtain the lowest mTREs, with a decrease up to 1.16 mm for the iterative method, whereas the results worsen using the FLAIR. When excluding the pathology, an improvement is observable for the CNN method using T1-CE. Both approaches utilizing the T1-CE sequence obtain the best mTREs, whereas the FLAIR is the least informative to guide the registration process. Besides, the exclusion of pathology from the distance measure computation improves the registration of the brain tissues surrounding the tumor. Thus, this work provides the first numerical evaluation of the influence of these parameters on the registration of longitudinal magnetic resonance images, and it can be helpful for developing future algorithms.

Deep learning facilitates fully automated brain image registration of optoacoustic tomography and magnetic resonance imaging.

Multispectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) which offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of MSOT-MRI images remains challenging, chiefly due to the entirely different image contrast rendered by these two modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation and quantification. Here we propose a fully automated registration method for MSOT-MRI multimodal imaging empowered by deep learning. The automated workflow includes neural network-based image segmentation to generate suitable masks, which are subsequently registered using an additional neural network. The performance of the algorithm is showcased with datasets acquired by cross-sectional MSOT and high-field MRI preclinical scanners. The automated registration method is further validated with manual and half-automated registration, demonstrating its robustness and accuracy.