Breast Image Registration Research Articles

Automated marker-free longitudinal infrared breast image registration by GA-PSO

The automated marker-free longitudinal Infrared (IR) breast image registration overcomes several challenges like no anatomic fiducial markers on the body surface, blurry boundaries, heat pattern variation by environmental and physiological factors, nonrigid deformation, etc, has the ability of quantitative pixel-wise analysis with the heat energy and patterns change in a time course study. To achieve the goal, scale-invariant feature transform, Harris corner, and Hessian matrix were employed to generate the feature points as anatomic fiducial markers, and hybrid genetic algorithm and particle swarm optimization minimizing the matching errors was used to find the appropriate corresponding pairs between the 1st IR image and the nth IR image. Moreover, the mechanism of the IR spectrogram hardware system has a high level of reproducibility. The performance of the proposed longitudinal image registration system was evaluated by the simulated experiments and the clinical trial. In the simulated experiments, the mean difference of our system is 1.64 mm, which increases 57.58% accuracy than manual determination and makes a 17.4% improvement than the previous study. In the clinical trial, 80 patients were captured several times of IR breast images during chemotherapy. Most of them were well aligned in the spatiotemporal domain. In the few cases with evident heat pattern dissipation and spatial deviation, it still provided a reliable comparison of vascular variation. Therefore, the proposed system is accurate and robust, which could be considered as a reliable tool for longitudinal approaches to breast cancer diagnosis.

Breast image registration for surgery: Insights on material mechanics modeling.

Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, more closely represents the surgical presentation compared to conventional diagnostic pendant positioning. Optimal utilization for surgical guidance, however, requires a fast and accurate image-to-physical registration from preoperative imaging to intraoperative surgical presentation. In this study, three registration methods were investigated on healthy volunteers' breasts (n=11) with the arm-down position simulating preoperative imaging and arm-up position simulating intraoperative data. The registration methods included: (1) point-based rigid registration using synthetic fiducials, (2) non-rigid biomechanical model-based registration using sparse data, and (3) a data-dense 3D diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. The average target registration errors (TRE) were 10.4 ± 2.3, 6.4 ± 1.5, and 2.8 ± 1.3 mm (mean ± standard deviation) and the average fiducial registration errors (FRE) were 7.8 ± 1.7, 2.5 ± 1.1, and 3.1 ± 1.1 mm (mean ± standard deviation) for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. Additionally, common mechanics-based deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field. The average metrics revealed anisotropic tissue behavior and a statistical difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy. Overall, registration accuracy significantly improved with increasingly flexible registration methods, which may inform future development of image guidance systems for lumpectomy procedures.

Review of Breast Cancer Pathologigcal Image Processing.

Breast cancer is one of the most common malignancies. Pathological image processing of breast has become an important means for early diagnosis of breast cancer. Using medical image processing to assist doctors to detect potential breast cancer as early as possible has always been a hot topic in the field of medical image diagnosis. In this paper, a breast cancer recognition method based on image processing is systematically expounded from four aspects: breast cancer detection, image segmentation, image registration, and image fusion. The achievements and application scope of supervised learning, unsupervised learning, deep learning, CNN, and so on in breast cancer examination are expounded. The prospect of unsupervised learning and transfer learning for breast cancer diagnosis is prospected. Finally, the privacy protection of breast cancer patients is put forward.

Preliminary Feasibility Study of Imaging Registration Between Supine and Prone Breast CT in Breast Cancer Radiotherapy Using Residual Recursive Cascaded Networks

Breast cancer is one of the most common malignancies in women. The prone position in Partial Breast Irradiation (PBI) can better protect the heart and lung during radiotherapy. Supine position is used for CT imaging during treatment planning. The posture change in these two different positions may cause large deformation of breast, which make breast registration become a great challenge. Existing registration approaches for supine and prone breast images mainly use biomechanical modeling and iterative deformable images registration methods. However, the ability of these methods to capture such large deformations is limited. To tackle these problems, we propose an end-to-end residual recursive cascade network (RRCN) for supine and prone breast images registration. Unlike traditional deep learning networks, an affine subnetwork and several deformable subnetworks are trained together, enabling cooperation between subnetworks. Moreover, by using residual network connection, we can accelerate registration speed and reduce radiation dose. Registration accuracy is evaluated by visualizing registered images and computing normalized cross correlation (NCC). The experiment results show that RRCN with an average NCC of 0.982 ± 0.010 outperform VoxelMorph with an average NCC of 0.769 ± 0.070 and Recursive Cascaded Networks (RCN) with an average NCC of 0.914 ± 0.063, demonstrating the superior performance of the proposed method for supine and prone breast image registration. Because accurate deformable registration for this large-scale deformation is of great importance to the success of breast cancer radiotherapy, RRCN method has a strong potential to be a promising tool for future clinical practice in breast cancer radiotherapy.

Evaluation of a Coherent Point Drift Algorithm for Breast Image Registration via Surface Markers

Breast Magnetic Resonance Imaging (MRI) is a reliable imagingtool for localization and evaluation of lesions prior to breast conservingsurgery (BCS). MR images typically will be used to determinethe size and location of the tumours before making the incisionin order to minimize the amount of tissue excised.The arm position and configuration of the breast during andprior to surgery are different and one question is whether it wouldbe possible to match the two configurations. This matching processcan potentially be used in development of tools to guide surgeonsin the incision process.Recently, a Thin-Plate-Spline (TPS) algorithm has been proposedto assess the feasibility of breast tissue matching using fiducialsurface markers in two different arm positions. The registrationalgorithm uses the surface markers only and does not employ theimage intensities.In this manuscript, we apply and evaluate a coherent point drift(CPD) algorithm for registration of three-dimensional breast MR imagesof six patient volunteers. In particular, we evaluate the resultsof the previous TPS registration technique to the proposed rigidCPD, affine CPD, and deformable CPD registration algorithms onthe same patient datasets.The preliminary results suggest that the CPD deformable registrationalgorithm is superior in correcting the motion of the breastcompared to CPD rigid, affine and TPS registration algorithms.

Symmetric Biomechanically Guided Prone-to-Supine Breast Image Registration.

Prone-to-supine breast image registration has potential application in the fields of surgical and radiotherapy planning, image guided interventions, and multi-modal cancer diagnosis, staging, and therapy response prediction. However, breast image registration of three dimensional images acquired in different patient positions is a challenging problem, due to large deformations induced to the soft breast tissue caused by the change in gravity loading. We present a symmetric, biomechanical simulation based registration framework which aligns the images in a central, virtually unloaded configuration. The breast tissue is modelled as a neo-Hookean material and gravity is considered as the main source of deformation in the original images. In addition to gravity, our framework successively applies image derived forces directly into the unloading simulation in place of a subsequent image registration step. This results in a biomechanically constrained deformation. Using a finite difference scheme avoids an explicit meshing step and enables simulations to be performed directly in the image space. The explicit time integration scheme allows the motion at the interface between chest and breast to be constrained along the chest wall. The feasibility and accuracy of the approach presented here was assessed by measuring the target registration error (TRE) using a numerical phantom with known ground truth deformations, nine clinical prone MRI and supine CT image pairs, one clinical prone-supine CT image pair and four prone-supine MRI image pairs. The registration reduced the mean TRE for the numerical phantom experiment from initially 19.3 to 0.9 mm and the combined mean TRE for all fourteen clinical data sets from 69.7 to 5.6 mm.

Simulation of mammographic breast compression in 3D MR images using ICP-based B-spline deformation for multimodality breast cancer diagnosis

Multimodality mammography using conventional 2D mammography and dynamic contrast-enhanced 3D magnetic resonance imaging (DCE-MRI) is frequently performed for breast cancer detection and diagnosis. Combination of both imaging modalities requires superimposition of corresponding structures in mammograms and MR images. This task is challenging due to large differences in (1) dimensionality and spatial resolution, (2) variations in tissue contrast, as well as (3) differences in breast orientation and deformation during the image acquisition. A new method for multimodality breast image registration was developed and tested. Combined diagnosis of mammograms and MRI datasets was achieved by simulation of mammographic breast compression to overcome large differences in breast deformation. Surface information was extracted from the 3D MR image, and back-projection of the 2D breast contour in the mammogram was done. B-spline-based 3D/3D surface-based registration was then used to approximate mammographic breast compression. This breast deformation simulation was performed on 14 MRI datasets with 19 corresponding mammograms. The results were evaluated by comparison with distances between corresponding structures identified by an expert observer. The evaluation revealed an average distance of 6.46 mm between corresponding structures, when an optimized initial alignment between both image datasets is performed. Without the optimization, the accuracy is 9.12 mm. A new surface-based method that approximates the mammographic deformation due to breast compression without using a specific complex model needed for finite-element-based methods was developed and tested with favorable results. The simulated compression can serve as foundation for a point-to-line correspondence between 2D mammograms and 3D MR image data.

Breast Image Registration for PET-CT and MR based on 3D Surface Matching

In this paper, we explore the PET-MRI 3D surface image registration method for the longitudinal study of cancer. Image registration gives new information to the radiologists by matching images with different modalities. The objective of this research is to develop 3D surface registration for PET/CT and MR images acquired by different medical imaging systems at different times. Our proposed method consists of three stages: 1) global surface matching, 2) fine adjustment stage and 3) image transformation. In this study, we perform the global registration with surface points within CT and MRI using ICP algorithm. Then we perform the fine adjusts stage using ECICP algorithm. Finally, PET image was transformed with various parameters and overlaid onto MR image.

Automatic multimodal 2D/3D breast image registration using biomechanical FEM models and intensity-based optimization

Due to their different physical origin, X-ray mammography and Magnetic Resonance Imaging (MRI) provide complementary diagnostic information. However, the correlation of their images is challenging due to differences in dimensionality, patient positioning and compression state of the breast. Our automated registration takes over part of the correlation task. The registration method is based on a biomechanical finite element model, which is used to simulate mammographic compression. The deformed MRI volume can be compared directly with the corresponding mammogram. The registration accuracy is determined by a number of patient-specific parameters. We optimize these parameters--e.g. breast rotation--using image similarity measures. The method was evaluated on 79 datasets from clinical routine. The mean target registration error was 13.2mm in a fully automated setting. On basis of our results, we conclude that a completely automated registration of volume images with 2D mammograms is feasible. The registration accuracy is within the clinically relevant range and thus beneficial for multimodal diagnosis.

Mono- and multimodal registration of optical breast images

Optical breast imaging offers the possibility of noninvasive, low cost, and high sensitivity imaging of breast cancers. Poor spatial resolution and a lack of anatomical landmarks in optical images of the breast make interpretation difficult and motivate registration and fusion of these data with subsequent optical images and other breast imaging modalities. Methods used for registration and fusion of optical breast images are reviewed. Imaging concerns relevant to the registration problem are first highlighted, followed by a focus on both monomodal and multimodal registration of optical breast imaging. Where relevant, methods pertaining to other imaging modalities or imaged anatomies are presented. The multimodal registration discussion concerns digital x-ray mammography, ultrasound, magnetic resonance imaging, and positron emission tomography.

Breast Image Registration and Deformation Modeling

Image-based examination of the breast facilitates the detection of breast diseases, particularly of present benign and malignant lesions. For computer-aided processing of serial and multimodal clinical data, both for visual correlation and quantitative analysis, automated image-registration methods are an indispensable tool. The wide range of modalities and the high variability of breast appearance have led to a large diversity of proposed approaches for tissue deformation modeling and image registration. In this article, we review current developments in breast image registration techniques, and comment on their clinical relevance, individual capabilities, and open challenges.

Semiautomated multimodal breast image registration.

Consideration of information from multiple modalities has been shown to have increased diagnostic power in breast imaging. As a result, new techniques such as microwave imaging continue to be developed. Interpreting these novel image modalities is a challenge, requiring comparison to established techniques such as the gold standard X-ray mammography. However, due to the highly deformable nature of breast tissues, comparison of 3D and 2D modalities is a challenge. To enable this comparison, a registration technique was developed to map features from 2D mammograms to locations in the 3D image space. This technique was developed and tested using magnetic resonance (MR) images as a reference 3D modality, as MR breast imaging is an established technique in clinical practice. The algorithm was validated using a numerical phantom then successfully tested on twenty-four image pairs. Dice's coefficient was used to measure the external goodness of fit, resulting in an excellent overall average of 0.94. Internal agreement was evaluated by examining internal features in consultation with a radiologist, and subjective assessment concludes that reasonable alignment was achieved.

Sci-Fri AM: Imaging - 03: Automated Registration of X-Ray Mammograms and Magnetic Resonance Breast Images

Breast cancer is a common and devastating form of cancer, with an estimated 22,700 new cases in 2009 in Canada alone. X‐ray mammography is the most commonly used imaging technique for detection and diagnosis, while magnetic resonance imaging(MRI) is used in some challenging cases. Both modalities rely on different properties of the tissue to form images, and thus contribute different and complimentary information about the breast. However, due to geometric distortions during the acquisition processes, it is difficult to identify and compare the same anatomical location on both modalities. In this work, a method to register 2D mammograms to projection images of MRI volumes is presented. In order to compare the 3D MRI to the 2D mammogram, a “simulated mammogram” is formed from the MRI volume. Three anatomical landmarks on the surface of the breast are identified on each image and aligned to distort the general shape of the mammogram to match that of the MR projection image. Final registration is then achieved by iteratively applying a non‐linear transformation to the mammogram until the mutual information of the two images is maximized. The registration method was tested on eight pairs of images from two volunteers (two mammographic views from each breast). Results from this small dataset are promising, with an average alignment error of 3.9%, measured as the difference in area between the two registered images. Future work will examine a larger dataset, including pathological cases, and quantification of internal alignment errors.

Validation of an algorithm for the nonrigid registration of longitudinal breast MR images using realistic phantoms

The authors present a method to validate coregistration of breast magnetic resonance images obtained at multiple time points during the course of treatment. In performing sequential registration of breast images, the effects of patient repositioning, as well as possible changes in tumor shape and volume, must be considered. The authors accomplish this by extending the adaptive bases algorithm (ABA) to include a tumor-volume preserving constraint in the cost function. In this study, the authors evaluate this approach using a novel validation method that simulates not only the bulk deformation associated with breast MR images obtained at different time points, but also the reduction in tumor volume typically observed as a response to neoadjuvant chemotherapy. For each of the six patients, high-resolution 3D contrast enhanced T1-weighted images were obtained before treatment, after one cycle of chemotherapy and at the conclusion of chemotherapy. To evaluate the effects of decreasing tumor size during the course of therapy, simulations were run in which the tumor in the original images was contracted by 25%, 50%, 75%, and 95%, respectively. The contracted area was then filled using texture from local healthy appearing tissue. Next, to simulate the post-treatment data, the simulated (i.e., contracted tumor) images were coregistered to the experimentally measured post-treatment images using a surface registration. By comparing the deformations generated by the constrained and unconstrained version of ABA, the authors assessed the accuracy of the registration algorithms. The authors also applied the two algorithms on experimental data to study the tumor volume changes, the value of the constraint, and the smoothness of transformations. For the six patient data sets, the average voxel shift error (mean +/- standard deviation) for the ABA with constraint was 0.45 +/- 0.37, 0.97 +/- 0.83, 1.43 +/- 0.96, and 1.80 +/- 1.17 mm for the 25%, 50%, 75%, and 95% contraction simulations, respectively. In comparison, the average voxel shift error for the unconstrained ABA was 0.46 +/- 0.29, 1.13 +/- 1.17, 2.40 +/- 2.04, and 3.53 +/- 2.89 mm, respectively. These voxel shift errors translate into compression of the tumor volume: The ABA with constraint returned volumetric errors of 2.70 +/- 4.08%, 7.31 +/- 4.52%, 9.28 +/- 5.55%, and 13.19 +/- 6.73% for the 25%, 50%, 75%, and 95% contraction simulations, respectively, whereas the unconstrained ABA returned volumetric errors of 4.00 +/- 4.46%, 9.93 +/- 4.83%, 19.78 +/- 5.657%, and 29.75 +/- 15.18%. The ABA with constraint yields a smaller mean shift error, as well as a smaller volume error (p = 0.031 25 for the 75% and 95% contractions), than the unconstrained ABA for the simulated sets. Visual and quantitative assessments on experimental data also indicate a good performance of the proposed algorithm. The ABA with constraint can successfully register breast MR images acquired at different time points with reasonable error. To the best of the authors' knowledge, this is the first report of an attempt to quantitatively assess in both phantoms and a set of patients the accuracy of a registration algorithm for this purpose.

Current and future trends in magnetic resonance imaging assessments of the response of breast tumors to neoadjuvant chemotherapy.

The current state-of-the-art assessment of treatment response in breast cancer is based on the response evaluation criteria in solid tumors (RECIST). RECIST reports on changes in gross morphology and divides response into one of four categories. In this paper we highlight how dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and diffusion-weighted MRI (DW-MRI) may be able to offer earlier, and more precise, information on treatment response in the neoadjuvant setting than RECIST. We then describe how longitudinal registration of breast images and the incorporation of intelligent bioinformatics approaches with imaging data have the potential to increase the sensitivity of assessing treatment response. We conclude with a discussion of the potential benefits of breast MRI at the higher field strength of 3T. For each of these areas, we provide a review, illustrative examples from clinical trials, and offer insights into future research directions.

Computerized method for nonrigid MR-to-PET breast-image registration

We have developed and tested a new simple computerized finite element method (FEM) approach to MR-to-PET nonrigid breast-image registration. The method requires five–nine fiducial skin markers (FSMs) visible in MRI and PET that need to be located in the same spots on the breast and two on the flanks during both scans. Patients need to be similarly positioned prone during MRI and PET scans. This is accomplished by means of a low gamma-ray attenuation breast coil replica used as the breast support during the PET scan. We demonstrate that, under such conditions, the observed FSM displacement vectors between MR and PET images, distributed piecewise linearly over the breast volume, produce a deformed FEM mesh that reasonably approximates nonrigid deformation of the breast tissue between the MRI and PET scans. This method, which does not require a biomechanical breast tissue model, is robust and fast. Contrary to other approaches utilizing voxel intensity-based similarity measures or surface matching, our method works for matching MR with pure molecular images (i.e. PET or SPECT only). Our method does not require a good initialization and would not be trapped by local minima during registration process. All processing including FSMs detection and matching, and mesh generation can be fully automated. We tested our method on MR and PET breast images acquired for 15 subjects. The procedure yielded good quality images with an average target registration error below 4 mm (i.e. well below PET spatial resolution of 6–7 mm). Based on the results obtained for 15 subjects studied to date, we conclude that this is a very fast and a well-performing method for MR-to-PET breast-image nonrigid registration. Therefore, it is a promising approach in clinical practice. This method can be easily applied to nonrigid registration of MRI or CT of any type of soft-tissue images to their molecular counterparts such as obtained using PET and SPECT.

Breast image registration techniques: a survey

Breast cancer is the most common type of cancer in women worldwide. Image registration plays an important role in breast cancer detection. This paper gives an overview of the current state-of-the-art in the breast image registration techniques. For the intramodality registration techniques, X-ray, MRI, and ultrasound are the primary focuses of interest. Intermodality techniques will cover the combination of different modalities. Validation of breast registration methods is also discussed.

MRI/PET nonrigid breast-image registration using skin fiducial markers

We propose a finite-element method (FEM) deformable breast model that does not require elastic breast data for nonrigid PET/MRI breast image registration. The model is applicable only if the stress conditions in the imaged breast are virtually the same in PET and MRI. Under these conditions, the observed intermodality displacements are solely due the imaging/reconstruction process. Similar stress conditions are assured by use of an MRI breast-antenna replica for breast support during PET, and use of the same positioning. The tetrahedral volume and triangular surface elements are used to construct the FEM mesh from the MRI image. Our model requires a number of fiducial skin markers (FSM) visible in PET and MRI. The displacement vectors of FSMs are measured followed by the dense displacement field estimation by first distributing the displacement, vectors linearly over the breast surface and then distributing them throughout the volume. Finally, the floating MRI image is warped to a fixed PET image, by using an appropriate shape function in the interpolation from mesh nodes to voxels. We tested our model on an elastic breast phantom with simulated internal lesions and on a small number of patients imaged, with FMS using PET and MRI. Using simulated lesions (in phantom) and real lesions (in patients) visible in both PET and MRI, we established that the target registration error (TRE) is below two pet voxels.

3D Breast Image Registration — A Review

Image registration is an important problem in breast imaging. It is used in a wide variety of applications that include better visualization of lesions on pre- and post-contrast breast MRI images, speckle tracking and image compounding in breast ultrasound images, alignment of positron emission, and standard mammography images on hybrid machines et cetera. It is a prerequisite to align images taken at different times to isolate small interval lesions. Image registration also has useful applications in monitoring cancer therapy. The field of breast image registration has gained considerable interest in recent years. While the primary focus of interest continues to be the registration of pre- and post-contrast breast MRI images, other areas like breast ultrasound registration have gained more attention in recent years. The focus of registration algorithms has also shifted from control point based semi-automated techniques, to more sophisticated voxel based automated techniques that use mutual information as a similarity measure. This paper visits the problem of breast image registration and provides an overview of the current state-of-the-art in this area.

Image matching using alpha-entropy measures and entropic graphs

Matching a reference image to a secondary image extracted from a database of transformed exemplars constitutes an important image retrieval task. Two related problems are: specification of a general class of discriminatory image features and an appropriate similarity measure to rank the closeness of the query to the database. In this paper we present a general method based on matching high dimensional image features, using entropic similarity measures that can be empirically estimated using entropic graphs such as the minimal spanning tree (MST). The entropic measures we consider are generalizations of the well-known Kullback Liebler (KL) distance, the mutual information (MI) measure, and the Jensen difference. Our entropic graph approach has the advantage of being implementable for high dimensional feature spaces for which other entropy-based pattern matching methods are computationally difficult. We compare our technique to previous entropy matching methods for a variety of continuous and discrete features sets including: single pixel gray levels: tag sub-image features: and independent component analysis (ICA) features. We illustrate the methodology for multimodal face retrieval and ultrasound (US) breast image registration.