Intensity-based Image Registration Technique Research Articles

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

TAI-GAN: Temporally and Anatomically Informed GAN for Early-to-Late Frame Conversion in Dynamic Cardiac PET Motion Correction.

The rapid tracer kinetics of rubidium-82 (82Rb) and high variation of cross-frame distribution in dynamic cardiac positron emission tomography (PET) raise significant challenges for inter-frame motion correction, particularly for the early frames where conventional intensity-based image registration techniques are not applicable. Alternatively, a promising approach utilizes generative methods to handle the tracer distribution changes to assist existing registration methods. To improve frame-wise registration and parametric quantification, we propose a Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) to transform the early frames into the late reference frame using an all-to-one mapping. Specifically, a feature-wise linear modulation layer encodes channel-wise parameters generated from temporal tracer kinetics information, and rough cardiac segmentations with local shifts serve as the anatomical information. We validated our proposed method on a clinical 82Rb PET dataset and found that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, motion estimation accuracy and clinical myocardial blood flow (MBF) quantification were improved compared to using the original frames. Our code is published at https://github.com/gxq1998/TAI-GAN.

Movement correction in DCE-MRI through windowed and reconstruction dynamic mode decomposition

Images of the kidneys using dynamic contrast-enhanced magnetic resonance renography (DCE-MRR) contains unwanted complex organ motion due to respiration. This gives rise to motion artefacts that hinder the clinical assessment of kidney function. However, due to the rapid change in contrast agent within the DCE-MR image sequence, commonly used intensity-based image registration techniques are likely to fail. While semi-automated approaches involving human experts are a possible alternative, they pose significant drawbacks including inter-observer variability, and the bottleneck introduced through manual inspection of the multiplicity of images produced during a DCE-MRR study. To address this issue, we present a novel automated, registration-free movement correction approach based on windowed and reconstruction variants of dynamic mode decomposition (WR-DMD). Our proposed method is validated on ten different healthy volunteers’ kidney DCE-MRI data sets. The results, using block-matching-block evaluation on the image sequence produced by WR-DMD, show the elimination of 99% of mean motion magnitude when compared to the original data sets, thereby demonstrating the viability of automatic movement correction using WR-DMD.

Implementation and analysis of template matching for image registration on DevKit-8500D

Image registration is a fundamental task in image processing used to match two different images of same object acquired under different situations and at different times. Implementation of image registration can be intensity value based or feature based. Implementing intensity based image registration technique with reduced time delay on embedded platform DevKit-8500D is presented in this paper. DevKit-8500D is used due to its compatibility for higher programming language such as C/C++ and hardware availability of on-board DSP making it adequate for image and video processing. Template matching on resized images by computation of cross-correlation is used for registration purpose to reduce the computational delay of whole process. A comparison between normal and resized images is also presented to show errors produced in results due to resizing operation and to emphasize on reduced amount of time-delay.

Image-Guided Transapical Aortic Valve Implantation Sensorless Tracking of Stenotic Valve Landmarks in Live Fluoroscopic Images

Aortic valve stenosis is one of the most frequently acquired valvular heart diseases, accounting for almost 70% of valvular cardiac surgery. Transapical transcatheter aortic valve implantation has recently become a suitable minimally invasive technique for high-risk and elderly patients with severe aortic stenosis. In this article, we aim to automatically define a target area of valve implantation, namely, the area between the coronary ostia and the lowest points of two aortic valve cusps. Therefore, we present a new image-based tracking method of these aortic landmarks to assist in the placement of aortic valve prosthesis under live 2D fluoroscopy guidance. We propose a rigid intensity-based image registration technique for tracking valve landmarks in 2D fluoroscopic image sequences, based on a real-time alignment of a contrast image including the initialized manual valve landmarks to each image of sequence. The contrast image is automatically detected to visualize aortic valve features when the aortic root is filled with a contrast agent. Our registration-based tracking method has been retrospectively applied to 10 fluoroscopic image sequences from routine transapical aortic valve implantation procedures. Most of all tested fluoroscopic images showed a successful tracking of valve landmarks, especially for the images without contrast agent injections. A new intraoperative image-based method has been developed for tracking aortic valve landmarks in live 2D fluoroscopic images to assist transapical aortic valve implantations and to increase the overall safety of surgery as well.

HYBRID REGISTRATION OF CORRESPONDING MAMMOGRAM IMAGES FOR AUTOMATIC DETECTION OF BREAST CANCER

Mammogram registration is a critical step in automatic detection of breast cancer. Much research has been devoted to registering mammograms using either feature-matching or similarity measure. However, a few studies have been done on combining these two methods. In this research, a hybrid mammogram registration method for the early detection of breast cancer is developed by combining feature-based and intensity-based image registration techniques. Besides, internal and external features were used simultaneously during the registration to obtain a global spatial transformation. The experimental results indicates that the similarity between the two mammograms increases significantly after a proper registration using the proposed TPS-registration procedures.

HYBRID REGISTRATION OF CORRESPONDING MAMMOGRAM IMAGES FOR AUTOMATIC DETECTION OF BREAST CANCER

Biomedical Engineering: Applications, Basis and CommunicationsVol. 19, No. 06, pp. 359-374 (2007) No AccessHYBRID REGISTRATION OF CORRESPONDING MAMMOGRAM IMAGES FOR AUTOMATIC DETECTION OF BREAST CANCERYih-Chih Chiou, Chern-Sheng Lin, and Cheng-Yu LinYih-Chih ChiouDepartment of Mechanical Engineering, Chung Hua University, Hsinchu, Taiwan, R. O. C. Search for more papers by this author , Chern-Sheng LinDepartment of Automatic Control Engineering, Feng Chia University, Taichung, Taiwan, R. O. C.Corresponding author: Chern Sheng Lin, Professor, Department of Automatic Control Engineering, Feng Chia University, Taichung, Taiwan, R. O. C. Search for more papers by this author , and Cheng-Yu LinDepartment of Mechanical Engineering, Chung Hua University, Hsinchu, Taiwan, R. O. C. Search for more papers by this author https://doi.org/10.4015/S101623720700046XCited by:2 PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail AbstractMammogram registration is a critical step in automatic detection of breast cancer. Much research has been devoted to registering mammograms using either feature-matching or similarity measure. However, a few studies have been done on combining these two methods. In this research, a hybrid mammogram registration method for the early detection of breast cancer is developed by combining feature-based and intensity-based image registration techniques. Besides, internal and external features were used simultaneously during the registration to obtain a global spatial transformation. The experimental results indicates that the similarity between the two mammograms increases significantly after a proper registration using the proposed TPS-registration procedures.Keywords:Mutual informationMammogram registrationThin-plate splinesFeature matching References S. V. Engelandet al., IEEE Trans. Med. Imag. 22(11), 1436 (2003), DOI: 10.1109/TMI.2003.819273. Crossref, ISI, Google ScholarG. Ertas, H. O. Gulcur and K. Gursoy, Alignment of mammogram pairs, Proceedings of the Second Joint EMBS/BMES Conference2 (2002) pp. 1013–1014. Google ScholarF. L. Bookstein, IEEE Trans. pattern Anal. Machine Intell. 11(6), 567 (1989), DOI: 10.1109/34.24792. Crossref, ISI, Google ScholarL. G. Brown, ACM Computing Surveys 24(4), 325 (1992), DOI: 10.1145/146370.146374. Crossref, ISI, Google ScholarH. M. Chen, P. K. Varshney and M. K. Arora, IEEE Trans. Geosci. Remote Sensing 41(11), 2445 (2003), DOI: 10.1109/TGRS.2003.817664. Crossref, ISI, Google ScholarM. L. Gigeret al., Computerized detection and characterization of mass lesions in digital mammography, Proceedings of IEEE International Conference on Systems, Man and Cybernetics2 (1992) pp. 1370–1372. Google ScholarR. L. Harder and R. N. Desmarais, Journal of Aircraft 9(2), 189 (1972), DOI: 10.2514/3.44330. Crossref, Google ScholarP. J. Kostelec, J. B. Weaver and D. M. Healy, Medical Physics 25(9), 1593 (1998), DOI: 10.1118/1.598403. Crossref, ISI, Google ScholarF. Maeset al., IEEE Trans. Med. Imag. 16(2), 187 (1997), DOI: 10.1109/42.563664. Crossref, ISI, Google Scholar K. Marias et al. , Registration and matching of temporal mammograms for detecting abnormalities , Third Conference Medical Image Analysis and Understanding ( 1999 ) . Google ScholarR. Marti, R. Zwiggelaar and C. Rubin, A novel similarity measure to evaluate image correspondence, International Conference on Pattern Recognition (IEEE Computer Society, 2000) pp. 167–170. Google ScholarR. Marti, R. Zwiggelaar and C. Rubin, Tracking mammographic structures over time, Proceedings of the 12th British Machine Vision Conference (2001) pp. 143–152. Google ScholarJ. P. W. Pluim, J. B. A. Maintz and M. A. Viergerver, IEEE Trans. Med. Imag. 22(8), 986 (2003), DOI: 10.1109/TMI.2003.815867. Crossref, ISI, Google ScholarD. Rueckertet al., Non-rigid Registration of Breast MR Images using Mutual Information, Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention (1998) pp. 1144–1152. Google ScholarM. Y. Sallam and K. W. Bowyer, Medical Image Analysis 3(2), 103 (1999), DOI: 10.1016/S1361-8415(99)80001-2. Crossref, Google ScholarR. Sivaramakrishna, Computer Systems Engineering, Medical Physics 25(11), 2249 (1998). ISI, Google ScholarE. A. Stamatakiset al., Detecting abnormalities on mammograms by bilateral comparison, IEEE Colloquium on Digital Mammography12 (1996) pp. 1–4. Google ScholarN. Vujovic and D. Brzakovic, IEEE Trans. Image Processing 6(10), 1388 (1997), DOI: 10.1109/83.624955. Crossref, ISI, Google Scholar Wirth M. A., A Nonrigid Approach to Medical Image Registration: Matching Images of the Breast, PhD Thesis, RMIT University, Melbourne, Australia, 2000 . Google ScholarM. A. Wirth, J. Narhan and D. Gray, Nonrigid Mammogram registration using mutual information, Medical Imaging: Image Processing, Proc SPIE 4684 (2002) pp. 562–573. Google ScholarB. Zitová and J. Flusser, Image and Vision Computing 21(11), 977 (2003). Crossref, ISI, Google Scholar Remember to check out the Most Cited Articles! Notable Biomedical TitlesAuthors from Harvard, Rutgers University, University College London and more! FiguresReferencesRelatedDetailsCited By 2LYMPHATIC INFILTRATION DETECTION IN BREAST CANCER H&E IMAGE PRIOR TO LYMPHADENECTOMYYung-Lung Kuo, Chien-Chuan Ko, and Ming-Ji Lee6 August 2014 | Biomedical Engineering: Applications, Basis and Communications, Vol. 26, No. 04Visual Perception Driven Registration of MammogramsA. Boucher, F. Cloppet, N. Vincent and P. Jouve1 Aug 2010 Recommended Vol. 19, No. 06 Metrics History Accepted 13 November 2007 KeywordsMutual informationMammogram registrationThin-plate splinesFeature matchingPDF download

Brain structural mapping using a novel hybrid implicit/explicit framework based on the level-set method

This paper presents a novel approach to feature-based brain image warping, by using a hybrid implicit/explicit framework, which unifies many prior approaches in a common framework. In the first step, we develop links between image warping and the level-set method, and we formulate the fundamental mathematics required for this hybrid implicit/explicit approach. In the second step, we incorporate the large-deformation models into these formulations, leading to a complete and elegant treatment of anatomical structure matching. In this latest approach, exact matching of anatomy is achieved by comparing the target to the warped source structure under the forward mapping and the source to the warped target structure under the backward mapping. Because anatomy is represented nonparametrically, a path is constructed linking the source to the target structure without prior knowledge of their point correspondence. The final point correspondence is constructed based on the linking path with the minimal energy. Intensity-similarity measures can be naturally incorporated in the same framework as landmark constraints by combining them in the gradient descent body forces. We illustrate the approach with two applications: (1) tensor-based morphometry of the corpus callosum in autistic children; and (2) matching cortical surfaces to measure the profile of cortical anatomic variation. In summary, the new mathematical techniques introduced here contribute fundamentally to the mapping of brain structure and its variation and provide a framework that unites feature and intensity-based image registration techniques.

Intensity-Based Image Registration Using Robust Correlation Coefficients

The ordinary sample correlation coefficient is a popular similarity measure for aligning images from the same or similar modalities. However, this measure can be sensitive to the presence of "outlier" objects that appear in one image but not the other, such as surgical instruments, the patient table, etc., which can lead to biased registrations. This paper describes an intensity-based image registration technique that uses a robust correlation coefficient as a similarity measure. Relative to the ordinary sample correlation coefficient, the proposed similarity measure reduces the influence of outliers. We also compared the performance of the proposed method with the mutual information-based method. The robust correlation-based method should be useful for image registration in radiotherapy (KeV to MeV X-ray images) and image-guided surgery applications. We have investigated the properties of the proposed method by theoretical analysis, computer simulations, a phantom experiment, and with functional magnetic resonance imaging data.

Patient setup error measurement using 3D intensity-based image registration techniques

Purpose Conformal radiotherapy requires accurate patient positioning with reference to the initial three-dimensional (3D) CT image. Patient setup is controlled by comparison with portal images acquired immediately before patient treatment. Several automatic methods have been proposed, generally based on segmentation procedures. However, portal images are of very low contrast, leading to segmentation inaccuracies. In this study, we propose an intensity-based (with no segmentation), fully automatic, 3D method, associating two portal images and a 3D CT scan to estimate patient setup. Methods and materials Images of an anthropomorphic phantom were used. A CT scan of the pelvic area was first acquired, then the phantom was installed in seven positions. The process is a 3D optimization of a similarity measure in the space of rigid transformations. To avoid time-consuming digitally reconstructed radiograph generation at each iteration, we used two-dimensional transformations and two sets of specific and pregenerated digitally reconstructed radiographs. We also propose a technique for computing intensity-based similarity measures between several couples of images. A correlation coefficient, chi-square, mutual information, and correlation ratio were used. Results The best results were obtained with the correlation ratio. The median root mean square error was 2.0 mm for the seven positions tested and was, respectively, 3.6, 4.4, and 5.1 for correlation coefficient, chi-square, and mutual information. Conclusion Full 3D analysis of setup errors is feasible without any segmentation step. It is fast and accurate and could therefore be used before each treatment session. The method presents three main advantages for clinical implementation—it is fully automatic, applicable to all tumor sites, and requires no additional device.