Image Registration Applications Research Articles

Application of Artificial Intelligence to Automate the Reconstruction of Muscle Cross-Sectional Area Obtained by Ultrasound.

Manual reconstruction (MR) of the vastus lateralis (VL) muscle cross sectional area (CSA) from sequential ultrasound (US) images is accessible, reproducible and has concurrent validity with magnetic resonance imaging. However, this technique requires numerous controls and procedures during image acquisition and reconstruction, making it laborious and time-consuming. The aim of this study was to determine the concurrent validity of VL CSA assessments between MR and computer vision-based automatic reconstruction (AR) of CSA from sequential images of the VL obtained by US. The images from each sequence were manually rotated to align the fascia between images and thus visualize the VL CSA. For the AR, an artificial neural network model was utilized to segment areas of interest in the image, such as skin, fascia, deep aponeurosis, and femur. This segmentation was crucial to impose necessary constraints for the main assembly phase. At this stage, an image registration application, combined with differential evolution, was employed to achieve appropriate adjustments between the images. Next, the VL CSA obtained from the MR (n = 488) and AR (n = 488) techniques were used to determine their concurrent validity. Our findings demonstrated a low coefficient of variation (CV) (1.51%) for AR compared to MR. The Bland-Altman plot showed low bias and close limits of agreement (+1.18 cm2, -1.19 cm2), containing more than 95% of the data points. The AR technique is valid compared to MR when measuring VL CSA in a heterogeneous sample.

Deep-learning-based deformable image registration of head CT and MRI scans

This work is motivated by the lack of publications on the direct application of multimodal image registration with deep-learning techniques for the enhancement of treatment planning in particle therapy. An unsupervised workflow, which seeks to improve image alignment, was developed and evaluated for computed tomography and magnetic resonance imaging scans of the head. The scans of 39 paediatric patients with brain tumours were available. The focus of the two-step workflow, including preprocessing of the scans for normalisation, is deformable image registration (DIR) with a deep neural network, which generates deformation vector fields (DVFs). To obtain a suitable configuration of the network, parameter tuning is performed by varying its parameters, e.g., layer size, regularisation (λ) of the DVF and learning rate (α). Image similarity was determined with the Dice similarity coefficient, mDSC, using segmented images and the mutual-information metric, mMI. The performance of the deep-learning models was assessed with the inverse consistency, mIC, and the Jacobian determinant, mJD. Inverse consistency is obtained for mIC = 0 mm, while the determinant of a deformed image is expected to be unity. The deep-learning models passed both performance checks, indicated by the mean values m̄IC=(0.57±1.00)mm and m̄JD=(1.00±0.07). Models with λ ≥ 1 yielded higher mDSC values than models with lower λ values. A small-architecture model with α = 10–4 was found to be most suitable for DIR, as improvement in image similarity of up to 12% was obtained in terms of mMI. The direct application of deep-learning models produced registered images improving image alignment between scans of different modalities.

Magnetic resonance imaging datasets with anatomical fiducials for quality control and registration

Tools available for reproducible, quantitative assessment of brain correspondence have been limited. We previously validated the anatomical fiducial (AFID) placement protocol for point-based assessment of image registration with millimetric (mm) accuracy. In this data descriptor, we release curated AFID placements for some of the most commonly used structural magnetic resonance imaging datasets and templates. The release of our accurate placements allows for rapid quality control of image registration, teaching neuroanatomy, and clinical applications such as disease diagnosis and surgical targeting. We release placements on individual subjects from four datasets (N = 132 subjects for a total of 15,232 fiducials) and 14 brain templates (4,288 fiducials), totalling more than 300 human rater hours of annotation. We also validate human rater accuracy of released placements to be within 1 – 2 mm (using more than 45,000 Euclidean distances), consistent with prior studies. Our data is compliant with the Brain Imaging Data Structure allowing for facile incorporation into neuroimaging analysis pipelines.

Automatic registration of dental CT and 3D scanned model using deep split jaw and surface curvature

Background and objectives: In the medical field, various image registration applications have been studied. In dentistry, the registration of computed tomography (CT) volume data and 3D optically scanned models is essential for various clinical applications, including orthognathic surgery, implant surgical planning, and augmented reality. Our purpose was to present a fully automatic registration method of dental CT data and 3D scanned models.Methods: We use a 2D convolutional neural network to regress a curve splitting the maxilla (i.e., upper jaw) and mandible (i.e., lower jaw) and the points specifying the front and back ends of the crown from the CT data. Using this regressed information, we extract the point cloud and vertices corresponding to the tooth crown from the CT and scanned data, respectively. We introduce a novel metric, called curvature variance of neighbor (CVN), to discriminate between highly fluctuating and smoothly varying regions of the tooth crown. The registration based on CVN enables more accurate fine registration while reducing the effects of metal artifacts. Moreover, the proposed method does not require any preprocessing such as extracting the iso-surface for the tooth crown from the CT data, thereby significantly reducing the computation time.Results: We evaluated the proposed method with the comparison to several promising registration techniques. Our experimental results using three datasets demonstrated that the proposed method exhibited higher registration accuracy (i.e., 2.85, 1.92, and 7.73 times smaller distance errors for individual datasets) and smaller computation time (i.e., 4.12 times faster registration) than one of the state-of-the-art methods. Moreover, the proposed method worked considerably well for partially scanned data, whereas other methods suffered from the unbalancing of information between the CT and scanned data.Conclusions: The proposed method was able to perform fully automatic and highly accurate registration of dental CT data and 3D scanned models, even with severe metal artifacts. In addition, it could achieve fast registration because it did not require any preprocessing for iso-surface reconstruction from the CT data.

Fine-grained, nonlinear registration of live cell movies reveals spatiotemporal organization of diffuse molecular processes.

We present an application of nonlinear image registration to align in microscopy time lapse sequences for every frame the cell outline and interior with the outline and interior of the same cell in a reference frame. The registration relies on a subcellular fiducial marker, a cell motion mask, and a topological regularization that enforces diffeomorphism on the registration without significant loss of granularity. This allows spatiotemporal analysis of extremely noisy and diffuse molecular processes across the entire cell. We validate the registration method for different fiducial markers by measuring the intensity differences between predicted and original time lapse sequences of Actin cytoskeleton images and by uncovering zones of spatially organized GEF- and GTPase signaling dynamics visualized by FRET-based activity biosensors in MDA-MB-231 cells. We then demonstrate applications of the registration method in conjunction with stochastic time-series analysis. We describe distinct zones of locally coherent dynamics of the cytoplasmic protein Profilin in U2OS cells. Further analysis of the Profilin dynamics revealed strong relationships with Actin cytoskeleton reorganization during cell symmetry-breaking and polarization. This study thus provides a framework for extracting information to explore functional interactions between cell morphodynamics, protein distributions, and signaling in cells undergoing continuous shape changes. Matlab code implementing the proposed registration method is available at https://github.com/DanuserLab/Mask-Regularized-Diffeomorphic-Cell-Registration.

Improved speeded-up robust features registration method based on particle swarm and gray wolf algorithm for coal mine images

Image registration is an electronic imaging technology that matches two or more images acquired at different times or under different conditions. Here, we highlight the application of image registration in monitoring robots (e.g., autonomous vehicles) of coal mines. In image registration processing, the scale-invariant feature transform algorithm and its improved versions achieve poor extraction effects for feature points on smooth-edge curves. In comparison, the speeded-up robust features (SURF) algorithm achieves more stable registration results in various images. To improve the accuracy of the SURF algorithm, it is optimized using the particle swarm optimization (PSO) algorithm and several improved PSOs, but it achieves unstable accuracy. Therefore, we optimize the improved gray wolf algorithm (IGWO) using advanced mathematical models in several computing steps. We add the memory function of both experienced individual and group optimal locations in PSO to update the particle positions. The evaluations performed on different datasets show that the accuracy and range of registration are substantially improved by IGWO and IGWO-PSO; however, the running time is extended. Thus, our improved method is competitive with other state-of-the-art image registration methods.

Adaptive radiation therapy strategies in the treatment of prostate cancer patients using hypofractionated VMAT.

PurposeTo perform a comprehensive evaluation of eight adaptive radiation therapy strategies in the treatment of prostate cancer patients who underwent hypofractionated volumetric modulated arc therapy (VMAT) treatment.Material and methodsThe retrospective study included 20 prostate cancer patients treated with 40 Gy total dose over five fractions (8 Gy/fraction) using VMAT. Daily cone beam computed tomography images were acquired before the delivery of every fraction and then, with the application of deformable image registration used for the estimation of daily dose, contouring and plan re‐optimization. Dosimetric benefits of the various ART strategies were quantified by the comparison of dose and dose‐volume metrics derived from treatment planning objectives for original treatment plan and adapted plans with the consideration of target volumes (PTV and CTV) as well as critical structures (bladder, rectum, left, and right femoral heads).ResultsPercentage difference (ΔD) between planning objectives and delivered dose in the D99% > 4000cGy (CTV) metric was −3.9% for the non‐ART plan and 2.1% to 4.1% for ART plans. For D99% > 3800cGy and Dmax < 4280cGy (PTV), ΔD was −11.2% and −6.5% for the non‐ART plan as well as −3.9% to −1.6% and −0.2% to 1.8% for ART plans, respectively. For D15% < 3200 cGy and D20% < 2800 cGy (bladder), ΔD was −62.4% and −68.8% for the non‐ART plan as well as −60.0% to −57.4% and −67.0% to −64.0% for ART plans. For D15% < 3200 cGy and D20% < 2800 cGy (rectum), ΔD was −11.4% and −8.15% for non‐ART plan as well as −14.9% to −9.0% and −11.8% to −5.1% for ART plans.ConclusionsDaily on‐line adaptation approaches were the most advantageous, although strategies adapting every other fraction were also impactful while reducing relative workload as well. Offline treatment adaptations were shown to be less beneficial due to increased dose delivered to bladder and rectum compared toother ART strategies.

Optimizing a medical image registration algorithm based on profiling data for real-time performance

Image registration is a commonly task in medical image analysis. Therefore, a significant number of algorithms have been developed to perform rigid and non-rigid image registration. Particularly, the free-form deformation algorithm is frequently used to carry out non-rigid registration task; however, it is a computationally very intensive algorithm. In this work, we describe an approach based on profiling data to identify potential parts of this algorithm for which parallel implementations can be developed. The proposed approach assesses the efficient of the algorithm by applying performance analysis techniques commonly available in traditional computer operating systems. Hence, this article provides guidelines to support researchers working on medical image processing and analysis to achieve real-time non-rigid image registration applications using common computing systems. According to our experimental findings, significant speedups can be accomplished by parallelizing sequential snippets, i.e., code regions that are executed more than once. For the selected costly functions previously identified in the studied free-form deformation algorithm, the developed parallelization decreased the runtime by up to seven times relatively to the related single thread based implementation. The implementations were developed based on the Open Multi-Processing application programming interface. In conclusion, this study confirms that based on the call graph visualization and detected performance bottlenecks, one can easily find and evaluate snippets which are potential optimization targets in addition to throughput in memory accesses.

An adaptive moving grid finite difference method

The grid generation is very crucial for the accuracy of the numerical solution of PDEs, especially for problems with very rapid variations or sharp layers, such as shock waves, wing leading and trailing edges, regions of separation, and boundary layers. The adaptive grid generation is an iterative approach to accommodate these complex structures. In this paper, we introduce a deformation based adaptive grid generation method, in which a differentiable and invertible transformation from computational domain to physical domain is constructed such that the cell volume (Jacobian determinant) of the new grid is equal to a prescribed monitor function. A vector field is obtained by solving the div-curl system and can be used to move the grids to the desired locations. By computing the inverse of Jacobian, any deformed grids can also be transformed back to the uniform grid. Several numerical results in two dimensions are presented. Some applications in image registration are discussed.

Deep learning in structural and functional lung image analysis.

The recent resurgence of deep learning (DL) has dramatically influenced the medical imaging field. Medical image analysis applications have been at the forefront of DL research efforts applied to multiple diseases and organs, including those of the lungs. The aims of this review are twofold: (i) to briefly overview DL theory as it relates to lung image analysis; (ii) to systematically review the DL research literature relating to the lung image analysis applications of segmentation, reconstruction, registration and synthesis. The review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. 479 studies were initially identified from the literature search with 82 studies meeting the eligibility criteria. Segmentation was the most common lung image analysis DL application (65.9% of papers reviewed). DL has shown impressive results when applied to segmentation of the whole lung and other pulmonary structures. DL has also shown great potential for applications in image registration, reconstruction and synthesis. However, the majority of published studies have been limited to structural lung imaging with only 12.9% of reviewed studies employing functional lung imaging modalities, thus highlighting significant opportunities for further research in this field. Although the field of DL in lung image analysis is rapidly expanding, concerns over inconsistent validation and evaluation strategies, intersite generalisability, transparency of methodological detail and interpretability need to be addressed before widespread adoption in clinical lung imaging workflow.

Application of Image Registration to Analyze the Clavicular Rotation of Normal Upper Limb Motion in the Sagittal Plane.

ObjectiveTo use image registration techniques to study the clavicular rotation of the shoulders in the sagittal plane.MethodsFrom 28 April 2019 to 20 May 2019, 13 healthy adults (7 males and 6 females) with no history of shoulder trauma surgery or chronic pain were recruited. Patients’ ages ranged from 22 to 42 years, with a mean age of 26.5 years. Three‐dimensional composite images of the sternum–clavicle–humerus were taken using CT images of upper limb movement in the sagittal plane in the 13 healthy adults. Four different postures were registered: (i) anatomical supine position; (ii) elbow joints lifted anteriorly in the supine position; (iii) posterosuperior hyperextension of the elbow joints in the prone position; and (iv) posteroinferior hyperextension of the elbow joints in the prone position. Image data from the humerus and clavicle in three of the postures were processed to calculate Euler angles for movements in the sagittal plane. SPSS 19 was used to perform statistical analyses.ResultsThere was no significant difference in the angles of change in the clavicle and humerus between the dominant and non‐dominant sides under different movement patterns. For upper limb movements in the sagittal plane, the clavicle displayed different Euler angles in different postures. The rotation angle from the anatomical to the horizontal position was the smallest angle, with an average value of 7.1°, whereas the rotation angle from horizontal to posterosuperior hyperextension was the largest, with an average value of 37.2°. When the upper limb moved from anterior protraction to a posterosuperior extension, the intrinsic rotation angle of the clavicle reached its maximum, with an average value of 27.9°; when moved from the anatomical to the horizontal position, 9.1% of the sagittal rotation was executed by the clavicle. During rotation from the horizontal position to posterosuperior hyperextension and from the anatomical to posterior extension, the clavicle showed relatively higher weights at 29.5% and 37.0%, respectively.ConclusionOur results showed that dominance was not a consideration when studying clavicular rotation. Image registration is an effective method that can be used to study upper limb scapular movements. Through comparing and analyzing the data, two postures had relatively large changes in the rotation angle. This can help improve indicators of clavicular rotational function during physical examinations and postoperative functional evaluations.

An analytical review of texture feature extraction approaches

Image registration has been an essential task in computer vision and image processing. There are many applications of image registration, for instance, in medical systems. Image registration consists of four main steps, including Features Extraction, Feature Matching, Transform Model Estimation, Resampling and Transformation. The Feature Extraction step makes the image registration process more accurate. Despite a large number of survey articles on texture feature extraction approaches, a comprehensive classification of approaches is still required, which also identifies the strengths and weaknesses of each approach. Therefore, the novelty of this paper, our analytical framework includes three major components: a complete classification of texture feature extraction approaches, using crucial evaluation criteria to present an analytical and qualitative comparison between each approach, which simplifies the accurate selection of the proposed approaches for the intended application. Our framework can also lead to the development of texture feature extraction approaches in future research of scientists.

Image alignment for synchrotron radiation based X-ray nano-CT

Synchrotron radiation-based X-ray nano-imaging is a powerful tool for non-destructively studying the internal nano-scale structure of matter. Here in this paper, we review the state-of-the-art image alignment technology in the field of nano-resolution imaging, and classify and analyze the technology according to the research stage. First, through the publications of image alignment algorithm, the development direction of future research is analyzed. Then, the most effective image alignment application in the field of nano imaging based on classic image alignment algorithms is summarized. The paper also presents the feature detection operators that are useful for nano-scale image registration selected from recent feature detection research, which has important guiding significance for the specific application and optimization of nano-imaging image registration. Finally, the state-of-the-art image registration method based on deep learning is introduced, the applicability and potential of deep learning in nano-imaging registration technology are discussed, and future research directions and challenges are prospected based on different neural network characteristics.

SymReg-GAN: Symmetric Image Registration With Generative Adversarial Networks.

Symmetric image registration estimates bi-directional spatial transformations between images while enforcing an inverse-consistency. Its capability of eliminating bias introduced inevitably by generic single-directional image registration allows more precise analysis in different interdisciplinary applications of image registration, e.g., computational anatomy and shape analysis. However, most existing symmetric registration techniques especially for multimodal images are limited by low speed from the commonly-used iterative optimization, hardship in exploring inter-modality relations or high labor cost for labeling data. We propose SymReg-GAN to shatter these limits, which is a novel generative adversarial networks (GAN) based approach to symmetric image registration. We formulate symmetric registration of unimodal/multimodal images as a conditional GAN and train it with a semi-supervised strategy. The registration symmetry is realized by introducing a loss for encouraging that the cycle composed of the geometric transformation from one image to another and its reverse should bring an image back. The semi-supervised learning enables both the precious labeled data and large amounts of unlabeled data to be fully exploited. Experimental results from six public brain magnetic resonance imaging (MRI) datasets and 1 our own computed tomography (CT) and MRI dataset demonstrate the superiority of SymReg-GAN to several existing state-of-the-art methods.

Research on Medical Image Registration Based on Graphic Neural Network Reinforcement Learning

On the basis of traditional medical image registration methods, aiming at the shortcomings of rigid body images, sequential rigid body images and non-rigid body images registration methods, this paper proposes a new method of medical image registration application based on graphic neural network reinforcement learning. Reinforcement learning adjusts the strategy from the actual system learning experience, and it is a process of gradually approaching the optimal strategy, mapping the image from high-dimensional gray space to low-dimensional feature category space; Then, the elastic registration of mutual information images is carried out in the feature class space. The advantage of this method is that it does not need extra supervision information, which effectively saves manpower and material resources. So as to make an accurate diagnosis or make a proper treatment plan.

Deforming to Best Practice: Key considerations for deformable image registration in radiotherapy.

Image registration is a process that underlies many new techniques in radiation oncology – from multimodal imaging and contour propagation in treatment planning to dose accumulation throughout treatment. Deformable image registration (DIR) is a subset of image registration subject to high levels of complexity in process and validation. A need for local guidance to assist in high‐quality utilisation and best practice was identified within the Australian community, leading to collaborative activity and workshops. This report communicates the current limitations and best practice advice from early adopters to help guide those implementing DIR in the clinic at this early stage. They are based on the state of image registration applications in radiotherapy in Australia and New Zealand (ANZ), and consensus discussions made at the ‘Deforming to Best Practice’ workshops in 2018. The current status of clinical application use cases is presented, including multimodal imaging, automatic segmentation, adaptive radiotherapy, retreatment, dose accumulation and response assessment, along with uptake, accuracy and limitations. Key areas of concern and preliminary suggestions for commissioning, quality assurance, education and training, and the use of automation are also reported. Many questions remain, and the radiotherapy community will benefit from continued research in this area. However, DIR is available to clinics and this report is intended to aid departments using or about to use DIR tools now.

Flexible numerical simulation framework for dynamic PET-MR data

This paper presents a simulation framework for dynamic PET-MR. The main focus of this framework is to provide motion-resolved MR and PET data and ground truth motion information. This can be used in the optimisation and quantitative evaluation of image registration and in assessing the error propagation due to inaccuracies in motion estimation in complex motion-compensated reconstruction algorithms. Contrast and tracer kinetics can also be simulated and are available as ground truth information. To closely emulate medical examination, input and output of the simulation are files in standardised open-source raw data formats. This enables the use of existing raw data as a template input and ensures seamless integration of the output into existing reconstruction pipelines.The proposed framework was validated in PET-MR and image registration applications. It was used to simulate a FDG-PET-MR scan with cardiac and respiratory motion. Ground truth motion information could be utilised to optimise parameters for PET and synergistic PET-MR image registration. In addition, a free-breathing dynamic contrast enhancement (DCE) abdominal scan of a patient with hepatic lesions was simulated. In order to correct for breathing motion, a motion-corrected image reconstruction scheme was used and a Toft’s model was fit to the DCE data to obtain quantitative DCE-MRI parameters. Utilising the ground truth motion information, the dependency of quantitative DCE-MR images on the accuracy of the motion estimation was evaluated. We demonstrated that respiratory motion had to be available with an average accuracy of at least the spatial resolution of the DCE-MR images in order to ensure an improvement in lesions visualisation and quantification compared to no motion correction.The proposed framework provides a valuable tool with a wide range of scientific PET and MR applications and will be available as part of the open-source project Synergistic Image Reconstruction Framework (SIRF).

Stochastic Distance Transform: Theory, Algorithms and Applications

Distance transforms (DTs) are standard tools in image analysis, with applications in image registration and segmentation. The DT is based on extremal (minimal) distance values and is therefore highly sensitive to noise. We present a stochastic distance transform (SDT) based on discrete random sets, in which a model of element-wise probability is utilized and the SDT is computed as the first moment of the distance distribution to the random set. We present two methods for computing the SDT and analyze them w.r.t. accuracy and complexity. Further, we propose a method, utilizing kernel density estimation, for estimating probability functions and associated random sets to use with the SDT. We evaluate the accuracy of the SDT and the proposed framework on images of thin line structures and disks corrupted by salt and pepper noise and observe excellent performance. We also insert the SDT into a segmentation framework and apply it to overlapping objects, where it provides substantially improved performance over previous methods. Finally, we evaluate the SDT and observe very good performance, on simulated images from localization microscopy, a state-of-the-art super-resolution microscopy technique which yields highly spatially localized but noisy point-clouds.

SIFT Image Feature Extraction for an Efficient Image Registration

This paper presents an image registration algorithm based on SIFT (Scale Invariant Feature Transform).The obtained descriptors and key points by the SIFT confirms that, the algorithm is very robust to scaling, noise, translation and rotation. At the beginning, the key points are extracted from the image. Later to Match the obtained points, dot products between the unit vectors are calculated. Finally, transformation matrix is obtained by applying RANSAC algorithm. Experimental results shows that the algorithm extracts the better key points, which can be used for used for image registration applications.

Deep learning in medical image registration: a survey

The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring, and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.