Intensity-based Image Registration Research Articles

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

A Majorization-Minimization Algorithm for Neuroimage Registration.

Intensity-based image registration is critical for neuroimaging tasks, such as 3D reconstruction, times-series alignment, and common coordinate mapping. The gradient-based optimization methods commonly used to solve this problem require a careful selection of step-length. This limitation imposes substantial time and computational costs. Here we propose a gradient-independent rigid-motion registration algorithm based on the majorization-minimization (MM) principle. Each iteration of our intensity-based MM algorithm reduces to a simple point-set rigid registration problem with a closed form solution that avoids the step-length issue altogether. The details of the algorithm are presented, and an error bound for its more practical truncated form is derived. The performance of the MM algorithm is shown to be more effective than gradient descent on simulated images and Nissl stained coronal slices of mouse brain. We also compare and contrast the similarities and differences between the MM algorithm and another gradient-free registration algorithm called the block-matching method. Finally, extensions of this algorithm to more complex problems are discussed.

Optimizing 3DCT image registration for interfractional changes in carbon-ion prostate radiotherapy

To perform setup procedures including both positional and dosimetric information, we developed a CT–CT rigid image registration algorithm utilizing water equivalent pathlength (WEPL)-based image registration and compared the resulting dose distribution with those of two other algorithms, intensity-based image registration and target-based image registration, in prostate cancer radiotherapy using the carbon-ion pencil beam scanning technique. We used the data of the carbon ion therapy planning CT and the four-weekly treatment CTs of 19 prostate cancer cases. Three CT–CT registration algorithms were used to register the treatment CTs to the planning CT. Intensity-based image registration uses CT voxel intensity information. Target-based image registration uses target position on the treatment CTs to register it to that on the planning CT. WEPL-based image registration registers the treatment CTs to the planning CT using WEPL values. Initial dose distributions were calculated using the planning CT with the lateral beam angles. The treatment plan parameters were optimized to administer the prescribed dose to the PTV on the planning CT. Weekly dose distributions using the three different algorithms were calculated by applying the treatment plan parameters to the weekly CT data. Dosimetry, including the dose received by 95% of the clinical target volume (CTV-D95), rectal volumes receiving > 20 Gy (RBE) (V20), > 30 Gy (RBE) (V30), and > 40 Gy (RBE) (V40), were calculated. Statistical significance was assessed using the Wilcoxon signed-rank test. Interfractional CTV displacement over all patients was 6.0 ± 2.7 mm (19.3 mm maximum standard amount). WEPL differences between the planning CT and the treatment CT were 1.2 ± 0.6 mm-H2O (< 3.9 mm-H2O), 1.7 ± 0.9 mm-H2O (< 5.7 mm-H2O) and 1.5 ± 0.7 mm-H2O (< 3.6 mm-H2O maxima) with the intensity-based image registration, target-based image registration, and WEPL-based image registration, respectively. For CTV coverage, the D95 values on the planning CT were > 95% of the prescribed dose in all cases. The mean CTV-D95 values were 95.8 ± 11.5% and 98.8 ± 1.7% with the intensity-based image registration and target-based image registration, respectively. The WEPL-based image registration was CTV-D95 to 99.0 ± 0.4% and rectal Dmax to 51.9 ± 1.9 Gy (RBE) compared to 49.4 ± 9.1 Gy (RBE) with intensity-based image registration and 52.2 ± 1.8 Gy (RBE) with target-based image registration. The WEPL-based image registration algorithm improved the target coverage from the other algorithms and reduced rectal dose from the target-based image registration, even though the magnitude of the interfractional variation was increased.

TAI-GAN: Temporally and Anatomically Informed GAN for Early-to-Late Frame Conversion in Dynamic Cardiac PET Motion Correction.

The rapid tracer kinetics of rubidium-82 (82Rb) and high variation of cross-frame distribution in dynamic cardiac positron emission tomography (PET) raise significant challenges for inter-frame motion correction, particularly for the early frames where conventional intensity-based image registration techniques are not applicable. Alternatively, a promising approach utilizes generative methods to handle the tracer distribution changes to assist existing registration methods. To improve frame-wise registration and parametric quantification, we propose a Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) to transform the early frames into the late reference frame using an all-to-one mapping. Specifically, a feature-wise linear modulation layer encodes channel-wise parameters generated from temporal tracer kinetics information, and rough cardiac segmentations with local shifts serve as the anatomical information. We validated our proposed method on a clinical 82Rb PET dataset and found that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, motion estimation accuracy and clinical myocardial blood flow (MBF) quantification were improved compared to using the original frames. Our code is published at https://github.com/gxq1998/TAI-GAN.

Three-Dimensional Analysis of Bone Volume Change at Donor Sites in Mandibular Body Bone Block Grafts by a Computer-Assisted Automatic Registration Method: A Retrospective Study

This study aimed to evaluate the bone volume change at donor sites in patients who received mandibular body bone block grafts using intensity-based automatic image registration. A retrospective study was conducted with 32 patients who received mandibular bone block grafts between 2017 and 2019 at the Pusan National University Dental Hospital. Cone-beam computed tomography (CBCT) images were obtained before surgery (T0), 1 day after surgery (T1), and 4 months after surgery (T2). Scattered artefacts were removed by manual segmentation. The T0 image was used as the reference image for registration of T1 and T2 images using intensity-based registration. A total of 32 donor sites were analyzed three-dimensionally. The volume and pixel value of the bones were measured and analyzed. The mean regenerated bone volume rate on follow-up images (T2) was 34.87% ± 17.11%. However, no statistically significant differences of regenerated bone volume were noted among the four areas of the donor site (upper anterior, upper posterior, lower anterior, and lower posterior). The mean pixel value rate of the follow-up images (T2) was 78.99% ± 16.9% compared with that of T1, which was statistically significant (p &lt; 0.05). Intensity-based registration with histogram matching showed that newly generated bone is generally qualitatively and quantitatively poorer than the original bone, thus revealing the feasibility of pixel value to evaluate bone quality in CBCT images. Considering the bone mass recovered in this study, 4 months may not be sufficient for a second harvesting, and a longer period of follow-up is required.

Accuracy and consistency of intensity-based deformable image registration in 4DCT for tumor motion estimation in liver radiotherapy planning.

We investigate the accuracy of intensity-based deformable image registration (DIR) for tumor localization in liver stereotactic body radiotherapy (SBRT). We included 4DCT scans to capture the breathing motion of eight patients receiving SBRT for liver metastases within a retrospective clinical study. Each patient had three fiducial markers implanted. The liver and the tumor were delineated in the mid-ventilation phase, and their positions in the other phases were estimated with deformable image registration. We tested referenced and sequential registrations strategies. The fiducial markers were the gold standard to evaluate registration accuracy. The registration errors related to measured versus estimated fiducial markers showed a mean value less than 1.6mm. The positions of some fiducial markers appeared not stable on the 4DCT throughout the respiratory phases. Markers’ center of mass tends to be a more reliable measurement. Distance errors of tumor location based on registration versus markers center of mass were less than 2mm. There were no statistically significant differences between the reference and the sequential registration, i.e., consistency and errors were comparable to resolution errors. We demonstrated that intensity-based DIR is accurate up to resolution level for locating the tumor in the liver during breathing motion.

Realistic C-arm to pCT registration for vertebral localization in spine surgery

Spine surgeries are vulnerable to wrong-level surgeries and postoperative complications because of their complex structure. Unavailability of the 3D intraoperative imaging device, low-contrast intraoperative X-ray images, variable clinical and patient conditions, manual analyses, lack of skilled technicians, and human errors increase the chances of wrong-site or wrong-level surgeries. State of the art work refers 3D-2D image registration systems and other medical image processing techniques to address the complications associated with spine surgeries. Intensity-based 3D-2D image registration systems had been widely practiced across various clinical applications. However, these frameworks are limited to specific clinical conditions such as anatomy, dimension of image correspondence, and imaging modalities. Moreover, there are certain prerequisites for these frameworks to function in clinical application, such as dataset requirement, speed of computation, requirement of high-end system configuration, limited capture range, and multiple local maxima. A simple and effective registration framework was designed with a study objective of vertebral level identification and its pose estimation from intraoperative fluoroscopic images by combining intensity-based and iterative control point (ICP)–based 3D-2D registration. A hierarchical multi-stage registration framework was designed that comprises coarse and finer registration. The coarse registration was performed in two stages, i.e., intensity similarity-based spatial localization and source-to-detector localization based on the intervertebral distance correspondence between vertebral centroids in projected and intraoperative X-ray images. Finally, to speed up target localization in the intraoperative application, based on 3D-2D vertebral centroid correspondence, a rigid ICP-based finer registration was performed. The mean projection distance error (mPDE) measurement and visual similarity between projection image at finer registration point and intraoperative X-ray image and surgeons’ feedback were held accountable for the quality assurance of the designed registration framework. The average mPDE after peak signal to noise ratio (PSNR)–based coarse registration was 20.41mm. After the coarse registration in spatial region and source to detector direction, the average mPDE reduced to 12.18mm. On finer ICP-based registration, the mean mPDE was finally reduced to 0.36 mm. The approximate mean time required for the coarse registration, finer registration, and DRR image generation at the final registration point were 10 s, 15 s, and 1.5 min, respectively. The designed registration framework can act as a supporting tool for vertebral level localization and its pose estimation in an intraoperative environment. The framework was designed with the future perspective of intraoperative target localization and its pose estimation irrespective of the target anatomy.Graphical abstract

Image registration method using representative feature detection and iterative coherent spatial mapping for infrared medical images with flat regions

In the registration of medical images, nonrigid registration targets, images with large displacement caused by different postures of the human body, and frequent variations in image intensity due to physiological phenomena are substantial problems that make medical images less suitable for intensity-based image registration modes. These problems also greatly increase the difficulty and complexity of feature detection and matching for feature-based image registration modes. This research introduces an automatic image registration algorithm for infrared medical images that offers the following benefits: effective detection of feature points in flat regions (cold patterns) that appear due to changes in the human body’s thermal patterns, improved mismatch removal through coherent spatial mapping for improved feature point matching, and large-displacement optical flow for optimal transformation. This method was compared with various classical gold standard image registration methods to evaluate its performance. The models were compared for the three key steps of the registration process—feature detection, feature point matching, and image transformation—and the results are presented visually and quantitatively. The results demonstrate that the proposed method outperforms existing methods in all tasks, including in terms of the features detected, uniformity of feature points, matching accuracy, and control point sparsity, and achieves optimal image transformation. The performance of the proposed method with four common image types was also evaluated, and the results verify that the proposed method has a high degree of stability and can effectively register medical images under a variety of conditions.

Application of an automated dose accumulation workflow in high-risk prostate cancer - validation and dose-volume analysis between planned and delivered dose.

Inter-fraction organ variations cause deviations between planned and delivered doses in patients receiving radiotherapy for prostate cancer. This study compared planned (DP) vs accumulated doses (DA) obtained from daily cone-beam computed tomography (CBCT) scans in high-risk- prostate cancer with pelvic lymph nodes irradiation. An intensity-based deformable image registration algorithm used to estimate contours for DA was validated using geometrical agreement between radiation oncologist's and deformable image registration algorithm propagated contours. Spearman rank correlations (rs) between geometric measures and changes in organ volumes were evaluated for 20 cases. Dose-volume (DV) differences between DA and DP were compared (Wilcoxon rank test, p < 0.05). A novel region-of-interest (ROI) method was developed and mean doses were analyzed. Geometrical measures for the prostate and organ-at-risk contours were within clinically acceptable criteria. Inter-group mean (± SD) CBCT volumes for the rectum were larger compared to planning CT (pCT) (51.1 ± 11.3 cm3vs 46.6 ± 16.1 cm3), and were moderately correlated with variations in pCT volumes, rs = 0.663, p < 0.01. Mean rectum DV for DA was higher at V30-40 Gy and lower at V70-75 Gy, p < 0.05. Mean bladder CBCT volumes were smaller compared to pCT (198.8 ± 55 cm3vs 211.5 ± 89.1 cm3), and was moderately correlated with pCT volumes, rs = 0.789, p < 0.01. Bladder DA was higher at V30-65 Gy and lower at V70-75 Gy (p < 0.05). For the ROI method, rectum and bladder DA were lower at 5 to 10 mm (p < 0.01) as compared to DP, whilst bladder DA was higher than DP at 20 to 50 mm (p < 0.01). Generated DA demonstrated significant differences in organ-at-risk doses as compared to DP. A well-constructed workflow incorporating a ROI DV-extraction method has been validated in terms of efficiency and accuracy designed for seamless integration in the clinic to guide future plan adaptation.

Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment

In the interest of developing an intelligent manufacturing environment with an agile, efficient, and optimally utilized transportation system, mobile robots need to achieve a certain level of autonomy as they play an important role in carrying out transportation tasks. Bearing this in mind, in the paper we propose a novel stereo visual servoing method for nonholonomic mobile robot control based on semantic segmentation. Semantic segmentation provides a rich body of information required for an adequate decision-making process in a clustered, dynamic, and ever-changing manufacturing environment. The innovative idea behind the new visual servoing system is to utilize semantic information of the scene for visual servoing, as well as for other mobile robot tasks, such as obstacle avoidance, scene understanding, and simultaneous localization and mapping. Semantic segmentation is carried out by exploiting fully convolutional neural networks. The new visual servoing algorithm utilizes an intensity-based image registration procedure, which results in the image transformation matrix. The transformation matrix encompasses the relations of images taken at the current and desired pose, and that information is directly used for visual servoing. The developed algorithm is deployed on our own developed wheeled differential drive mobile robot RAICO (Robot with Artificial Intelligence based COgnition). The experimental evaluation is carried out in the 3D simulation environment and in the laboratory model of the real manufacturing environment. The experimental results show that the accuracy of the proposed approach is improved when compared to the state-of-the-art approaches while being robust to the partial occlusions of the scene and illumination changes.

Lung Perfused Blood Volume Calculated From Dynamic Contrast Enhanced DECT for Radiation Treatment Planning of Lung Cancer

<h3>Purpose/Objective(s)</h3> In radiation therapy planning for lung cancer, it is desirable to minimize irradiation of functional lung tissue which may be mapped by the spatial distribution of perfused blood volume (PBV). In this exploratory study, we investigate the feasibility of generating PBV distribution based on equilibrium contrast enhanced dual-energy CT (DECT) routinely acquired at the time of CT simulation for lung cancer patients. <h3>Materials/Methods</h3> DECT data with iodine-based intravenous (IV) contrast enhancement along with 4DCT acquired using a Dual Source DECT scanner (Drive, Siemens Healthcare, Germany) for 5 lung cancer patients during routine CT simulation were analyzed. The delay of the IV contrast injection was calculated for each patient via a circular scan monitoring a region of interest (ROI) in the pulmonary trunk for a CT number increase of 150 HU. This increase triggers a further 7 second delay after which the scanner rapidly performs two simultaneous acquisitions, the first at a tubevoltage of 80 kV and the second at a tube voltage of 140 kV with Sn filtration. For each patient the PBV was automatically calculated via post-processing at the console. The obtained PBV maps were compared to ventilation maps calculated in a programming environment using a HU-based method from the 4DCT, which was sorted into 10 bins based on respiratory phase. The image set of the end of the inhale phase was warped to the image set of the end of the exhale phase (target images) in MIM software using an intensity-based deformable image registration algorithm. The HU values of the warped and target images in each associated voxel pair inside lung contours were used to calculate local lung volume changes. <h3>Results</h3> The obtained lung PBV maps demonstrated heterogeneity in lung function for all patients. The average PBV in arbitrary units was 52.3 ± 16.7. The PBV distribution for 4 patients had a single peak, while therewas a bifurcation for one case. For one of the patients, the image quality was degraded by high Z materials in the patient's shoulders. For one of the patients there was a correlation between the PBV map and the ventilation map. The tidal volumes calculated using the ventilation images agree within 5% with those measured through volume difference of lung contours on end of exhale and end of inhale images, indicating the accuracy of the ventilation images. <h3>Conclusion</h3> Lung PBV maps derived from equilibrium contrast enhanced DECT acquired for radiation therapy planning may provide a valuable new tool to assess the spatial distribution of normal lung function, alternative to the lung ventilation map. With additional study the PBV maps may be used to guide radiation treatment planning for lung cancer patients.

Adaptive planning based on single beam optimization in passive scattering carbon ion radiotherapy for patients with pancreatic cancer

BackgroundDaily anatomical deviations may distort the dose distribution in carbon ion radiotherapy (CIRT), which may cause treatment failure. Therefore, this study aimed to perform re-planning to maintain the dose coverage in patients with pancreatic cancer with passive scattering CIRT.MethodsEight patients with pancreatic cancer and 95 daily computed tomography (CT) sets were examined. Two types of adaptive plans based on new range compensators (RCs) (AP-1) and initial RCs (AP-2) were generated. In AP-2, each beam was optimized by manually adjusting the range shifter thickness and spread-out Bragg peak size to make dose reduction by < 3% of the original plan. Doses of the original plan with bone matching (BM) and tumor matching (TM) were examined for comparison. We calculated the accumulated dose using the contour and intensity-based deformable image registration algorithm. The dosimetric differences in respect to the original plan were compared between methods.ResultsUsing TM and BM, mean ± standard deviations of daily CTV V95 (%) difference from the original plan was − 5.1 ± 6.2 and − 8.8 ± 8.8, respectively, but 1.2 ± 3.4 in AP-1 and − 0.5 ± 2.1 in AP-2 (P < 0.001). AP-1 and AP-2 enabled to maintain a satisfactory accumulated dose in all patients. The dose difference was 1.2 ± 2.8, − 2,1 ± 1.7, − 7.1 ± 5.2, and − 16.5 ± 15.0 for AP-1, AP-2, TM, and BM, respectively. However, AP-2 caused a dose increase in the duodenum, especially in the left–right beam.ConclusionsThe possible dose deterioration should be considered when performing the BM, even TM. Re-planning based on single beam optimization in passive scattering CIRT seems an effective and safe method of ensuring the treatment robustness in pancreatic cancer. Further study is necessary to spare healthy tissues, especially the duodenum.

Neutron microtomography to investigate the bone-implant interface—comparison with histological analysis

Bone properties and especially its microstructure around implants are crucial to evaluate the osseointegration of prostheses in orthopaedic, maxillofacial and dental surgeries. Given the intrinsic heterogeneous nature of the bone microstructure, an ideal probing tool to understand and quantify bone formation must be spatially resolved. X-ray imaging has often been employed, but is limited in the presence of metallic implants, where severe artifacts generally arise from the high attenuation of metals to x-rays. Neutron tomography has recently been proposed as a promising technique to study bone-implant interfaces, thanks to its lower interaction with metals. The aim of this study is to assess the potential of neutron tomography for the characterisation of bone tissue in the vicinity of a metallic implant. A standardised implant with a bone chamber was implanted in rabbit bone. Four specimens were imaged with neutron tomography and subsequently compared to non-decalcified histology to stain soft and mineralised bone tissues, used here as a ground-truth reference. An intensity-based image registration procedure was performed to place the 12 histological slices within the corresponding 3D neutron volume. Significant correlations (p < 0.01) were obtained between the two modalities for the bone-implant contact (BIC) ratio (R = 0.77) and the bone content inside the chamber (R = 0.89). The results indicate that mineralised bone tissue can be reliably detected by neutron tomography. However, the BIC ratio and bone content were found to be overestimated with neutron imaging, which may be explained by its sensitivity to non-mineralised soft tissues, as revealed by histological staining. This study highlights the suitability of neutron tomography for the analysis of the bone-implant interface. Future work will focus on further distinguishing soft tissues from bone tissue, which could be aided by the adoption of contrast agents.

Technical Note: Patient-morphed mesh-type phantoms to support personalized nuclear medicine dosimetry - a proof of concept study.

Current standard practice for clinical radionuclide dosimetry utilizes reference phantoms, where defined organ dimensions represent population averages for a given sex and age. Greater phantom personalization would support more accurate dose estimations and personalized dosimetry. Tailoring phantoms is traditionally accomplished using operator-intensive organ-level segmentation of anatomic images. Modern mesh phantoms provide enhanced anatomical realism, which has motivated their integration within Monte Carlo codes. Here, we present an automatable strategy for generating patient-specific phantoms/dosimetry using intensity-based deformable image registration between mesh reference phantoms and patient CT images. This work demonstrates a proof-of-concept personalized dosimetry workflow, presented in comparison to the manual segmentation approach. A linear attenuation coefficient phantom was generated by resampling the PSRK-Man reference phantom onto a voxel grid and defining organ regions with corresponding Hounsfield unit (HU) reference values. The HU phantom was co-registered with a patient CT scan using Plastimatch B-spline deformable registration. In parallel, major organs were manually contoured to generate a "ground truth" patient-specific phantom for comparisons. Monte Carlo derived S-values, which support nuclear medicine dosimetry, were calculated using both approaches and compared. Application of the derived B-spline transform to the polygon vertices comprising the PSRK-Man yielded a deformed variant more closely matching the patient's body contour and most organ volumes as-evaluated by Hausdorff distance and Dice metrics. S-values computed for fluorine-18 for the deformed phantom using the Particle and Heavy Ion Transport code System showed improved agreement with those derived from the patient-specific analog. Deformable registration techniques can be used to create a personalized phantom and better support patient-specific dosimetry. This method is shown to be easier and faster than manual segmentation. Our study is limited to a proof-of-concept scope, but demonstrates that integration of personalized phantoms into clinical dosimetry workflows can reasonably be achieved when anatomical images (CT) are available.

Performance-aware programming for intraoperative intensity-based image registration on graphics processing units

PurposeIntensity-based image registration has been proven essential in many applications accredited to its unparalleled ability to resolve image misalignments. However, long registration time for image realignment prohibits its use in intra-operative navigation systems. There has been much work on accelerating the registration process by improving the algorithm’s robustness, but the innate computation required by the registration algorithm has been unresolved.MethodsIntensity-based registration methods involve operations with high arithmetic load and memory access demand, which supposes to be reduced by graphics processing units (GPUs). Although GPUs are widespread and affordable, there is a lack of open-source GPU implementations optimized for non-rigid image registration. This paper demonstrates performance-aware programming techniques, which involves systematic exploitation of GPU features, by implementing the diffeomorphic log-demons algorithm.ResultsBy resolving the pinpointed computation bottlenecks on GPU, our implementation of diffeomorphic log-demons on Nvidia GTX Titan X GPU has achieved ~ 95 times speed-up compared to the CPU and registered a 1.3-M voxel image in 286 ms. Even for large 37-M voxel images, our implementation is able to register in 8.56 s, which attained ~ 258 times speed-up. Our solution involves effective employment of GPU computation units, memory, and data bandwidth to resolve computation bottlenecks.ConclusionThe computation bottlenecks in diffeomorphic log-demons are pinpointed, analyzed, and resolved using various GPU performance-aware programming techniques. The proposed fast computation on basic image operations not only enhances the computation of diffeomorphic log-demons, but is also potentially extended to speed up many other intensity-based approaches. Our implementation is open-source on GitHub at https://bit.ly/2PYZxQz.

Development of panoramic infrared images for surface temperature analysis of buildings and infrastructures

The surface temperature distribution of buildings and infrastructures is a key piece of information in the analysis of built environments especially in urban areas. It not only helps understand and model the thermal performance of buildings, but also reveal the thermal environments exposed by pedestrians. The information may further help improve the energy efficiency of buildings and reduce the urban heat island (UHI) effect. However, effective methods are currently lacking to visualize and analyze the overall surface temperature of buildings and urban infrastructures in a large scale. To cover this gap, a systematic approach is developed in this study to generate panoramic infrared images. The approach includes: (1) use of a ground-based imaging system to capture visual and infrared images, (2) development of spatial transformation models between the visual and infrared images using intensity-based image registration method, (3) creation of the panoramic visual image and fusion of the panoramic visual and infrared images, (4) geometrical transformation of the panoramic images with a modified cylindrical projection method. Three cases are used to illustrate the potential applications of the approach in the analysis of building surfaces. The cases indicate that the methods can effectively reveal the temporal and spatial temperature variations of large building surfaces, providing insights on the effects of cladding materials on surface temperature.

Evaluation of four-dimensional cone beam computed tomography ventilation images acquired with two different linear accelerators at various gantry speeds using a deformable lung phantom.

We evaluated four-dimensional cone beam computed tomography (4D-CBCT) ventilation images (VICBCT) acquired with two different linear accelerator systems at various gantry speeds using a deformable lung phantom. The 4D-CT and 4D-CBCT scans were performed using a computed tomography (CT) scanner, an X-ray volume imaging system (Elekta XVI) mounted in Versa HD, and an On-Board Imager (OBI) system mounted in TrueBeam. Intensity-based deformable image registration (DIR) was performed between peak-exhale and peak-inhale images. VICBCT- and 4D-CT-based ventilation images (VICT) were derived by DIR using two metrics: one based on the Jacobian determinant and one on changes in the Hounsfield unit (HU). Three different DIR regularization values (λ) were used for VICBCT. Correlations between the VICBCT and VICT values were evaluated using voxel-wise Spearman's rank correlation coefficient (r). In case of both metrics, the Jacobian-based VICBCT with a gantry speed of 0.6deg/sec in Versa HD showed the highest correlation for all the gantry speeds (e.g., λ=0.05 and r=0.68). Thus, the r value of the Jacobian-based VICBCT was greater or equal to that of the HU-based VICBCT. In addition, the ventilation accuracy of VICBCT increased at low gantry speeds. Thus, the image quality of VICBCT was affected by the change in gantry speed in both the imaging systems. Additionally, DIR regularization considerably influenced VICBCT in both the imaging systems. Our results have the potential to assist in designing CBCT protocols, incorporating VICBCT imaging into the functional avoidance planning process.

Approach of spectral information-based image registration similarity

For local intensity-based image registration methods, a template of a predefined size is adopted. Some measure of similarity between the images is used to determine when the optimal alignment has occurred. Most intensity-based similarity utilizes the statistical and spatial information of an image. However, there are significant nonlinear radiometric differences between infrared and visible data. Images of natural scenes usually do not have enough spatial features within the template. Hence, many similarities will be useless when dealing with infrared and visible data. Minor eigenvalues (ME) image registration similarity is presented with the exploitation the spectral properties of remote sensing images. ME similarity is based on the linear spectral mixture model and detects control points through searching the minimum of ME of the covariance matrix. Experiments on Landsat-7 satellite Enhanced Thematic Mapper Plus data are performed to verify and evaluate the effectiveness. Transformation performance curves, correct match ratio (CMR), and registration accuracy are also discussed. According to the data, root-mean-square error of phase correlations is 0.0722 pixels and the CMR of ME similarity is nearly 100%. The results on the basis of TM1, TM2, TM3, and TM4 band images indicate that the proposed similarity holds promise for infrared and color image registration in natural scenes, with advantages over previous normal mutual information and gradient mutual information similarities.

4DCT Ventilation Map Construction Using Biomechanics-base Image Registration and Enhanced Air Segmentation.

Current lung radiation therapy (RT) treatment planning algorithms used in most centers assume homogeneous lung function. However, co-existing pulmonary dysfunctions present in many non-small cell lung cancer (NSCLC) patients, particularly smokers, cause regional variations in both perfusion and ventilation, leading to inhomogeneous lung function. An adaptive RT treatment planning that deliberately avoids highly functional lung regions can potentially reduce pulmonary toxicity and morbidity. The ventilation component of lung function can be measured using a variety of techniques. Recently, 4DCT ventilation imaging has emerged as a cost-effective and accessible method. Current 4DCT ventilation calculation methods, including the intensity-based and Jacobian models, suffer from inaccurate estimations of air volume distribution and unreliability of intensity-based image registration algorithms. In this study, we propose a novel method that utilizes a biomechanical model-based registration along with an accurate air segmentation algorithm to calculate 4DCT ventilation maps. The results show a successful development of ventilation maps using the proposed method.

A supervised network for fast image-guided radiotherapy (IGRT) registration.

3D/3D image registration in IGRT, which aligns planning Computed Tomography (CT) image set with on board Cone Beam CT (CBCT) image set in a short time with high accuracy, is still a challenge due to its high computational cost and complex anatomical structure of medical image. In order to overcome these difficulties, a new method is proposed which contains a coarse registration and a fine registration. For the coarse registration, a supervised regression convolutional neural networks (CNNs) is used to optimize the spatial variation by minimizing the loss when combine the CT images with the CBCT images. For the fine registration, intensity-based image registration is used to calculate the accurate spatial difference of the input image pairs. A coarse registration can get a rough result with a wide capture range in less than 0.5s. Sequentially a fine registration can get accurate results in a reasonable short time. RSD-111T chest phantom was used to test our new method. The set-up error was calculated in less than 10s in time scale, and was reduced to sub-millimeter level in spatial scale. The average residual errors in translation and rotation are within ±0.5mm and ± 0.2°.