Mean Structural Similarity Index Measure Research Articles

Enhancing the Rainfall Forecasting Accuracy of Ensemble Numerical Prediction Systems via Convolutional Neural Networks

Abstract Ensemble prediction systems are commonly used to demonstrate the potential uncertainty of weather forecasts. These systems also help provide weather predictions to prepare for disasters. Specifically, a type of prediction called a quantitative precipitation forecast (QPF) is calculated from the average of multiple forecasts. The probability-matched mean (PMM) method is used to improve these QPF predictions when they underestimate rainfall. However, the PMM method can exhibit limitations when dealing with certain types of rain patterns. To address this issue, this study focuses on improving short-term heavy rainfall predictions using an artificial intelligence (AI) algorithm that combines deep neural networks (DNNs), including convolutional neural networks (CNNs), with a space-based attention mechanism (convolutional block attention module [CBAM]). Four experiments were conducted to evaluate the new method, including those performed to optimize the timing of a 24-hour QPF model, adjust the training dataset, and refine the training algorithm. Two specific weather events were assessed: rainfall influenced by southwest winds after typhoons and afternoon thunderstorm rainfall. A comparison of the results obtained via the root mean square error (RMSE) and structural similarity index measure (SSIM) reveals that the accuracy and distribution of the QPF predictions significantly improved over those of the PMM method. Compared with the PMM method, our approach can reduce the RMSE from 77.51 to 19.52 in the Mei-yu case, an improvement of approximately 74.82%. The SSIM also increased from 0.33 to 0.56, indicating a 70.9% enhancement. Overall, the new approach successfully enhances rainfall prediction accuracy via AI techniques and has the potential to be applied in disaster preparedness operations.

Accelerated cardiac cine with spatio-coil regularized deep learning reconstruction.

To develop an iterative deep learning (DL) reconstruction with spatio-coil regularization and multichannel k-space data consistency for accelerated cine imaging. This study proposes a Spatio-Coil Regularized DL (SCR-DL) approach for iterative deep learning reconstruction incorporating multicoil information in data consistency and regularizer. SCR-DL uses shift-invariant convolutional kernels to interpolate missing k-space lines and reconstruct individual coil images, followed by a regularizer that operates simultaneously across spatial and coil dimensions using learned image priors. At 8-fold acceleration, SCR-DL was compared with Generalized Autocalibrating Partially Parallel Acquisition (GRAPPA), sensitivity encoding (SENSE)-based DL and spatio-temporal regularized (STR)-DL reconstruction. In the retrospective undersampled cine, images were quantitatively evaluated using normalized mean square error (NMSE) and structural similarity index measure (SSIM). Additionally, agreement for left-ventricular ejection fraction and left-ventricular mass were assessed using prospectively accelerated cine images at 2-fold and 8-fold accelerations. The SCR-DL algorithm successfully reconstructed highly accelerated cine images. SCR-DL had significant improvements in NMSE (0.03 ± 0.02) and SSIM (91.4% ± 2.7%) compared with GRAPPA (NMSE: 0.09 ± 0.04, SSIM: 69.9% ± 11.1%; p < 0.001), SENSE-DL (NMSE: 0.07 ± 0.04, SSIM: 86.9% ± 3.2%; p < 0.001), and STR-DL (NMSE: 0.04 ± 0.03, SSIM: 90.0% ± 2.5%; p < 0.001) with retrospective undersampled cine. Despite the 3-fold reduction in scan time, there was no difference between left-ventricular ejection fraction (59.8 ± 4.5 vs. 60.8 ± 4.8, p = 0.46) or left-ventricular mass (73.6 ± 19.4 g vs. 73.2 ± 19.7 g, p = 0.95) between R = 2 and R = 8 prospectively accelerated cine images. SCR-DL enabled highly accelerated cardiac cine imaging, significantly reducing breath-hold time. Compared with GRAPPA or SENSE-DL, images reconstructed with SCR-DL showed superior NMSE and SSIM.

Using a patient-specific diffusion model to generate CBCT-based synthetic CTs for CBCT-guided adaptive radiotherapy.

Cone beam computed tomography (CBCT) can be used to evaluate the inter-fraction anatomical changes during the entire course for image-guided radiotherapy (IGRT). However, CBCT artifacts from various sources restrict the full application of CBCT-guided adaptive radiation therapy (ART). Inter-fraction anatomical changes during ART, including variations in tumor size and normal tissue anatomy, can affect radiation therapy (RT) efficacy. Acquiring high-quality CBCT images that accurately capture patient- and fraction-specific (PFS) anatomical changes is crucial for successful IGRT. To enhance CBCT image quality, we proposed PFS lung diffusion models (PFS-LDMs). The proposed PFS models use a pre-trained general lung diffusion model (GLDM) as a baseline, which is trained on historical deformed CBCT (dCBCT)-planning CT (pCT) paired data. For a given patient, a new PFS model is fine-tuned on a CBCT-deformed pCT (dpCT) pair after each fraction to learn the PFS knowledge for generating personalized synthetic CT (sCT) with quality comparable to pCT or dpCT. The learned PFS knowledge is the specific mapping relationships, including personal inter-fraction anatomical changes between personalized CBCT-dpCT pairs. The PFS-LDMs were evaluated on an institutional lung cancer dataset, quantified by mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and structural similarity index measure (SSIM) metrics. We also compared our PFS-LDMs with a mainstream GAN-based model, demonstrating that our PFS fine-tuning strategy could be applied to existing generative models. Our models showed remarkable improvements across all four evaluation metrics. The proposed PFS-LDMs outperformed the GLDM, demonstrating the effectiveness of our proposed fine-tuning strategy. The PFS model fine-tuned with CBCT images from four prior fractions, reduced the MAE from 103.95 to 15.96 Hounsfield units (HU), and increased the mean PSNR, NCC, and SSIM from 25.36dB to 33.57dB, 0.77 to 0.98, and 0.75 to 0.97, respectively. Applying our PFS fine-tuning strategy to a Cycle GAN model also showed improvements, with all four fine-tuned PFS Cycle GAN (PFS-CG) models outperforming the general Cycle GAN model. Overall, our proposed PFS fine-tuning strategy improved CBCT image quality compared to both the pre-correction and non-fine-tuned general models, with our proposed PFS-LDMs yielding better performance than the GAN-based model across all metrics. Our proposed PFS-LDMs significantly improve CBCT image quality with increased HU accuracy and fewer artifacts, thus better capturing inter-fraction anatomical changes. This lays the groundwork for enabling CBCT-based ART, which could enhance clinical efficiency and achieve personalized high-precision treatment by accounting for inter-fraction anatomical changes.

Nested CNN architecture for three-dimensional dose distribution prediction in tomotherapy for prostate cancer.

The hypothesis of changing network layers to increase the accuracy of dose distribution prediction, instead of expanding their dimensions, which requires complex calculations, has been considered in our study. Atotal of 137 prostate cancer patients treated with the tomotherapy technique were categorized as 80% training and validating as well as 20% testing for the nested UNet and UNet architectures. Mean absolute error (MAE) was used to measure the dosimetry indices of dose-volume histograms (DVHs), and geometry indices, including the structural similarity index measure (SSIM), dice similarity coefficient (DSC), and Jaccard similarity coefficient (JSC), were used to evaluate the isodose volume (IV) similarity prediction. To verify a statistically significant difference, the two-way statistical Wilcoxon test was used at alevel of 0.05 (p < 0.05). Use of anested UNet architecture reduced the predicted dose MAE in DVH indices. The MAE for planning target volume (PTV), bladder, rectum, and right and left femur were D98% = 1.11 ± 0.90; D98% = 2.27 ± 2.85, Dmean = 0.84 ± 0.62; D98% = 1.47 ± 12.02, Dmean = 0.77 ± 1.59; D2% = 0.65 ± 0.70, Dmean = 0.96 ± 2.82; and D2% = 1.18 ± 6.65, Dmean = 0.44 ± 1.13, respectively. Additionally, the greatest geometric similarity was observed in the mean SSIM for UNet and nested UNet (0.91 vs.0.94, respectively). The nested UNet network can be considered asuitable network due to its ability to improve the accuracy of dose distribution prediction compared to the UNet network in an acceptable time.

Perfusion parameter map generation from TOF-MRA in stroke using generative adversarial networks

PurposeTo generate perfusion parameter maps from Time-of-flight magnetic resonance angiography (TOF-MRA) images using artificial intelligence to provide an alternative to traditional perfusion imaging techniques. Materials and methodsThis retrospective study included a total of 272 patients with cerebrovascular diseases; 200 with acute stroke (from 2010 to 2018), and 72 with steno-occlusive disease (from 2011 to 2014). For each patient the TOF MRA image and the corresponding Dynamic susceptibility contrast magnetic resonance imaging (DSC-MRI) were retrieved from the datasets. The authors propose an adapted generative adversarial network (GAN) architecture, 3D pix2pix GAN, that generates common perfusion maps (CBF, CBV, MTT, TTP, Tmax) from TOF-MRA images. The performance was evaluated by the structural similarity index measure (SSIM). For a subset of 20 patients from the acute stroke dataset, the Dice coefficient was calculated to measure the overlap between the generated and real hypoperfused lesions with a time-to-maximum (Tmax) > 6 s. ResultsThe GAN model exhibited high visual overlap and performance for all perfusion maps in both datasets: acute stroke (mean SSIM 0.88–0.92, mean PSNR 28.48–30.89, mean MAE 0.02–0.04 and mean NRMSE 0.14–0.37) and steno-occlusive disease patients (mean SSIM 0.83–0.98, mean PSNR 23.62–38.21, mean MAE 0.01–0.05 and mean NRMSE 0.03–0.15). For the overlap analysis for lesions with Tmax>6 s, the median Dice coefficient was 0.49. ConclusionOur AI model can successfully generate perfusion parameter maps from TOF-MRA images, paving the way for a non-invasive alternative for assessing cerebral hemodynamics in cerebrovascular disease patients. This method could impact the stratification of patients with cerebrovascular diseases. Our results warrant more extensive refinement and validation of the method.

Motion Correction for Brain MRI Using Deep Learning and a Novel Hybrid Loss Function

Purpose: Motion-induced magnetic resonance imaging (MRI) artifacts can deteriorate image quality and reduce diagnostic accuracy, but motion by human subjects is inevitable and can even be caused by involuntary physiological movements. Deep-learning-based motion correction methods might provide a solution. However, most studies have been based on directly applying existing models, and the trained models are rarely accessible. Therefore, we aim to develop and evaluate a deep-learning-based method (Motion Correction-Net, or MC-Net) for suppressing motion artifacts in brain MRI scans. Methods: A total of 57 subjects, providing 20,889 slices in four datasets, were used. Furthermore, 3T 3D sagittal magnetization-prepared rapid gradient-echo (MP-RAGE) and 2D axial fluid-attenuated inversion-recovery (FLAIR) sequences were acquired. The MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network to remove motion artifacts. Evaluation used simulated T1- and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans. The performance indices included the peak-signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), and visual reading scores from three blinded clinical readers. A one-sided Wilcoxon signed-rank test was used to compare reader scores, with p &lt; 0.05 considered significant. Intraclass correlation coefficients (ICCs) were calculated for inter-rater evaluations. Results: The MC-Net outperformed other methods in terms of PSNR and SSIM for the T1 axial test set. The MC-Net significantly improved the quality of all T1-weighted images for all directions (i.e., the mean SSIM of axial, sagittal, and coronal slices improved from 0.77, 0.64, and 0.71 to 0.92, 0.75, and 0.84; the mean PSNR improved from 26.35, 24.03, and 24.55 to 29.72, 24.40, and 25.37, respectively) and for simulated as well as real motion artifacts, both using quantitative measures and visual scores. However, MC-Net performed poorly for images with untrained T2-weighted contrast because the T2 contrast was unseen during training and is different from T1 contrast. Conclusion: The proposed two-stage multi-loss MC-Net can effectively suppress motion artifacts in brain MRI without compromising image quality. Given the efficiency of MC-Net (with a single-image processing time of ~40 ms), it can potentially be used in clinical settings.

Improving Automated Hemorrhage Detection at Sparse-View CT via U-Net-based Artifact Reduction.

Purpose To explore the potential benefits of deep learning-based artifact reduction in sparse-view cranial CT scans and its impact on automated hemorrhage detection. Materials and Methods In this retrospective study, a U-Net was trained for artifact reduction on simulated sparse-view cranial CT scans in 3000 patients, obtained from a public dataset and reconstructed with varying sparse-view levels. Additionally, EfficientNet-B2 was trained on full-view CT data from 17 545 patients for automated hemorrhage detection. Detection performance was evaluated using the area under the receiver operating characteristic curve (AUC), with differences assessed using the DeLong test, along with confusion matrices. A total variation (TV) postprocessing approach, commonly applied to sparse-view CT, served as the basis for comparison. A Bonferroni-corrected significance level of .001/6 = .00017 was used to accommodate for multiple hypotheses testing. Results Images with U-Net postprocessing were better than unprocessed and TV-processed images with respect to image quality and automated hemorrhage detection. With U-Net postprocessing, the number of views could be reduced from 4096 (AUC: 0.97 [95% CI: 0.97, 0.98]) to 512 (0.97 [95% CI: 0.97, 0.98], P < .00017) and to 256 views (0.97 [95% CI: 0.96, 0.97], P < .00017) with a minimal decrease in hemorrhage detection performance. This was accompanied by mean structural similarity index measure increases of 0.0210 (95% CI: 0.0210, 0.0211) and 0.0560 (95% CI: 0.0559, 0.0560) relative to unprocessed images. Conclusion U-Net-based artifact reduction substantially enhanced automated hemorrhage detection in sparse-view cranial CT scans. Keywords: CT, Head/Neck, Hemorrhage, Diagnosis, Supervised Learning Supplemental material is available for this article. © RSNA, 2024.

Swin Transformer and the Unet Architecture to Correct Motion Artifacts in Magnetic Resonance Image Reconstruction.

We present a deep learning-based method that corrects motion artifacts and thus accelerates data acquisition and reconstruction of magnetic resonance images. The novel model, the Motion Artifact Correction by Swin Network (MACS-Net), uses a Swin transformer layer as the fundamental block and the Unet architecture as the neural network backbone. We employ a hierarchical transformer with shifted windows to extract multiscale contextual features during encoding. A new dual upsampling technique is employed to enhance the spatial resolutions of feature maps in the Swin transformer-based decoder layer. A raw magnetic resonance imaging dataset is used for network training and testing; the data contain various motion artifacts with ground truth images of the same subjects. The results were compared to six state-of-the-art MRI image motion correction methods using two types of motions. When motions were brief (within 5 s), the method reduced the average normalized root mean square error (NRMSE) from 45.25% to 17.51%, increased the mean structural similarity index measure (SSIM) from 79.43% to 91.72%, and increased the peak signal-to-noise ratio (PSNR) from 18.24 to 26.57 dB. Similarly, when motions were extended from 5 to 10 s, our approach decreased the average NRMSE from 60.30% to 21.04%, improved the mean SSIM from 33.86% to 90.33%, and increased the PSNR from 15.64 to 24.99 dB. The anatomical structures of the corrected images and the motion-free brain data were similar.

Research and Implementation of Denoising Algorithm for Brain MRIs via Morphological Component Analysis and Adaptive Threshold Estimation

The inevitable noise generated in the acquisition and transmission process of MRIs seriously affects the reliability and accuracy of medical research and diagnosis. The denoising effect for Rician noise, whose distribution is related to MR image signal, is not good enough. Furthermore, the brain has a complex texture structure and a small density difference between different parts, which leads to higher quality requirements for brain MR images. To upgrade the reliability and accuracy of brain MRIs application and analysis, we designed a new and dedicated denoising algorithm (named VST–MCAATE), based on their inherent characteristics. Comparative experiments were performed on the same simulated and real brain MR datasets. The peak signal-to-noise ratio (PSNR), and mean structural similarity index measure (MSSIM) were used as objective image quality evaluation. The one-way ANOVA was used to compare the effects of denoising between different approaches. p &lt; 0.01 was considered statistically significant. The experimental results show that the PSNR and MSSIM values of VST–MCAATE are significantly higher than state-of-the-art methods (p &lt; 0.01), and also that residual images have no anatomical structure. The proposed denoising method has advantages in improving the quality of brain MRIs, while effectively removing the noise with a wide range of unknown noise levels without damaging texture details, and has potential clinical promise.

SwinUNet: a multiscale feature learning approach to cardiovascular magnetic resonance parametric mapping for myocardial tissue characterization.

Objective.Cardiovascular magnetic resonance (CMR) can measure T1 and T2 relaxation times for myocardial tissue characterization. However, the CMR procedure for T1/T2 parametric mapping is time-consuming, making it challenging to scan heart patients routinely in clinical practice. This study aims to accelerate CMR parametric mapping with deep learning.Approach. A deep-learning model, SwinUNet, was developed to accelerate T1/T2 mapping. SwinUNet used a convolutional UNet and a Swin transformer to form a hierarchical 3D computation structure, allowing for analyzing CMR images spatially and temporally with multiscale feature learning. A comparative study was conducted between SwinUNet and an existing deep-learning model, MyoMapNet, which only used temporal analysis for parametric mapping. The T1/T2 mapping performance was evaluated globally using mean absolute error (MAE) and structural similarity index measure (SSIM). The clinical T1/T2 indices for characterizing the left-ventricle myocardial walls were also calculated and evaluated using correlation and Bland-Altman analysis.Main results. We performed accelerated T1 mapping with ≤4 heartbeats and T2 mapping with 2 heartbeats in reference to the clinical standard, which required 11 heartbeats for T1 mapping and 3 heartbeats for T2 mapping. SwinUNet performed well in all the experiments (MAE < 50 ms, SSIM > 0.8, correlation > 0.75, and Bland-Altman agreement limits < 100 ms for T1 mapping; MAE < 1 ms, SSIM > 0.9, correlation > 0.95, and Bland-Altman agreement limits < 1.5 ms for T2 mapping). When the maximal acceleration was used (2 heartbeats), SwinUNet outperformed MyoMapNet and gave measurement accuracy similar to the clinical standard.Significance. SwinUNet offers an optimal solution to CMR parametric mapping for assessing myocardial diseases quantitatively in clinical cardiology.

Impact of cone-beam computed tomography artifacts on dose calculation accuracy for lung cancer.

To implement image-guided adaptive radiotherapy (IGART), many studies investigated dose calculations on cone-beam computed tomography (CBCT). A high HU accuracy is crucial for a high dose calculation accuracy and many imaging sites showed satisfactory results. It has been shown that the dose calculation accuracy for lung cancer lags behind. To examine why the dose calculation accuracy for lung is insufficient, the relative effects of the field-of-view (FOV), breathing motion, and scatter on dose calculation accuracy were studied. A framework was built to simulate CBCT scans for lung cancer patients by forward projecting repeat CT (rCT) scans for two scan geometries: small (SFOV) and medium FOV (MFOV). Breathing motion was modeled by applying a 4D deformation vector field to the mid-position rCT. Scatter was modeled by Monte-Carlo simulations with/without an anti-scatter grid (ASG). Simulated projections were reconstructed using filtered back-projection with/without scatter correction. In case of the SFOV, the CBCT images were patched with the planning CT scan in axial direction. The treatment plan was recalculated on the rCT and simulated CBCT. The mean Hounsfield unit (HU) difference (ΔHUmean), the structural similarity index measure (SSIM), and γ metrics were calculated for the CBCT datasets of various imaging settings. The differences in HU, SSIM and dose calculation accuracy for CBCTs with and without breathing motion were negligible (mean ΔHUmean=6.4vs. 13.7, mean SSIM=0.941vs. 0.957, mean γ (ref=MFOV)=0.75). The SFOV resulted in a lower HU (mean ΔHUmean=-9.2vs. 13.7) and SSIM (mean SSIM=0.912vs. 0.957), and therefore in dose differences compared to the MFOV (mean γ=1.22). Scatter led to considerable discrepancies in all metrics. Adding only the ASG improved the results more than only applying a scatter correction algorithm. Combining ASG and scatter correction algorithm resulted in an even higher dose calculation accuracy. Scatter and FOV are the main contributors to dose inaccuracies and motion has only a minor effect on dose calculation accuracy. Therefore, utilizing an appropriate scatter correction and FOV is important to achieve sufficient dose calculation accuracy to facilitate IGART for lung.

Deep learning-based prediction of the retinal structural alterations after epiretinal membrane surgery

To generate and evaluate synthesized postoperative OCT images of epiretinal membrane (ERM) based on preoperative OCT images using deep learning methodology. This study included a total 500 pairs of preoperative and postoperative optical coherence tomography (OCT) images for training a neural network. 60 preoperative OCT images were used to test the neural networks performance, and the corresponding postoperative OCT images were used to evaluate the synthesized images in terms of structural similarity index measure (SSIM). The SSIM was used to quantify how similar the synthesized postoperative OCT image was to the actual postoperative OCT image. The Pix2Pix GAN model was used to generate synthesized postoperative OCT images. Total 60 synthesized OCT images were generated with training values at 800 epochs. The mean SSIM of synthesized postoperative OCT to the actual postoperative OCT was 0.913. Pix2Pix GAN model has a possibility to generate predictive postoperative OCT images following ERM removal surgery.

Image Translation for Estimating Two-Dimensional Axial Amyloid-Beta PET From Structural MRI.

Amyloid-beta and brain atrophy are hallmarks for Alzheimer's Disease that can be targeted with positron emission tomography (PET) and MRI, respectively. MRI is cheaper, less-invasive, and more available than PET. There is a known relationship between amyloid-beta and brain atrophy, meaning PET images could be inferred from MRI. To build an image translation model using a Conditional Generative Adversarial Network able to synthesize Amyloid-beta PET images from structural MRI. Retrospective. Eight hundred eighty-two adults (348 males/534 females) with different stages of cognitive decline (control, mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment). Five hundred fifty-two subjects for model training and 331 for testing (80%:20%). 3 T, T1-weighted structural (T1w). The testing cohort was used to evaluate the performance of the model using the Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR), comparing the likeness of the overall synthetic PET images created from structural MRI with the overall true PET images. SSIM was computed in the overall image to include the luminance, contrast, and structural similarity components. Experienced observers reviewed the images for quality, performance and tried to determine if they could tell the difference between real and synthetic images. Pixel wise Pearson correlation was significant, and had an R2 greater than 0.96 in example images. From blinded readings, a Pearson Chi-squared test showed that there was no significant difference between the real and synthetic images by the observers (P = 0.68). A high degree of likeness across the evaluation set, which had a mean SSIM = 0.905 and PSNR = 2.685. The two observers were not able to determine the difference between the real and synthetic images, with accuracies of 54% and 46%, respectively. Amyloid-beta PET images can be synthesized from structural MRI with a high degree of similarity to the real PET images. 3 TECHNICAL EFFICACY: Stage 1.

Development of Pericardial Fat Count Images Using a Combination of Three Different Deep-Learning Models: Image Translation Model From Chest Radiograph Image to Projection Image of Three-Dimensional Computed Tomography

Pericardial fat (PF)-the thoracic visceral fat surrounding the heart-promotes the development of coronary artery disease by inducing inflammation of the coronary arteries. To evaluate PF, we generated pericardial fat count images (PFCIs) from chest radiographs (CXRs) using a dedicated deep-learning model. We reviewed data of 269 consecutive patients who underwent coronary computed tomography (CT). We excluded patients with metal implants, pleural effusion, history of thoracic surgery, or malignancy. Thus, the data of 191 patients were used. We generated PFCIs from the projection of three-dimensional CT images, wherein fat accumulation was represented by a high pixel value. Three different deep-learning models, including CycleGAN were combined in the proposed method to generate PFCIs from CXRs. A single CycleGAN-based model was used to generate PFCIs from CXRs for comparison with the proposed method. To evaluate the image quality of the generated PFCIs, structural similarity index measure (SSIM), mean squared error (MSE), and mean absolute error (MAE) of (i) the PFCI generated using the proposed method and (ii) the PFCI generated using the single model were compared. The mean SSIM, MSE, and MAE were 8.56×10-1, 1.28×10-2, and 3.57×10-2, respectively, for the proposed model, and 7.62×10-1, 1.98×10-2, and 5.04×10-2, respectively, for the single CycleGAN-based model. PFCIs generated from CXRs with the proposed model showed better performance than those generated with the single model. The evaluation of PF without CT may be possible using the proposed method.

Fusion of Magnetic Resonance Elastography Images With Computed Tomography and Magnetic Resonance Imaging Using the Human Visual System.

Magnetic resonance elastography (MRE) is used to assess the stiffness of the liver to rule out cirrhosis or fibrosis. The image, nevertheless, is regarded as shear-wave imaging and does not depict any anatomical features. Multimodality medical image fusion (MMIF), such as the fusion of MRE with computed tomography (CT) scan or magnetic resonance imaging (MRI), can help doctors optimize the advantages of each imaging technique. As a result, perceptions serve as valid and valuable assessment criteria. The contrast sensitivity function (CSF), which describes the rates of visual contrast sensitivity through the changing of spatial frequencies, is used mathematically to characterize the human visual system (HVS). As a result, we suggest novel methods for fusing images that use discrete wavelets transform (DWT) based on HVS and CSF models. Images from MRI or CT scan were combined with MRE images, and the outcomes were assessed both subjectively and objectively. Visual inspection of merging images was done throughout the qualitative analysis. The CT-MRE fused images in all four datasets were shown to be superior at maintaining bones and spatial resolution, despite the MRI-MRE being better at exhibiting soft tissues and contrast resolution. It is clear from all four datasets that the liver soft tissue in MRI and CT images mixed successfully with the red-colored stiffness distribution seen in MRE images. The proposed approach outperformed DWT, which produced visual artifacts such as signal loss. Quantitative evaluation using mean, standard deviation, and entropy showed that the generated images from the proposed technique performed better than the source images and DWT. Additionally, peak signal-to-noise ratio, mean square error, correlation coefficient, and structural similarity index measure were employed to compare the two fusion approaches, namely, MRI-MRE and CT-MRE. The comparison did not show the superiority of one approach over the other. In conclusion, both subjective and objective evaluation approaches revealed that the combined images contained more information and characteristics. Hence, the proposed method might be a useful procedure to diagnose and localize the stiffness regions on the liver soft tissue by fusion of MRE with MRI or CT.

Short-axis PET image quality improvement based on a uEXPLORER total-body PET system through deep learning.

The axial field of view (AFOV) of a positron emission tomography (PET) scanner greatly affects the quality of PET images. Although a total-body PET scanner (uEXPLORER) with a large AFOV is more sensitive, it is more expensive and difficult to widely use. Therefore, we attempt to utilize high-quality images generated by uEXPLORER to optimize the quality of images from short-axis PET scanners through deep learning technology while controlling costs. The experiments were conducted using PET images of three anatomical locations (brain, lung, and abdomen) from 335 patients. To simulate PET images from different axes, two protocols were used to obtain PET image pairs (each patient was scanned once). For low-quality PET (LQ-PET) images with a 320-mm AFOV, we applied a 300-mm FOV for brain reconstruction and a 500-mm FOV for lung and abdomen reconstruction. For high-quality PET (HQ-PET) images, we applied a 1940-mm AFOV during the reconstruction process. A 3D Unet was utilized to learn the mapping relationship between LQ-PET and HQ-PET images. In addition, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) were employed to evaluate the model performance. Furthermore, two nuclear medicine doctors evaluated the image quality based on clinical readings. The generated PET images of the brain, lung, and abdomen were quantitatively and qualitatively compatible with the HQ-PET images. In particular, our method achieved PSNR values of 35.41 ± 5.45dB (p < 0.05), 33.77 ± 6.18dB (p < 0.05), and 38.58 ± 7.28dB (p < 0.05) for the three beds. The overall mean SSIM was greater than 0.94 for all patients who underwent testing. Moreover, the total subjective quality levels of the generated PET images for three beds were 3.74 ± 0.74, 3.69 ± 0.81, and 3.42 ± 0.99 (the highest possible score was 5, and the minimum score was 1) from two experienced nuclear medicine experts. Additionally, we evaluated the distribution of quantitative standard uptake values (SUV) in the region of interest (ROI). Both the SUV distribution and the peaks of the profile show that our results are consistent with the HQ-PET images, proving the superiority of our approach. The findings demonstrate the potential of the proposed technique for improving the image quality of a PET scanner with a 320mm or even shorter AFOV. Furthermore, this study explored the potential of utilizing uEXPLORER to achieve improved short-axis PET image quality at a limited economic cost, and computer-aided diagnosis systems that are related can help patients and radiologists.

A graphics-accelerated deep neural network approach for turbomachinery flows based on large eddy simulation

In turbomachinery, strongly unsteady rotor–stator interaction triggers complex three-dimensional turbulent flow phenomena such as flow separation and vortex dynamics. Large eddy simulation (LES) is an advanced numerical method that has recently been used to resolve large-scale turbulent motions and model subgrid-scale turbulence in turbomachinery. To largely reduce the computing cost of LES for turbomachinery flow, a graphics processing unit (GPU)-accelerated deep neural network-based flow field prediction approach is explored, which combines convolutional neural network autoencoder (CNN-AE) with long short-term memory (LSTM). CNN-AE extracts spatial features of turbomachinery flow by mapping high-dimensional flow fields into low-dimensional space, while LSTM is used to predict the temporal evolution of fluid dynamics. Automatic mixed precision (AMP) is employed to achieve rapid neural network training using Nvidia GTX 1080 Ti GPU, which shows a significant speedup compared with that without AMP. We evaluated the proposed CNN-AE-LSTM (CAL) method against gated recurrent units (GRU) and simple recurrent network (SRN) on two types of turbomachinery, i.e., centrifugal and axial flow pumps. The results show that the proposed CAL shows better capability of capturing the vortex structure details of turbomachinery. When predicting the temporal vorticity field, the mean square error of CAL results is 0.105%–0.124% for centrifugal pumps and 0.071%–0.072% for axial flow pumps. Meanwhile, the structural similarity index measure of the CAL results is 92.51%–92.77% for centrifugal pumps and 93.81%–94.61% for axial flow pumps. The proposed CAL is noticeably better than GRU and SRN in terms of both mean square error and structural similarity index measure.

Shortening image registration time using a deep neural network for patient positional verification in radiotherapy.

We sought to accelerate 2D/3D image registration computation time using image synthesis with a deep neural network (DNN) to generate digitally reconstructed radiographic (DRR) images from X-ray flat panel detector (FPD) images. And we explored the feasibility of using our DNN in the patient setup verification application. Images of the prostate and of the head and neck (H&N) regions were acquired by two oblique X-ray fluoroscopic units and the treatment planning CT. DNN was designed to generate DRR images from the FPD image data. We evaluated the quality of the synthesized DRR images to compare the ground-truth DRR images using the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Image registration accuracy and computation time were evaluated by comparing the 2D-3D image registration algorithm using DRR and FPD image data with DRR and synthesized DRR images. Mean PSNR values were 23.4 ± 3.7 dB and 24.1 ± 3.9 dB for the pelvic and H&N regions, respectively. Mean SSIM values for both cases were also similar (= 0.90). Image registration accuracy was degraded by a mean of 0.43mm and 0.30°, it was clinically acceptable. Computation time was accelerated by a factor of 0.69. Our DNN successfully generated DRR images from FPD image data, and improved 2D-3D image registration computation time up to 37% in average.

Generation of Conventional 18F-FDG PET Images from 18F-Florbetaben PET Images Using Generative Adversarial Network: A Preliminary Study Using ADNI Dataset.

Background and Objectives: 18F-fluorodeoxyglucose (FDG) positron emission tomography (PET) (PETFDG) image can visualize neuronal injury of the brain in Alzheimer's disease. Early-phase amyloid PET image is reported to be similar to PETFDG image. This study aimed to generate PETFDG images from 18F-florbetaben PET (PETFBB) images using a generative adversarial network (GAN) and compare the generated PETFDG (PETGE-FDG) with real PETFDG (PETRE-FDG) images using the structural similarity index measure (SSIM) and the peak signal-to-noise ratio (PSNR). Materials and Methods: Using the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, 110 participants with both PETFDG and PETFBB images at baseline were included. The paired PETFDG and PETFBB images included six and four subset images, respectively. Each subset image had a 5 min acquisition time. These subsets were randomly sampled and divided into 249 paired PETFDG and PETFBB subset images for the training datasets and 95 paired subset images for the validation datasets during the deep-learning process. The deep learning model used in this study is composed of a GAN with a U-Net. The differences in the SSIM and PSNR values between the PETGE-FDG and PETRE-FDG images in the cycleGAN and pix2pix models were evaluated using the independent Student's t-test. Statistical significance was set at p ≤ 0.05. Results: The participant demographics (age, sex, or diagnosis) showed no statistically significant differences between the training (82 participants) and validation (28 participants) groups. The mean SSIM between the PETGE-FDG and PETRE-FDG images was 0.768 ± 0.135 for the cycleGAN model and 0.745 ± 0.143 for the pix2pix model. The mean PSNR was 32.4 ± 9.5 and 30.7 ± 8.0. The PETGE-FDG images of the cycleGAN model showed statistically higher mean SSIM than those of the pix2pix model (p < 0.001). The mean PSNR was also higher in the PETGE-FDG images of the cycleGAN model than those of pix2pix model (p < 0.001). Conclusions: We generated PETFDG images from PETFBB images using deep learning. The cycleGAN model generated PETGE-FDG images with a higher SSIM and PSNR values than the pix2pix model. Image-to-image translation using deep learning may be useful for generating PETFDG images. These may provide additional information for the management of Alzheimer's disease without extra image acquisition and the consequent increase in radiation exposure, inconvenience, or expenses.

Fast optical coherence tomography angiography image acquisition and reconstruction pipeline for skin application.

Traditional high-quality OCTA images require multi-repeated scans (e.g., 4-8 repeats) in the same position, which may cause the patient to be uncomfortable. We propose a deep-learning-based pipeline that can extract high-quality OCTA images from only two-repeat OCT scans. The performance of the proposed image reconstruction U-Net (IRU-Net) outperforms the state-of-the-art UNet vision transformer and UNet in OCTA image reconstruction from a two-repeat OCT signal. The results demonstrated a mean peak-signal-to-noise ratio increased from 15.7 to 24.2; the mean structural similarity index measure improved from 0.28 to 0.59, while the OCT data acquisition time was reduced from 21 seconds to 3.5 seconds (reduced by 83%).