Image Similarity Metrics Research Articles

Automatic Identification of Game Stuttering via Gameplay Videos Analysis

Modern video games are extremely complex software systems and, as such, they might suffer from several types of post-release issues. A particularly insidious issue is constituted by drops in the frame rate ( i.e. , stuttering events), which might have a negative impact on the user experience. Stuttering events are frequently documented in the million of hours of gameplay videos shared by players on platforms such as Twitch or YouTube. From the developers’ perspective, these videos represent a free source of documented “testing activities”. However, especially for popular games, the quantity and length of these videos make impractical their manual inspection. We introduce HASTE, an approach for the automatic detection of stuttering events in gameplay videos that can be exploited to generate candidate bug reports. HASTE firstly splits a given video into visually coherent slices, with the goal of filtering-out those that not representing actual gameplay ( e.g ., navigating the game settings). Then, it identifies the subset of pixels in the video frames which actually show the game in action excluding additional elements on screen such as the logo of the YouTube channel, on-screen chats etc. In this way, HASTE can exploit state-of-the-art image similarity metrics to identify candidate stuttering events, namely subsequent frames being almost identical in the pixels depicting the game. We evaluate the different steps behind HASTE on a total of 105 videos showing that it can correctly extract video slices with a 76% precision, and can correctly identify the slices related to gameplay with a recall and precision higher than 77%. Overall, HASTE achieves 71% recall and 89% precision for the identification of stuttering events in gameplay videos.

Generating synthetic computed tomography for radiotherapy: SynthRAD2023 challenge report

Radiation therapy plays a crucial role in cancer treatment, necessitating precise delivery of radiation to tumors while sparing healthy tissues over multiple days. Computed tomography (CT) is integral for treatment planning, offering electron density data crucial for accurate dose calculations. However, accurately representing patient anatomy is challenging, especially in adaptive radiotherapy, where CT is not acquired daily. Magnetic resonance imaging (MRI) provides superior soft-tissue contrast. Still, it lacks electron density information, while cone beam CT (CBCT) lacks direct electron density calibration and is mainly used for patient positioning.Adopting MRI-only or CBCT-based adaptive radiotherapy eliminates the need for CT planning but presents challenges. Synthetic CT (sCT) generation techniques aim to address these challenges by using image synthesis to bridge the gap between MRI, CBCT, and CT. The SynthRAD2023 challenge was organized to compare synthetic CT generation methods using multi-center ground truth data from 1080 patients, divided into two tasks: (1) MRI-to-CT and (2) CBCT-to-CT. The evaluation included image similarity and dose-based metrics from proton and photon plans.The challenge attracted significant participation, with 617 registrations and 22/17 valid submissions for tasks 1/2. Top-performing teams achieved high structural similarity indices (≥0.87/0.90) and gamma pass rates for photon (≥98.1%/99.0%) and proton (≥97.3%/97.0%) plans. However, no significant correlation was found between image similarity metrics and dose accuracy, emphasizing the need for dose evaluation when assessing the clinical applicability of sCT.SynthRAD2023 facilitated the investigation and benchmarking of sCT generation techniques, providing insights for developing MRI-only and CBCT-based adaptive radiotherapy. It showcased the growing capacity of deep learning to produce high-quality sCT, reducing reliance on conventional CT for treatment planning.

Diffusion model enables quantitative CBF analysis of Alzheimer's Disease.

Cerebral blood flow (CBF) measured by arterial spin labeling (ASL) is a promising biomarker for Alzheimer's Disease (AD). ASL data from multiple vendors were included in the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. However, the M0 images were missing in Siemens ASL data, prohibiting CBF quantification. Here, we utilized a generative diffusion model to impute the missing M0 and validated generated CBF data with acquired data from GE. A conditional latent diffusion model was trained to generate the M0 image and validate it on an in-house dataset (N=55) based on image similarity metrics, accuracy of CBF quantification, and consistency with the physical model. This model was then applied to the ADNI dataset (Siemens: N=211) to impute the missing M0 for CBF calculation. We further compared the imputed data (Siemens) and acquired data (GE) regarding regional CBF differences by AD stages, their classification accuracy for AD prediction, and CBF trajectory slopes estimated by a mixed effect model. The trained diffusion model generated the M0 image with high fidelity (Structural similarity index, SSIM=0.924±0.019; peak signal-to-noise ratio, PSNR=33.348±1.831) and caused minimal bias in CBF values (mean difference in whole brain is 1.07±2.12ml/100g/min). Both generated and acquired CBF data showed similar differentiation patterns by AD stages, similar classification performance, and decreasing slopes with AD progression in specific AD-related regions. Generated CBF data also improved accuracy in classifying AD stages compared to qualitative perfusion data. This study shows the potential of diffusion models for imputing missing modalities for large-scale studies of CBF variation with AD.

Lung CT harmonization of paired reconstruction kernel images using generative adversarial networks.

The kernel used in CT image reconstruction is an important factor that determines the texture of the CT image. Consistency of reconstruction kernel choice is important for quantitative CT-based assessment as kernel differences can lead to substantial shifts in measurements unrelated to underlying anatomical structures. In this study, we investigate kernel harmonization in a multi-vendor low-dose CT lung cancer screening cohort and evaluate our approach's validity in quantitative CT-based assessments. Using the National Lung Screening Trial, we identified CT scan pairs of the same sessions with one reconstructed from a soft tissue kernel and one from a hard kernel. In total, 1000 pairs of five different paired kernel types (200 each) were identified. We adopt the pix2pix architecture to train models for kernel conversion. Each model was trained on 100 pairs and evaluated on 100 withheld pairs. A total of 10 models were implemented. We evaluated the efficacy of kernel conversion based on image similarity metrics including root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM) as well as the capability of the models to reduce measurement shifts in quantitative emphysema and body composition measurements. Additionally, we study the reproducibility of standard radiomic features for all kernel pairs before and after harmonization. Our approach effectively converts CT images from one kernel to another in all paired kernel types, as indicated by the reduction in RMSE (p<0.05) and an increase in the PSNR (p<0.05) and SSIM (p<0.05) for both directions of conversion for all pair types. In addition, there is an increase in the agreement for percent emphysema, skeletal muscle area, and subcutaneous adipose tissue (SAT) area for both directions of conversion. Furthermore, radiomic features were reproducible when compared with the ground truth features. Kernel conversion using deep learning reduces measurement variation in percent emphysema, muscle area, and SAT area.

Deformable image registration for composite planned doses during adaptive radiation therapy

IntroductionSome patients have significant anatomic changes during radiotherapy, necessitating an adaptive repeat CT-simulation and re-planning. This yields two unique planning datasets that introduce uncertainty into total dose records. This study explored the impact of using deformable image registration (DIR) to spatially align repeat CT-simulation images and calculate total planned dose distributions. Materials & methodsData from 5 head-and-neck, 5 lung, and 5 sarcoma patients who had unanticipated re-planning during radiotherapy were analyzed in a treatment planning system (RayStation v6.1 RaySearch Laboratories). Total planned doses to normal tissues were calculated using two methods and the previously generated manual contours defined on each CT. The first method, termed ‘parameter addition’, simply sums the relevant DVH metrics from the initial and re-planned distributions without spatially registering the CTs. The second, termed ‘dose accumulation’, uses a validated hybrid contour/intensity-based DIR algorithm to deform initial CT and dose distribution onto the repeat CT and re-planning dose distribution. DVH metrics from the summed distribution on the repeat CT are then calculated. Dose differences for organs-at-risk between parameter addition and dose accumulation ≥100 cGy were assumed to be clinically relevant. To elucidate whether relevant differences were due to registration accuracy or contouring variability between CTs, the analysis was repeated using contours on the first CT and the same contours deformed to the repeat CT with DIR. ResultsFor all patients, high overall DIR accuracy was verified visually (qualitatively) and numerically (quantitatively) using image similarity and contour-based metrics. All head-and-neck and lung patients, and one sarcoma patient (11 of 15 total) had dose differences between parameter addition and dose accumulation ≥100 cGy, with absolute mean differences of 160 cGy (range 101–436 cGy) seen in 41 of 205 total DVH criteria. In 22 of these 41 criteria, these differences were attributed to contouring variability between CTs. After correcting for contouring variations using DIR, the mean absolute differences in 7 of these 22 criteria with a relevant result (across 6 patients) was 146 cGy (range 100–502 cGy). In only 4 DVH criteria, the DIR mapped contours had higher variations than the original contours. One lung patient had a DVH criteria exceeding the clinical dose constraint by 125 cGy with parameter addition, and with accurate DIR and dose accumulation, the criteria was actually 97 cGy lower than the constraint. ConclusionsThe use of DIR to generate total planned dose records revealed substantial dose differences in most cases compared to commonly used clinical methods (i.e. parameter addition), and altered the planned acceptance criteria in a minority. DIR is recommended to be used for future adaptive re-plans to generate total planned dose records and facilitate accurate re-contouring. More accurate dose records may also improve our understanding of clinical outcomes.

Layout Cross-Browser Failure Classification for Mobile Responsive Design Web Applications: Combining Classification Models Using Feature Selection

Cross-browser incompatibilities (XBIs) are defined as inconsistencies that can be observed in Web applications when they are rendered in a specific browser compared to others. These inconsistencies are associated with differences in the way each browser implements its capabilities and renders Web applications. The inconsistencies range from minor layout differences to lack of core functionalities of Web applications when rendered in specific browsers. The state of the art proposes different approaches for detecting XBIs and many of them are based on classification models, using features extracted from the document object model (DOM) structure (DOM-based approaches) and screenshots (computer vision approaches) of Web applications. To the best of our knowledge, a comparison between DOM-based and computer vision classification models has not yet been reported in the literature, and a combination between both approaches could possibly lead to increased accuracy of classification models. In this article, we extend the use of these classification models for detecting layout XBIs in responsive design Web applications, rendered on different browser viewport widths and devices (iPhone 12 mini, iPhone 12, iPhone 12 Pro Max, and Pixel XL). We investigate the use of state-of-the-art classification models (Browserbite, CrossCheck, and our previous work) for detecting layout cross-browser failures, which consist of layout XBIs that negatively affect the layout of responsive design Web applications. Furthermore, we propose an enhanced classification model that combines features from different state-of-the-art classification models (DOM based and computer vision) using feature selection. We built two datasets for evaluating the efficacy of classification models in separately detecting external and internal layout failures using data from 72 responsive design Web applications. The proposed classification model reported the highest F1-score for detecting external layout failures (0.65) and internal layout failures (0.35), and these results reported significant differences compared to Browserbite and CrossCheck classification models. Nevertheless, the experiment showed a lower accuracy in the classification of internal layout failures and suggests the use of other image similarity metrics or deep learning models for increasing the efficacy of classification models.

Evaluation of synthetically generated computed tomography for use in transcranial focused ultrasound procedures.

Transcranial focused ultrasound (tFUS) is a therapeutic ultrasound method that focuses sound through the skull to a small region noninvasively and often under magnetic resonance imaging (MRI) guidance. CT imaging is used to estimate the acoustic properties that vary between individual skulls to enable effective focusing during tFUS procedures, exposing patients to potentially harmful radiation. A method to estimate acoustic parameters in the skull without the need for CT is desirable. We synthesized CT images from routinely acquired T1-weighted MRI using a 3D patch-based conditional generative adversarial network and evaluated the performance of synthesized CT (sCT) images for treatment planning with tFUS. We compared the performance of sCT with real CT (rCT) images for tFUS planning using Kranion and simulations using the acoustic toolbox, k-Wave. Simulations were performed for 3 tFUS scenarios: (1)no aberration correction, (2)correction with phases calculated from Kranion, and (3)phase shifts calculated from time reversal. From Kranion, the skull density ratio, skull thickness, and number of active elements between rCT and sCT had Pearson's correlation coefficients of 0.94, 0.92, and 0.98, respectively. Among 20 targets, differences in simulated peak pressure between rCT and sCT were largest without phase correction () and smallest with Kranion phases (). The distance between peak focal locations between rCT and sCT was for all simulation cases. Real and synthetically generated skulls had comparable image similarity, skull measurements, and acoustic simulation metrics. Our work demonstrated similar results for 10 testing cases comparing MR-sCTs and rCTs for tFUS planning. Source code and a docker image with the trained model are available at https://github.com/han-liu/SynCT_TcMRgFUS.

Deep-Learning-Aided Evaluation of Spondylolysis Imaged with Ultrashort Echo Time Magnetic Resonance Imaging.

Isthmic spondylolysis results in fracture of pars interarticularis of the lumbar spine, found in as many as half of adolescent athletes with persistent low back pain. While computed tomography (CT) is the gold standard for the diagnosis of spondylolysis, the use of ionizing radiation near reproductive organs in young subjects is undesirable. While magnetic resonance imaging (MRI) is preferable, it has lowered sensitivity for detecting the condition. Recently, it has been shown that ultrashort echo time (UTE) MRI can provide markedly improved bone contrast compared to conventional MRI. To take UTE MRI further, we developed supervised deep learning tools to generate (1) CT-like images and (2) saliency maps of fracture probability from UTE MRI, using ex vivo preparation of cadaveric spines. We further compared quantitative metrics of the contrast-to-noise ratio (CNR), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) between UTE MRI (inverted to make the appearance similar to CT) and CT and between CT-like images and CT. Qualitative results demonstrated the feasibility of successfully generating CT-like images from UTE MRI to provide easier interpretability for bone fractures thanks to improved image contrast and CNR. Quantitatively, the mean CNR of bone against defect-filled tissue was 35, 97, and 146 for UTE MRI, CT-like, and CT images, respectively, being significantly higher for CT-like than UTE MRI images. For the image similarity metrics using the CT image as the reference, CT-like images provided a significantly lower mean MSE (0.038 vs. 0.0528), higher mean PSNR (28.6 vs. 16.5), and higher SSIM (0.73 vs. 0.68) compared to UTE MRI images. Additionally, the saliency maps enabled quick detection of the location with probable pars fracture by providing visual cues to the reader. This proof-of-concept study is limited to the data from ex vivo samples, and additional work in human subjects with spondylolysis would be necessary to refine the models for clinical use. Nonetheless, this study shows that the utilization of UTE MRI and deep learning tools could be highly useful for the evaluation of isthmic spondylolysis.

Rethinking two-dimensional camera motion estimation assessment for digital video stabilization: A camera motion field-based metric

Digital video stabilization aims to remove camera motion jitters through software implementation. The first step of the classical video stabilization methodology is called camera motion estimation, which is usually performed using only RGB frames of the unstable video. Despite recent advances in camera motion estimation strategies, methods classified as two-dimensional are still not properly evaluated, even though it is well known that motion estimation is a crucial step for classical approaches to video stabilization. The main purpose of this work is to draw attention to two-dimensional camera motion estimation assessment and reinforce its importance on video stabilization progress. We proposed a new approach to perform this evaluation using camera motion fields in a pixel-by-pixel comparison and demonstrated through experimental results that our metrics are reliable for diverse scenarios comparing them to image similarity metrics. In addition, we showed and analyzed the results of our metrics for a global and a local method of camera motion estimation. We believe that our assessment and study presented in this work is an important starting point for a more rigorous analysis of this task. In addition, this can be a foundation for coming 2D camera motion estimation methods based on deep learning.

A deep learning method for replicate-based analysis of chromosome conformation contacts using Siamese neural networks

The organisation of the genome in nuclear space is an important frontier of biology. Chromosome conformation capture methods such as Hi-C and Micro-C produce genome-wide chromatin contact maps that provide rich data containing quantitative and qualitative information about genome architecture. Most conventional approaches to genome-wide chromosome conformation capture data are limited to the analysis of pre-defined features, and may therefore miss important biological information. One constraint is that biologically important features can be masked by high levels of technical noise in the data. Here we introduce a replicate-based method for deep learning from chromatin conformation contact maps. Using a Siamese network configuration our approach learns to distinguish technical noise from biological variation and outperforms image similarity metrics across a range of biological systems. The features extracted from Hi-C maps after perturbation of cohesin and CTCF reflect the distinct biological functions of cohesin and CTCF in the formation of domains and boundaries, respectively. The learnt distance metrics are biologically meaningful, as they mirror the density of cohesin and CTCF binding. These properties make our method a powerful tool for the exploration of chromosome conformation capture data, such as Hi-C capture Hi-C, and Micro-C.

Image Perceptual Similarity Metrics for the Assessment of Basal Cell Carcinoma.

Efficient management of basal cell carcinomas (BCC) requires reliable assessments of both tumors and post-treatment scars. We aimed to estimate image similarity metrics that account for BCC's perceptual color and texture deviation from perilesional skin. In total, 176 clinical photographs of BCC were assessed by six physicians using a visual deviation scale. Internal consistency and inter-rater agreement were estimated using Cronbach's α, weighted Gwet's AC2, and quadratic Cohen's kappa. The mean visual scores were used to validate a range of similarity metrics employing different color spaces, distances, and image embeddings from a pre-trained VGG16 neural network. The calculated similarities were transformed into discrete values using ordinal logistic regression models. The Bray-Curtis distance in the YIQ color model and rectified embeddings from the 'fc6' layer minimized the mean squared error and demonstrated strong performance in representing perceptual similarities. Box plot analysis and the Wilcoxon rank-sum test were used to visualize and compare the levels of agreement, conducted on a random validation round between the two groups: 'Human-System' and 'Human-Human.' The proposed metrics were comparable in terms of internal consistency and agreement with human raters. The findings suggest that the proposed metrics offer a robust and cost-effective approach to monitoring BCC treatment outcomes in clinical settings.

Synthetic computed tomography for low-field magnetic resonance-only radiotherapy in head-and-neck cancer using residual vision transformers.

Synthetic computed tomography (sCT) scans are necessary for dose calculation in magnetic resonance (MR)-only radiotherapy. While deep learning (DL) has shown remarkable performance in generating sCT scans from MR images, research has predominantly focused on high-field MR images. This study presents the first implementation of a DL model for sCT generation in head-and-neck (HN) cancer using low-field MR images. Specifically, the use of vision transformers (ViTs) was explored. The dataset consisted of 31 patients, resulting in 196 pairs of deformably-registered computed tomography (dCT) and MR scans. The latter were obtained using a balanced steady-state precession sequence on a 0.35T scanner. Residual ViTs were trained on 2D axial, sagittal, and coronal slices, respectively, and the final sCTs were generated by averaging the models' outputs. Different image similarity metrics, dose volume histogram (DVH) deviations, and gamma analyses were computed on the test set (n=6). The overlap between auto-contours on sCT scans and manual contours on MR images was evaluated for different organs-at-risk using the Dice score. The median [range] value of the test mean absolute error was 57 [37-74] HU. DVH deviations were below 1% for all structures. The median gamma passing rates exceeded 94% in the 2%/2mm analysis (threshold=90%). The median Dice scores were above 0.7 for all organs-at-risk. The clinical applicability of DL-based sCT generation from low-field MR images in HN cancer was proved. High sCT-dCT similarity and dose metric accuracy were achieved, and sCT suitability for organs-at-risk auto-delineation was shown.

Deep learning based synthetic CT from cone beam CT generation for abdominal paediatric radiotherapy

Objective. Adaptive radiotherapy workflows require images with the quality of computed tomography (CT) for re-calculation and re-optimisation of radiation doses. In this work we aim to improve the quality of on-board cone beam CT (CBCT) images for dose calculation using deep learning. Approach. We propose a novel framework for CBCT-to-CT synthesis using cycle-consistent Generative Adversarial Networks (cycleGANs). The framework was tailored for paediatric abdominal patients, a challenging application due to the inter-fractional variability in bowel filling and small patient numbers. We introduced to the networks the concept of global residuals only learning and modified the cycleGAN loss function to explicitly promote structural consistency between source and synthetic images. Finally, to compensate for the anatomical variability and address the difficulties in collecting large datasets in the paediatric population, we applied a smart 2D slice selection based on the common field-of-view (abdomen) to our imaging dataset. This acted as a weakly paired data approach that allowed us to take advantage of scans from patients treated for a variety of malignancies (thoracic-abdominal-pelvic) for training purposes. We first optimised the proposed framework and benchmarked its performance on a development dataset. Later, a comprehensive quantitative evaluation was performed on an unseen dataset, which included calculating global image similarity metrics, segmentation-based measures and proton therapy-specific metrics. Main results. We found improved performance for our proposed method, compared to a baseline cycleGAN implementation, on image-similarity metrics such as Mean Absolute Error calculated for a matched virtual CT (55.0 ± 16.6 HU proposed versus 58.9 ± 16.8 HU baseline). There was also a higher level of structural agreement for gastrointestinal gas between source and synthetic images measured using the dice similarity coefficient (0.872 ± 0.053 proposed versus 0.846 ± 0.052 baseline). Differences found in water-equivalent thickness metrics were also smaller for our method (3.3 ± 2.4% proposed versus 3.7 ± 2.8% baseline). Significance. Our findings indicate that our innovations to the cycleGAN framework improved the quality and structure consistency of the synthetic CTs generated.

Investigation of a water equivalent depth method for dosimetric accuracy evaluation of synthetic CT

PurposeTo provide a metric that reflects the dosimetric utility of the synthetic CT (sCT) and can be rapidly determined. MethodsRetrospective CT and atlas-based sCT of 62 (53 IMRT and 9 VMAT) prostate cancer patients were used. For image similarity measurements, the sCT and reference CT (rCT) were aligned using clinical registration parameters. Conventional image similarity metrics including the mean absolute error (MAE) and mean error (ME) were calculated. The water equivalent depth (WED) was automatically determined for each patient on the rCT and sCT as the distance from the skin surface to the treatment plan isocentre at 36 equidistant gantry angles, and the mean WED difference (ΔWED¯) between the two scans was calculated. Doses were calculated on each scan pair for the clinical plan in the treatment planning system. The image similarity measurements and ΔWED¯ were then compared to the isocentre dose difference (ΔDiso) between the two scans. ResultsWhile no particular relationship to dose was observed for the other image similarity metrics, the ME results showed a linear trend against ΔDiso with R2 = 0.6, and the 95 % prediction interval for ΔDiso between −1.2 and 1 %. The ΔWED¯ results showed an improved linear trend (R2 = 0.8) with a narrower 95 % prediction interval from −0.8 % to 0.8 %. ConclusionΔWED¯ highly correlates with ΔDiso for the reference and synthetic CT scans. This is easy to calculate automatically and does not require time-consuming dose calculations. Therefore, it can facilitate the process of developing and evaluating new sCT generation algorithms.

Synthetic computed tomography for low-field magnetic resonance-guided radiotherapy in the abdomen.

The requirement of computed tomography (CT) for radiotherapy planning may be bypassed by synthetic CT (sCT) generated from magnetic resonance (MR), which has recently led to the clinical introduction of MR-only radiotherapy for specific sites. Further developments are required for abdominal sCT, mostly due to the presence of mobile air pockets affecting the dose calculation. In this study we aimed to overcome this limitation for abdominal sCT at a low field (0.35 T) hybrid MR-Linac. A retrospective analysis was conducted enrolling 168 patients corresponding to 215 MR-CT pairs. After the exclusion criteria, 152 volumetric images were used to train the cycle-consistent generative adversarial network (CycleGAN) and 34 to test the sCT. Image similarity metrics and dose recalculation analysis were performed. The generated sCT faithfully reproduced the original CT and the location of the air pockets agreed with the MR scan. The dose calculation did not require manual bulk density overrides and the mean deviations of the dose-volume histogram dosimetric points were within 1 % of the CT, without any outlier above 2 %. The mean gamma passing rates were above 99 % for the 2 %/ 2 mm analysis and no cases below 95 % were observed. This study presented the implementation of CycleGAN to perform sCT generation in the abdominal region for a low field hybrid MR-Linac. The sCT was shown to correctly allocate the electron density for the mobile air pockets and the dosimetric analysis demonstrated the potential for future implementation of MR-only radiotherapy in the abdomen.

Conditional GANs based system for fibrosis detection and quantification in Hematoxylin and Eosin whole slide images.

Assessing the degree of liver fibrosis is fundamental for the management of patients with chronic liver disease, in liver transplants procedures, and in general liver disease research. The fibrosis stage is best assessed by histopathologic evaluation, and Masson's Trichrome stain (MT) is the stain of choice for this task in many laboratories around the world. However, the most used stain in histopathology is Hematoxylin Eosin (HE) which is cheaper, has a faster turn-around time and is the primary stain routinely used for evaluation of liver specimens. In this paper, we propose a novel digital pathology system that accurately detects and quantifies the footprint of fibrous tissue in HE whole slide images (WSI). The proposed system produces virtual MT images from HE using a deep learning model that learns deep texture patterns associated with collagen fibers. The training pipeline is based on conditional generative adversarial networks (cGAN), which can achieve accurate pixel-level transformation. Our comprehensive training pipeline features an automatic WSI registration algorithm, which qualifies the HE/MT training slides for the cGAN model. Using liver specimens collected during liver transplantation procedures, we conducted a range of experiments to evaluate the detected footprint of selected anatomical features. Our evaluation includes both image similarity and semantic segmentation metrics. The proposed system achieved enhanced results in the experiments with significant improvement over the state-of-the-art CycleGAN learning style, and over direct prediction of fibrosis in HE without having the virtual MT step.

Fusion of Different Image Sources for Improved Monitoring of Agricultural Plots.

In the Valencian Community, the applications of precision agriculture in multiannual woody crops with high added value (fruit trees, olive trees, almond trees, vineyards, etc.) are of priority interest. In these plots, canopies do not fully cover the soil and the planting frames are incompatible with the Resolution of Sentinel 2. The present work proposes a procedure for the fusion of images with different temporal and spatial resolutions and with different degrees of spectral quality. It uses images from the Sentinel 2 mission (low resolution, high spectral quality, high temporal resolution), orthophotos (high resolution, low temporal resolution) and images obtained with drones (very high spatial resolution, low temporal resolution). The procedure is applied to generate time series of synthetic RGI images (red, green, infrared) with the same high resolution of orthophotos and drone images, in which gray levels are reassigned from the combination of their own RGI bands and the values of the B3, B4 and B8 bands of Sentinel 2. Two practical examples of application are also described. The first shows the NDVI images that can be generated after the process of merging two RGI Sentinel 2 images obtained on two specific dates. It is observed how, after the merging, different NDVI values can be assigned to the soil and vegetation, which allows them to be distinguished (contrary to the original Sentinel 2 images). The second example shows how graphs can be generated to describe the evolution throughout the vegetative cycle of the estimated values of three spectral indices (NDVI, GNDVI, GCI) on a point in the image corresponding to soil and on another assigned to vegetation. The robustness of the proposed algorithm has been validated by using image similarity metrics.

Performance comparison of quantitative metrics for analysis of in vivo Cherenkov imaging incident detection during radiotherapy.

Examine the responses of multiple image similarity metrics to detect patient positioning errors in radiotherapy observed through Cherenkov imaging, which may be used to optimize automated incident detection. An anthropomorphic phantom mimicking patient vasculature, a biological marker seen in Cherenkov images, was simulated for a breast radiotherapy treatment. The phantom was systematically shifted in each translational direction, and Cherenkov images were captured during treatment delivery at each step. The responses of mutual information (MI) and the γ passing rate (%GP) were compared to that of existing field-shape matching image metrics, the Dice coefficient, and mean distance to conformity (MDC). Patient images containing other incidents were analyzed to verify the best detection algorithm for different incident types. Positional shifts in all directions were registered by both MI and %GP, degrading monotonically as the shifts increased. Shifts in intensity, which may result from erythema or bolus-tissue air gaps, were detected most by %GP. However, neither metric detected beam-shape misalignment, such as that caused by dose to unintended areas, as well as currently employed metrics (Dice and MDC). This study indicates that different radiotherapy incidents may be detected by comparing both inter- and intrafractional Cherenkov images with a corresponding image similarity metric, varying with the type of incident. Future work will involve determining appropriate thresholds per metric for automatic flagging. Classifying different algorithms for the detection of various radiotherapy incidents allows for the development of an automatic flagging system, eliminating the burden of manual review of Cherenkov images.

Skull phantom-based methodology to validate MRI co-registration accuracy for Gamma Knife radiosurgery.

Target localization, for stereotactic radiosurgery (SRS) treatment with Gamma Knife, has become increasingly reliant on the co-registration between the planning MRI and the stereotactic cone-beam computed tomography (CBCT). Validating image registration between modalities would be particularly beneficial when considering the emergence of novel functional and metabolic MRI pulse sequences for target delineation. This study aimed to develop a phantom-based methodology to quantitatively compare the co-registration accuracy of the standard clinical imaging protocol to a representative MRI sequence that was likely to fail co-registration. The comparative methodology presented in this study may serve as a useful tool to evaluate the clinical translatability of novel MRI sequences. A realistic human skull phantom with fiducial marker columns was designed and manufactured to fit into a typical MRI head coil and the Gamma Knife patient positioning system. A series of "optimized" 3D MRI sequences-T1 -weighted Dixon, T1 -weighted fast field echo (FFE), and T2 -weighted fluid-attenuated inversion recovery (FLAIR)-were acquired and co-registered to the CBCT. The same sequences were "compromised" by reconstructing without geometric distortion correction and re-collecting with lower signal-to-noise-ratio (SNR) to simulate a novel MRI sequence with poor co-registration accuracy. Image similarity metrics-structural similarity (SSIM) index, mean squared error (MSE), and peak SNR (PSNR)-were used to quantitatively compare the co-registration of the optimized and compromised MR images. The ground truth fiducial positions were compared to positions measured from each optimized image volume revealing a maximum median geometric uncertainty of 0.39mm (LR), 0.92mm (AP), and 0.13mm (SI) between the CT and CBCT, 0.60mm (LR), 0.36mm (AP), and 0.07mm (SI) between the CT and T1 -weighted Dixon, 0.42mm (LR), 0.23mm (AP), and 0.08mm (SI) between the CT and T1 -weighted FFE, and 0.45mm (LR), 0.19mm (AP), and 1.04mm (SI) between the CT and T2 -weighted FLAIR. Qualitatively, pairs of optimized and compromised image slices were compared using a fusion image where separable colors were used to differentiate between images. Quantitatively, MSE was the most predictive and SSIM the second most predictive metric for evaluating co-registration similarity. A clinically relevant threshold of MSE, SSIM, and/or PSNR may be defined beyond which point an MRI sequence should be rejected for target delineation based on its dissimilarity to an optimized sequence co-registration. All dissimilarity thresholds calculated using correlation coefficients with in-plane geometric uncertainty would need to be defined on a sequence-by-sequence basis and validated with patient data. This study utilized a realistic skull phantom and image similarity metrics to develop a methodology capable of quantitatively assessing whether a modern research-based MRI sequence can be co-registered to the Gamma Knife CBCT with equal or less than equal accuracy when compared to a clinically accepted protocol.

Imperceptible Image Steganography Using Symmetry-Adapted Deep Learning Techniques

Digital image steganography is the process of embedding information within a cover image in a secure, imperceptible, and recoverable way. This research extends a symmetry-adapted deep-learning approach to identify hidden patterns of images using two-dimensional convolutional neural networks (CNN). The proposed method (SteGuz) uses three CNNs to implement the different phases of the steganography process on digital image covers. SteGuz introduced a gain function, based on several image similarity metrics, to maximize the imperceptibility of the hiding process. Using 10 different pairs of cover-secret images, the performance of the proposed method was measured in terms of standard metrics such as peak signal to noise ratio (PSNR) and structured similarity index measurement (SSIM). The results showed that the proposed methodology outperformed the original method in terms of both imperceptibility and recoverability. In addition, when compared with some existing methods, SteGuz proved the outstanding performance of achieving a very high PSNR value while maintaining high accuracy of the extracted image.