Image Pairs Research Articles

A Multi-aperture Coaxial Projector Balancing Shadow Suppression and Deblurring.

This paper proposes a projection system that optically removes the cast shadow in projection mapping. Specifically, we realize the large-aperture (LA) projection using a large-format Fresnel lens to suppress cast shadows by condensing the projection light from a wide viewing angle. However, the resolution and contrast of the projected results are significantly degraded by defocus blur, veiling glare, and stray light caused by the aberration of an LA Fresnel lens. To solve the technical problems, we employ two different approaches: optical and digital image processing methods. First, we introduce a residual projector with a typical aperture lens on the same optical axis as the LA projector, projecting the residual (i.e., high-frequency) components attenuated in the LA projection. These projectors play different roles in shadow suppression and blur compensation, both achieved by projecting simultaneously. Secondly, we optimize the pair of projection images that can balance the shadow suppression and deblurring performance of our projection system. We implemented a proof-of-concept prototype and validated the above-mentioned techniques through projection experiments and a user study.

Multi-vendor validation of automated left ventricular analysis from 3D echocardiography using artificial intelligence

Abstract Background Accurate echocardiographic assessment of left ventricular (LV) function is dependent on both image acquisition and analysis. Although 3D echocardiography (3DE) is increasingly recommended for volume quantification, manually derived volumes are limited by operator subjectivity, variable image quality, and the lengthy duration of analysis. Consequently, several automated approaches based on artificial intelligence (AI) have been proposed to segment the LV cavity from 3DE and have demonstrated better accuracy and reproducibility compared to human experts. While the utility of AI models in the clinic depends on their ability to generalise to different populations and ultrasound vendors, the external validation of developed models remains limited. Purpose This study evaluates the ability of an existing AI model trained using data from a single vendor to accurately estimate routine 3DE LV indices in an independent external dataset acquired using equipment from a different vendor. Methods A previously published and internally validated deep learning model for segmentation of the LV cavity and myocardium from 3DE was used to automatically derive LV indices including end-diastolic volume (EDV), end-systolic volume (ESV), LV mass, and ejection fraction (EF) [1]. A second independent dataset consisting of paired 3DE and CMR images from 65 marathon runners from a different centre (using equipment from a different vendor) was subsequently used for external validation. Images from both datasets were subjectively scored from 1 (poor) to 5 (excellent) as a measure of image quality. Agreement with CMR as the reference was assessed using a two-way mixed effects intraclass correlation coefficient (ICC), supplemented by Bland-Altman analysis, to determine the model bias (between 3DE and CMR) and 95% limits of agreement (LOA). To assess whether performance could be improved by inclusion of multi-vendor data during training, the initial model was fine-tuned using a small subset (n=9) of images from the second centre. Results The mean image quality score from the second dataset was 2.5 (± 1.1), which was significantly lower than the mean of 3.4 (± 1.0) from the first dataset. Despite vendor and image quality differences, the initial model produced good agreement with CMR in EDV, ESV, and LV mass but only poor agreement in EF (Fig 1). Fine tuning greatly improved agreement in EF with reduced bias and narrower 95% LOA (Fig 2). Conclusion With the addition of a small number of cases for fine-tuning, AI models for automated LV analysis can achieve good performance analysing 3DE images acquired from a different vendor and population. Advancing model generalisability through more sophisticated data augmentation strategies may enable truly vendor-agnostic automated LV analysis from 3DE.

The correlation between cardiac magnetic resonance imaging and echocardiography in patients with severe ischaemic cardiomyopathy

Abstract Background Left ventricular ejection fraction is the single metric that guides therapeutic decision-making, risk stratification and prognosis in patients with severely impaired systolic function. Although cardiac magnetic resonance imaging (CMR) is the current reference standard for quantification of ventricular volume and function, echocardiography is used more often as it is more widely available and less expensive.(1) Aims To evaluate the correlation between CMR and echocardiography in quantifying left ventricular (LV) volume and function and to determine whether these measurements are associated with clinical outcomes in patients with severe ischemic cardiomyopathy (ICM). Methods Participants recruited to the REVIVED-BCIS2 trial who had undergone baseline CMR or echocardiography were included in this analysis. All scans were analyzed in blinded core laboratories as previously reported.(2,3) All volumes were indexed to body surface area. Spearman correlation, Bland Altman analysis and coverage probability methods were performed to assess the correlation between the two modalities. A Cox proportional hazards model was used to assess the association between imaging metrics and the primary outcome, a composite of all-cause death and aborted sudden death. A sensitivity analysis was conducted a priori, restricted to patients undergoing both scans within 60 days of each other. Results A total of 373 participants with paired imaging data were included: mean age 69±9 years, 87% male, BMI 28±5. A moderate correlation was detected for volume measurements between the modalities (end diastolic volume index (EDVI) r=0.66 end systolic volume index (ESVI) r=0.68), with a weaker correlation for LV ejection fraction (EF) (r=0.41). Bland-Altman plots showed moderate agreement in LVEF measurements (Table 1). The sensitivity analysis included 199 patients, and showed similar results (EDVI r=0.72, ESVI r=0.73 LVEF r=0.43, all p<0.000). Only 7.5% of the paired LVEF measurements were within 5% of each other. 30% of participants were misclassified by echocardiography as having an LVEF>35%. (Figure 1) CMR ESVI was associated with the occurrence of the primary outcome (HR 1.05 per 10ml increment, 95% CI 1.00-1.10) however echo ESVI (HR 1.05 per 10ml increment 95% CI 0.99 – 1.10) and all other imaging metrics were not. Conclusion Left ventricular volumes and ejection fraction on CMR and echocardiography were only modestly correlated in this population with severe ICM. When using the binary EF threshold of 35% that is commonly used to select patients of implantable defibrillator therapy, almost a third of patients at risk would have been missed by echocardiography alone. The additional insights provided by CMR, including scar burden, further support preferential use of this modality in the assessment such patients.

Interpretable diagnosis of breast lesions in ultrasound imaging using deep multi-stage reasoning

Objective.Ultrasound is the primary screening test for breast cancer. However, providing an interpretable auxiliary diagnosis of breast lesions is a challenging task. This study aims to develop an interpretable auxiliary diagnostic method to enhance usability in human-machine collaborative diagnosis.Approach.To address this issue, this study proposes the deep multi-stage reasoning method (DMSRM), which provides individual and overall breast imaging-reporting and data system (BI-RADS) assessment categories for breast lesions. In the first stage of the DMSRM, the individual BI-RADS assessment network (IBRANet) is designed to capture lesion features from breast ultrasound images. IBRANet performs individual BI-RADS assessments of breast lesions using ultrasound images, focusing on specific features such as margin, contour, echogenicity, calcification, and vascularity. In the second stage, evidence reasoning (ER) is employed to achieve uncertain information fusion and reach an overall BI-RADS assessment of the breast lesions.Main results.To evaluate the performance of DMSRM at each stage, two test sets are utilized: the first for individual BI-RADS assessment, containing 4322 ultrasound images; the second for overall BI-RADS assessment, containing 175 sets of ultrasound image pairs. In the individual BI-RADS assessment of margin, contour, echogenicity, calcification, and vascularity, IBRANet achieves accuracies of 0.9491, 0.9466, 0.9293, 0.9234, and 0.9625, respectively. In the overall BI-RADS assessment of lesions, the ER achieves an accuracy of 0.8502. Compared to independent diagnosis, the human-machine collaborative diagnosis results of three radiologists show increases in positive predictive value by 0.0158, 0.0427, and 0.0401, in sensitivity by 0.0400, 0.0600 and 0.0434, and in area under the curve by 0.0344, 0.0468, and 0.0255.Significance.This study proposes a DMSRM that enhances the transparency of the diagnostic reasoning process. Results indicate that DMSRM exhibits robust BI-RADS assessment capabilities and provides an interpretable reasoning process that better suits clinical needs.

Dyadic Partition-Based Training Schemes for TV/TGV Denoising

AbstractDue to their ability to handle discontinuous images while having a well-understood behavior, regularizations with total variation (TV) and total generalized variation (TGV) are some of the best-known methods in image denoising. However, like other variational models including a fidelity term, they crucially depend on the choice of their tuning parameters. A remedy is to choose these automatically through multilevel approaches, for example by optimizing performance on noisy/clean image pairs. In this work, we consider such methods with space-dependent parameters which are piecewise constant on dyadic grids, with the grid itself being part of the minimization. We prove existence of minimizers for fixed discontinuous parameters under mild assumptions on the data, which lead to existence of finite optimal partitions. We further establish that these assumptions are equivalent to the commonly used box constraints on the parameters. On the numerical side, we consider a simple subdivision scheme for optimal partitions built on top of any other bilevel optimization method for scalar parameters, and demonstrate its improved performance on some representative test images when compared with constant optimized parameters.

Vegetative Index Intercalibration Between PlanetScope and Sentinel-2 Through a SkySat Classification in the Context of “Riserva San Massimo” Rice Farm in Northern Italy

Rice farming in Italy accounts for about 50% of the EU’s rice area and production. Precision agriculture has entered the scene to enhance sustainability, cut pollution, and ensure food security. Various studies have used remote sensing tools like satellites and drones for multispectral imaging. While Sentinel-2 is highly regarded for precision agriculture, it falls short for specific applications, like at the “Riserva San Massimo” (Gropello Cairoli, Lombardia, Northern Italy) rice farm, where irregularly shaped crops need higher resolution and frequent revisits to deal with cloud cover. A prior study that compared Sentinel-2 and the higher-resolution PlanetScope constellation for vegetative indices found a seasonal miscalibration in the Normalized Difference Vegetation Index (NDVI) and in the Normalized Difference Red Edge Index (NDRE). Dr. Agr. G.N. Rognoni, a seasoned agronomist working with this farm, stresses the importance of studying the radiometric intercalibration between the PlanetScope and Sentinel-2 vegetative indices to leverage the knowledge gained from Sentinel-2 for him to apply variable rate application (VRA). A high-resolution SkySat image, taken almost simultaneously with a pair of Sentinel-2 and PlanetScope images, offered a chance to examine if the irregular distribution of vegetation and barren land within rice fields might be a factor in the observed miscalibration. Using an unsupervised pixel-based image classification technique on SkySat imagery, it is feasible to split rice into two subclasses and intercalibrate them separately. The results indicated that combining histograms and agronomists’ expertise could confirm SkySat classification. Moreover, the uneven spatial distribution of rice does not affect the seasonal miscalibration object of past studies, which can be adjusted using the methods described here, even with images taken four days apart: the first method emphasizes accuracy using linear regression, histogram shifting, and histogram matching; whereas the second method is faster and utilizes only histogram matching.

Image Stitching of Low-Resolution Retinography Using Fundus Blur Filter and Homography Convolutional Neural Network

Great advances in stitching high-quality retinal images have been made in recent years. On the other hand, very few studies have been carried out on low-resolution retinal imaging. This work investigates the challenges of low-resolution retinal images obtained by the D-EYE smartphone-based fundus camera. The proposed method uses homography estimation to register and stitch low-quality retinal images into a cohesive mosaic. First, a Siamese neural network extracts features from a pair of images, after which the correlation of their feature maps is computed. This correlation map is fed through four independent CNNs to estimate the homography parameters, each specializing in different corner coordinates. Our model was trained on a synthetic dataset generated from the Microsoft Common Objects in Context (MSCOCO) dataset; this work added an important data augmentation phase to improve the quality of the model. Then, the same is evaluated on the FIRE retina and D-EYE datasets for performance measurement using the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The obtained results are promising: the average PSNR was 26.14 dB, with an SSIM of 0.96 on the D-EYE dataset. Compared to the method that uses a single neural network for homography calculations, our approach improves the PSNR by 7.96 dB and achieves a 7.86% higher SSIM score.

Medical Image Fusion for Multiple Diseases Features Enhancement

ABSTRACTThroughout the past 20 years, medical imaging has found extensive application in clinical diagnosis. Doctors may find it difficult to diagnose diseases using only one imaging modality. The main objective of multimodal medical image fusion (MMIF) is to improve both the accuracy and quality of clinical assessments by extracting structural and spectral information from source images. This study proposes a novel MMIF method to assist doctors and postoperations such as image segmentation, classification, and further surgical procedures. Initially, the intensity‐hue‐saturation (IHS) model is utilized to decompose the positron emission tomography (PET)/single photon emission computed tomography (SPECT) image, followed by a hue‐angle mapping method to discriminate high‐ and low‐activity regions in the PET images. Then, a proposed structure feature adjustment (SFA) mechanism is used as a fusion strategy for high‐ and low‐activity regions to obtain structural and anatomical details with minimum color distortion. In the second step, a new multi‐discriminator generative adversarial network (MDcGAN) approach is proposed for obtaining the final fused image. The qualitative and quantitative results demonstrate that the proposed method is superior to existing MMIF methods in preserving the structural, anatomical, and functional details of the PET/SPECT images. Through our assessment, involving visual analysis and subsequent verification using statistical metrics, it becomes evident that color changes contribute substantial visual information to the fusion of PET and MR images. The quantitative outcomes demonstrate that, in the majority of cases, the proposed algorithm consistently outperformed other methods. Yet, in a few instances, it achieved the second‐highest results. The validity of the proposed method was confirmed using diverse modalities, encompassing a total of 1012 image pairs.

Diffusion probabilistic priors for zero-shot low-dose CT image denoising.

Denoising low-dose computed tomography (CT) images is a critical task in medical image computing. Supervised deep learning-based approaches have made significant advancements in this area in recent years. However, these methods typically require pairs of low-dose and normal-dose CT images for training, which are challenging to obtain in clinical settings. Existing unsupervised deep learning-based methods often require training with a large number of low-dose CT images or rely on specially designed data acquisition processes to obtain training data. To address these limitations, we propose a novel unsupervised method that only utilizes normal-dose CT images during training, enabling zero-shot denoising of low-dose CT images. Our method leverages the diffusion model, a powerful generative model. We begin by training a cascaded unconditional diffusion model capable of generating high-quality normal-dose CT images from low-resolution to high-resolution. The cascaded architecture makes the training of high-resolution diffusion models more feasible. Subsequently, we introduce low-dose CT images into the reverse process of the diffusion model as likelihood, combined with the priors provided by the diffusion model and iteratively solve multiple maximum a posteriori (MAP) problems to achieve denoising. Additionally, we propose methods to adaptively adjust the coefficients that balance the likelihood and prior in MAP estimations, allowing for adaptation to different noise levels in low-dose CT images. We test our method on low-dose CT datasets of different regions with varying dose levels. The results demonstrate that our method outperforms the state-of-the-art unsupervised method and surpasses several supervised deep learning-based methods. Our method achieves PSNR of 45.02 and 35.35dB on the abdomen CT dataset and the chest CT dataset, respectively, surpassing the best unsupervised algorithm Noise2Sim in the comparative methods by 0.39 and 0.85dB, respectively. We propose a novel low-dose CT image denoising method based on diffusion model. Our proposed method only requires normal-dose CT images as training data, greatly alleviating the data scarcity issue faced by most deep learning-based methods. At the same time, as an unsupervised algorithm, our method achieves very good qualitative and quantitative results. The Codes are available in https://github.com/DeepXuan/Dn-Dp.

Sibling Discrimination Using Linear Fusion on Deep Learning Face Recognition Models

Facial recognition technology has revolutionised human identification, providing a non-invasive alternative to traditional biometric methods like signatures and voice recognition. The integration of deep learning has significantly enhanced the accuracy and adaptability of these systems, now widely used in criminal identification, access control, and security. Initial research focused on recognising full-frontal facial features, but recent advancements have tackled the challenge of identifying partially visible faces, a scenario that often reduces recognition accuracy. This study aims to identify siblings based on facial features, particularly in cases where only partial features like eyes, nose, or mouth are visible. Utilising advanced deep learning models such as VGG19, VGG16, VGGFace, and FaceNet, the research introduces a framework to differentiate between sibling images effectively. To boost discrimination accuracy, the framework employs a linear fusion technique that merges insights from all the models used. The methodology involves preprocessing image pairs, extracting embeddings with pre-trained models, and integrating information through linear fusion. Evaluation metrics, including confusion matrix analysis, assess the framework's robustness and precision. Custom datasets of cropped sibling facial areas form the experimental basis, testing the models under various conditions like different facial poses and cropped regions. Model selection emphasises accuracy and extensive training on large datasets to ensure reliable performance in distinguishing subtle facial differences. Experimental results show that combining multiple models' outputs using linear fusion improves the accuracy and realism of sibling discrimination based on facial features. Findings indicate a minimum accuracy of 96% across different facial regions. Although this is slightly lower than the accuracy achieved by a single model like VGG16 with full-frontal poses, the fusion approach provides a more realistic outcome by incorporating insights from all four models. This underscores the potential of advanced deep learning techniques in enhancing facial recognition systems for practical applications.

Optimal Illumination Distance Metrics for Person Re-identification in Complex Lighting Conditions

Person Re-identification is extensively applied in public security and surveillance. However, environmental factors like time and location often lead to varying lighting conditions in captured pedestrian images, significantly impacting identification accuracy. Current approaches mitigate this issue through lighting transformation techniques, aiming to normalize images to a standard lighting condition for consistent person re-identification results. Yet, these methods overlook the fact that different content may hold distinct identification values under diverse lighting conditions. To address this, we conducted an analysis on the identification distance between images of the same or different pedestrians under predefined lighting conditions. From this analysis, we introduce the concept of optimal lighting: a condition where the distance between image pairs is minimized compared to other lighting scenarios. We propose utilizing this optimal lighting distance in the image retrieval process for final ranking. Our study, validated on synthetic datasets Market-IA and Duke-IA, demonstrates that optimal lighting is independent of image texture information. Each image pair exhibits a unique optimal lighting, yet consistently shows a minimum distance value.

Conditional probabilistic diffusion model driven synthetic radiogenomic applications in breast cancer.

This study addresses the heterogeneity of Breast Cancer (BC) by employing a Conditional Probabilistic Diffusion Model (CPDM) to synthesize Magnetic Resonance Images (MRIs) based on multi-omic data, including gene expression, copy number variation, and DNA methylation. The lack of paired medical images and genomics data in previous studies presented a challenge, which the CPDM aims to overcome. The well-trained CPDM successfully generated synthetic MRIs for 726 TCGA-BRCA patients, who lacked actual MRIs, using their multi-omic profiles. Evaluation metrics such as Frechet's Inception Distance (FID), Mean Square Error (MSE), and Structural Similarity Index Measure (SSIM) demonstrated the CPDM's effectiveness, with an FID of 2.02, an MSE of 0.02, and an SSIM of 0.59 based on the 15-fold cross-validation. The synthetic MRIs were used to predict clinical attributes, achieving an Area Under the Receiver-Operating-Characteristic curve (AUROC) of 0.82 and an Area Under the Precision-Recall Curve (AUPRC) of 0.84 for predicting ER+/HER2+ subtypes. Additionally, the MRIs served to accurately predicted BC patient survival with a Concordance-index (C-index) score of 0.88, outperforming other baseline models. This research demonstrates the potential of CPDMs in generating MRIs based on BC patients' genomic profiles, offering valuable insights for radiogenomic research and advancements in precision medicine. The study provides a novel approach to understanding BC heterogeneity for early detection and personalized treatment.

WTCL-Dehaze: Rethinking Real-World Image Dehazing via Wavelet Transform and Contrastive Learning

Images captured in hazy outdoor conditions often suffer from colour distortion, low contrast, and loss of detail, which impair high-level vision tasks. Single image dehazing is essential for applications such as autonomous driving and surveillance, with the aim of restoring image clarity. In this work, we propose WTCL-Dehaze an enhanced semi-supervised dehazing network that integrates Contrastive Loss and Discrete Wavelet Transform (DWT). We incorporate contrastive regularization to enhance feature representation by contrasting hazy and clear image pairs. Additionally, we utilize DWT for multi-scale feature extraction, effectively capturing high-frequency details and global structures. Our approach leverages both labelled and unlabelled data to mitigate the domain gap and improve generalization. The model is trained on a combination of synthetic and real-world datasets, ensuring robust performance across different scenarios. Extensive experiments demonstrate that our proposed algorithm achieves superior performance and improved robustness compared to state-of-the-art single image dehazing methods on both benchmark datasets and real-world images.

KBCNet: Independently keypoint learning for Small Object Semantic Correspondence

Establishing semantic correspondence between pairs of images within the same category or similar scenes is challenging due to large intra-class variations. In this paper, we introduce a novel problem termed ’Small Object Semantic Correspondence (SOSC).’ The challenge in SOSC arises from the close proximity of keypoints in small objects, leading to the fusion of their respective features, which complicates accurate keypoint prediction and correspondence identification. To address this issue, we propose the Keypoint Bounding box-centered Cropping (KBC) method, designed to increase the spatial separation between keypoints in small objects, thereby enabling the independent learning of these keypoints. The KBC method is seamlessly integrated into our proposed inference pipeline and can be easily incorporated into other methodologies, resulting in significant performance improvements. Furthermore, we introduce a novel framework called KBCNet, which serves as our baseline model. KBCNet includes a Cross-Scale Feature Alignment (CSFA) module and an efficient 4D convolutional decoder. The CSFA module aligns multi-scale features, enriching keypoint representations by integrating fine-grained details with deep semantic features. Meanwhile, the 4D convolutional decoder, based on efficient 4D convolution, ensures both high efficiency and rapid convergence. To empirically validate the effectiveness of our approach, we conducted extensive experiments on three widely used benchmarks: PF-PASCAL, PF-WILLOW, and SPair-71k. Our KBC method demonstrated a substantial performance improvement of 7.5% on the SPair-71k dataset, providing strong evidence of its efficacy.

Quantifying Uncertainty in Atmospheric Winds Retrieved from Optical Flow: Dependence on Weather Regime

Abstract This study explores the performance of a dense optical flow method in comparison to pattern-matching techniques for retrieving atmospheric motion vectors (AMVs) from water vapor images. Using high-resolution simulated datasets that represent various weather phenomena, we assess the performance of these methods across different weather regimes, time intervals, and pressure levels and quantify the uncertainties associated with retrieved winds. The optical flow algorithm consistently outperforms the feature matching approach. Notably, it produces wind speeds and AMVs that closely resemble the wind fields from the simulations, and unlike the feature matching algorithm, the optical flow algorithm exhibits consistent performance across different time intervals. In contrast, the feature matching approach yields vector fields that exhibit oversmoothing in certain areas and erratic behavior in others, while also producing less detailed, regionally constant speed maps. Furthermore, unlike feature matching, the optical flow method effectively calculates AMV near regions with missing data, generating a dense AMV field for every pixel in a pair of images. This superior performance and flexibility significantly influence the planning for future satellite missions aimed at retrieving atmospheric winds. As such, our work plays a critical role in determining the mission architecture and projected instrument performance for future atmospheric wind retrieval satellite missions. The study underscores the potential of the optical flow algorithm as a robust and efficient approach for atmospheric motion retrieval, thus contributing to advances in climate research and weather prediction. Significance Statement This research investigates the efficacy of two methods, optical flow and feature matching, for detecting atmospheric winds, referred to as atmospheric motion vectors, from satellite images of water vapor. Employing detailed simulated datasets that replicate real-world weather patterns, we found that optical flow consistently outperforms feature matching in various aspects. Notably, the optical flow method is not only more precise but also maintains its accuracy across different scenarios. These insights are critical for the design of future satellite missions focused on advancing our understanding of the atmosphere and enhancing weather predictions. This study contributes to advancements in climate research and supports improved weather forecasting, benefiting both scientific and societal needs.

Similarity learning networks uniquely identify individuals of four marine and terrestrial species

AbstractEstimating the size of animal populations plays an important role in evidence‐based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (Tursiops truncatus, 2008–2019) and harbor seals (Phoca vitulina, 2015–2019), a citizen science dataset of western leopard toads (Sclerophrys pantherina, unknown dates), and a publicly available repository of humpback whale images (Megaptera novaeangliae, unknown dates). For these species, our best‐performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross‐entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.

Artificial Intelligence Smartphone Application for Detection of Simulated Skin Changes: An In Vivo Pilot Study.

The development of artificial intelligence (AI) is rapidly expanding, showing promise in the dermatological field. Skin checks are a resource-heavy challenge that could potentially benefit from AI-tool assistance, particularly if provided in widely available AI solutions. A novel smartphone application(app)-based AI system, "SCAI," was developed and trained to recognize spots in paired images of skin, pursuing identification of new skin lesions. This pilot study aimed to investigate the feasibility of the SCAI-app to identify simulated skin changes in vivo. The study was conducted in a controlled setting with healthy volunteers and standardized, simulated skin changes (test spots), consisting of customized 3-mm adhesive spots in three colors (black, brown, and red). Each volunteer had a total of eight test spots adhered to four areas on back and legs. The SCAI-app collected smartphone- and template-guided standardized images before and after test spot application, using its backend AI algorithms to identify changes between the paired images. Twenty-four volunteers were included, amounting to a total of 192 test spots. Overall, the detection algorithms identified test spots with a sensitivity of 92.0% (CI: 88.1-95.9) and a specificity of 95.5% (CI: 95.0-96.0). The SCAI-app's positive predictive value was 38.0% (CI: 31.0-44.9), while the negative predictive value was 99.7% (CI: 99.0-100). This pilot study showed that SCAI-app could detect simulated skin changes in a controlled in vivo setting. The app's feasibility in a clinical setting with real-life skin lesions remains to be investigated, where the challenge with false positives in particular needs to be addressed.

Sub‐Pixel Displacement Estimation With Deep Learning: Application to Optical Satellite Images Containing Sharp Displacements

AbstractOptical image correlation is a powerful method for remotely constraining ground movement from optical satellite imagery related to natural disasters (e.g., earthquakes, volcanoes, landslides). This approach enables the characterization, and identification of the causal factors and mechanisms underlying such processes. By employing sub‐pixel correlation algorithms, one can obtain highly accurate (m‐to‐cm level) displacement fields at high spatial resolution (dm‐to‐cm) by comparing satellite images acquired before and after a period of movement. However, this method generally assumes a homogeneous translation of all pixels within a given correlation window, which will lead to biased estimates of ground displacement if the real case is not well represented by such a simplification, especially when resolving ground displacements next to sharp gradients in displacement, such as those found in the near‐field of earthquake surface ruptures. In this paper, we present an innovative deep learning method estimating sub‐pixel displacement maps from optical satellite images for the retrieval of ground displacement. From the generation of a realistic simulated database, comprising Landsat‐8 satellite image pairs containing simulated sub‐pixel shifts and sharp discontinuities, we develop a Convolutional Neural Network able to retrieve sub‐pixel displacements. The comparison to state‐of‐the‐art correlation methods shows that our pipeline significantly reduces by 32% the estimation bias around fault ruptures, leading to more accurate characterization of the near‐field strain in surface rupturing earthquakes. Application of our model to the 2019 Ridgecrest earthquake demonstrates the ability of our model to accurately and quickly resolve ground displacement using real satellite images. Code is made available at https://gricad‐gitlab.univ‐grenoble‐alpes.fr/montagtr/cnn4l‐discontinuities.

Supervised deep learning-based synthetic computed tomography from kilovoltage cone-beam computed tomography images for adaptive radiation therapy in head and neck cancer.

To generate and investigate a supervised deep learning algorithm for creating synthetic computed tomography (sCT) images from kilovoltage cone-beam computed tomography (kV-CBCT) images for adaptive radiation therapy (ART) in head and neck cancer (HNC). This study generated the supervised U-Net deep learning model using 3,491 image pairs from planning computed tomography (pCT) and kV-CBCT datasets obtained from 40 HNC patients. The dataset was split into 80% for training and 20% for testing. The evaluation of the sCT images compared to pCT images focused on three aspects: Hounsfield units accuracy, assessed using mean absolute error (MAE) and root mean square error (RMSE); image quality, evaluated using the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) between sCT and pCT images; and dosimetric accuracy, encompassing 3D gamma passing rates for dose distribution and percentage dose difference. MAE, RMSE, PSNR, and SSIM showed improvements from their initial values of 53.15 ± 40.09, 153.99 ± 79.78, 47.91 ± 4.98 dB, and 0.97 ± 0.02 to 41.47 ± 30.59, 130.39 ± 78.06, 49.93 ± 6.00 dB, and 0.98 ± 0.02, respectively. Regarding dose evaluation, 3D gamma passing rates for dose distribution within sCT images under 2%/2 mm, 3%/2 mm, and 3%/3 mm criteria, yielded passing rates of 92.1% ± 3.8%, 93.8% ± 3.0%, and 96.9% ± 2.0%, respectively. The sCT images exhibited minor variations in the percentage dose distribution of the investigated target and structure volumes. However, it is worth noting that the sCT images exhibited anatomical variations when compared to the pCT images. These findings highlight the potential of the supervised U-Net deep learningmodel in generating kV-CBCT-based sCT images for ART in patients with HNC.

Image dehazing via self-supervised depth guidance

Self-supervised learning methods have demonstrated promising benefits to feature representation learning for image dehazing tasks, especially for avoiding the laborious work of collecting hazy-clean image pairs, while also enabling better generalization abilities of the model. Despite the long-standing interests in depth estimation for image dehazing tasks, few works have fully explored the interactions between depth and dehazing tasks in an unsupervised manner. In this paper, a self-supervised image dehazing framework under the guidance of self-supervised depth estimation has been proposed, to fully exploit the interactions between depth and hazes for image dehazing. Specifically, the hazy image and the corresponding depth estimation are generated and optimized from the clear image in a dual-network self-supervised manner. The correlations between depth and hazy images are exploited in depth-guided hybrid attention Transformer blocks, which adaptively leverage both the cross-attention and self-attention to effectively model hazy densities via cross-modality fusion and capture global context information for better feature representations. In addition, the depth estimations of hazy images are further explored for the detection tasks on hazy images. Extensive experiments demonstrate that the depth estimation not only enhances the model generalization ability across different dehazing datasets, leading to state-of-the-art self-supervised dehazing performance, but also benefits downstream detection tasks on hazy images. Our code is available at https://github.com/DongLiangSXU/Depth-Guidance-dehazing.git.