Frechet Inception Distance Research Articles

SAR vehicle image generation with integrated deep imaging geometric information

Deep learning is widely applied in synthetic aperture radar (SAR) target recognition. However, the high cost of collecting real SAR target data leads to insufficient data volume and diversity of SAR target datasets, making it challenging to support deep learning models. Therefore, it is necessary to expand SAR target datasets through data generation. However, existing SAR target data generation methods do not fully utilize the SAR imaging geometric information of targets. This results in generating image data that does not conform to SAR physical imaging laws. To address this problem, this paper proposes a SAR vehicle data generation method that integrates deep imaging geometric information. Compared to SAR, optical data, which contains rich geometric structural information of targets, is relatively easy to collect. Therefore, the proposed method computes the dominant scattering edge (DSE) of targets under different radar incident angles with optical data. Then, based on generative adversarial networks (GAN), a SAR vehicle target data generation model is constructed. To guide the model in learning SAR imaging geometric laws of targets, the computed DSE is integrated into the generation network with multi-scale attention mechanism modules. Compared to existing SAR target data generation methods, our proposed method can generate SAR vehicle target data with distinct dominant scattering edges, and effectively addresses the issue of generated target data not conforming to SAR imaging physical laws. Metrics such as Fréchet Inception Distance (FID), scattering intensity difference, and Structural Similarity Index (SSIM) are adopted for evaluation. The SSIM between the generated and the real SAR target data reaches 0.834, the FID minimum of 96.98, and the scattering intensity difference minimum of 0.37 dB, all indicating that the SAR target data generated by the proposed method is similar to real ones in terms of deep feature, scattering intensity, and image structure. Finally, the SAR target data generated by the proposed method is verified through target recognition experiments with VGG16 and ResNet50 as backbone network. The average recognition accuracy increases by 2.80% and 2.14%, respectively. The target recognition results demonstrate that the generated SAR data can help improve target recognition accuracy, thereby validating the effectiveness of data generated by the proposed method.

Multi-planar dual adversarial network based on dynamic 3D features for MRI-CT head and neck image synthesis

Objective. Head and neck radiotherapy planning requires electron densities from different tissues for dose calculation. Dose calculation from imaging modalities such as MRI remains an unsolved problem since this imaging modality does not provide information about the density of electrons. Approach. We propose a generative adversarial network (GAN) approach that synthesizes CT (sCT) images from T1-weighted MRI acquisitions in head and neck cancer patients. Our contribution is to exploit new features that are relevant for improving multimodal image synthesis, and thus improving the quality of the generated CT images. More precisely, we propose a Dual branch generator based on the U-Net architecture and on an augmented multi-planar branch. The augmented branch learns specific 3D dynamic features, which describe the dynamic image shape variations and are extracted from different view-points of the volumetric input MRI. The architecture of the proposed model relies on an end-to-end convolutional U-Net embedding network. Results. The proposed model achieves a mean absolute error (MAE) of 18.76±5.167 in the target Hounsfield unit (HU) space on sagittal head and neck patients, with a mean structural similarity (MSSIM) of 0.95±0.09 and a Frechet inception distance (FID) of 145.60±8.38 . The model yields a MAE of 26.83±8.27 to generate specific primary tumor regions on axial patient acquisitions, with a Dice score of 0.73±0.06 and a FID distance equal to 122.58±7.55 . The improvement of our model over other state-of-the-art GAN approaches is of 3.8%, on a tumor test set. On both sagittal and axial acquisitions, the model yields the best peak signal-to-noise ratio of 27.89±2.22 and 26.08±2.95 to synthesize MRI from CT input. Significance. The proposed model synthesizes both sagittal and axial CT tumor images, used for radiotherapy treatment planning in head and neck cancer cases. The performance analysis across different imaging metrics and under different evaluation strategies demonstrates the effectiveness of our dual CT synthesis model to produce high quality sCT images compared to other state-of-the-art approaches. Our model could improve clinical tumor analysis, in which a further clinical validation remains to be explored.

Stabilized GAN models training with kernel-histogram transformation and probability mass function distance

Image generation using generative adversarial networks (GANs) has been extensively researched in recent years. Despite active developments, the chronic issue of training instability in GANs remains unresolved. To alleviate this problem, this study proposes a model named probability mass function GANs (PMF-GAN), which handles the inherent limitation of GANs. The PMF-GAN framework employs kernels, histogram transformation, and probability mass function (PMF) distance for distribution learning. The configuration of PMF-GAN kernel and PMF distance offers flexibility, allowing for optimal settings tailored to datasets and experimental environments. In this study, experiments were conducted using the gaussian kernel across five different distances. The experiments demonstrated that PMF-GAN outperforms the baselines in terms of visual quality and evaluation metrics, such as Inception score and Frechet Inception distance (FID). For example, in the CIFAR-10 dataset, Euclidean-based PMF-GAN applying with 3 bins showed a 21.5 % and 32.8 % improvement in Inception score and FID, respectively, compared to conventional WGAN-GP. Similarly, in the AFHQ dataset with the same settings, the improvements were 56.9 % and 61.5 %. As a result, this study presents the potential to achieve stable training processes in GAN models with modified loss function structures. The flexibility of the proposed model allows for simultaneous application to various models, contributing to the overall improvement of generative model training processes in the future.

Evaluation Metrics for Generative Models: An Empirical Study

Generative models such as generative adversarial networks, diffusion models, and variational auto-encoders have become prevalent in recent years. While it is true that these models have shown remarkable results, evaluating their performance is challenging. This issue is of vital importance to push research forward and identify meaningful gains from random noise. Currently, heuristic metrics such as the inception score (IS) and Fréchet inception distance (FID) are the most common evaluation metrics, but what they measure is not entirely clear. Additionally, there are questions regarding how meaningful their score actually is. In this work, we propose a novel evaluation protocol for likelihood-based generative models, based on generating a high-quality synthetic dataset on which we can estimate classical metrics for comparison. This new scheme harnesses the advantages of knowing the underlying likelihood values of the data by measuring the divergence between the model-generated data and the synthetic dataset. Our study shows that while FID and IS correlate with several f-divergences, their ranking of close models can vary considerably, making them problematic when used for fine-grained comparison. We further use this experimental setting to study which evaluation metric best correlates with our probabilistic metrics.

Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology.

Artificial Intelligence (AI)-based image analysis has immense potential to support diagnostic histopathology, including cancer diagnostics. However, developing supervised AI methods requires large-scale annotated datasets. A potentially powerful solution is to augment training data with synthetic data. Latent diffusion models, which can generate high-quality, diverse synthetic images, are promising. However, the most common implementations rely on detailed textual descriptions, which are not generally available in this domain. This work proposes a method that constructs structured textual prompts from automatically extracted image features. We experiment with the PCam dataset, composed of tissue patches only loosely annotated as healthy or cancerous. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fréchet Inception Distance (FID) by 88.6. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55. Finally, we show that synthetic data effectively train AI models.

Mask-guided dual-perception generative adversarial network for synthesizing complex maize diseased leaves to augment datasets

In practice, acquiring and annotating data in specialized domains can be costly, thereby constraining the performance and applicability of deep learning. Utilizing generative models to synthesize data proves to be an effective augmentation technique. Therefore, this research proposes a diseased leaf generation pipeline to diversify the maize disease datasets. We introduce the Dual-Perception Cycle-Consistent Generative Adversarial Network (DP-CycleGAN). During training, incorporating our proposed Structure Perception (SP) loss and Texture Perception (TP) loss functions. These losses guide the model's attention areas through activation reconstruction and mask mechanisms, thereby improving the overall perceptual quality of the generated images and the realism of the disease lesions. We constructed a maize leaf mixed disease dataset to simulate the complex conditions of real-world disease occurrence. Experimental results show that the DP-CycleGAN generates higher-quality and more realistic diseased leaf images. Compared to CycleGAN and state-of-the-art method, DP-CycleGAN shows a 29.6% and 15.7% reduction in Fréchet Inception Distance (FID) scores and a 125.3% and 61.5% increase in Structural Similarity (SSIM) values, respectively. Simultaneously, by incorporating synthetic data during training, our approach significantly enhances the performance of the recognition model in scenarios of both data abundance and scarcity, with improvement rates exceeding two times those of existing state-of-the-art methods. This contributes to the application of Artificial Intelligence (AI) in agricultural production practices.

O-251 Generative artificial intelligence substantially enhances the accuracy of embryo selection models

Abstract Study question Can generative artificial intelligence (AI) produce high-fidelity human embryo images to improve AI-based embryo selection models? Summary answer Generative AI models exhibit the capability to generate high-fidelity human embryo images and can substantially improve the performance of AI models through extensive training data. What is known already The integration of AI into in vitro fertilization (IVF) procedures holds the potential to enhance objectivity and automate embryo selection for transfer. However, the effectiveness of AI is limited by data scarcity and ethical concerns related to patient data privacy. Generative Adversarial Networks (GAN) have emerged as a promising approach to alleviate data limitations by generating synthetic data that closely approximate real images. However, the advantage of applying GAN in IVF embryo images is yet to be determined. Study design, size, duration Embryo images were retrieved as training data from time-lapse microscopy (TLM) videos (n = 328). Embryo key morphokinetic variables including blastomere division from 1-cell stage to 9 (and more) cells-stage, compaction of morula, and blastocyst formation were manually annotated by six embryologists. A style-based GAN was fine-tuned as the generative model and a residual network as the morphokinetic classification model. Participants/materials, setting, methods We configured generative models with data augmentation (AUG) and pretrained weights (Pretrained-T/R) to test their refinement to synthetic image quality. We analyzed quantitative metrics including Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) to assess the quality and fidelity of the generated images. Subsequently, we evaluated qualitative performance through a visual Turing test. Then, we compared the performance of a classifier by training real images alone and by integrating real images with generated data. Main results and the role of chance During the training process, we observed consistent improvement in image quality that was measured by FID and KID scores. The AUG+Pretrained-R model showed the highest performance of the evaluated five configurations with FID and KID scores of 15.2 and 0.004 following 5,000 training iterations. Subsequently, we carried out a visual Turing test, such that a total of 60 individuals including IVF embryologists (Group I, n = 25), IVF laboratory technicians (Group II, n = 15), and non-experts (Group III, n = 20) evaluated the synthetic blastocyst-stage embryo images. Group I displayed accuracy of 55.7% (±6.2), sensitivity of 65.9% (±14.1), and specificity of 45.1% (±17.8). Group II showed accuracy of 54.2% (±4.7), sensitivity of 65.9% (±17.6), and specificity of 42.0% (±19.6). Group III had the accuracy of 50.1% (±3.8), sensitivity of 49.3% (±6.3), and specificity was 50.9% (±10.4). Statistical analysis indicated significant differences in accuracy (Kruskal-Wallis test, P=1.2x10-3) and sensitivity (Kruskal-Wallis test, P=1.6x10-4) among the three groups. In the embryonic stage classification model, integrated with generated data outperformed the model using real data solely, with an increase of validation accuracy from 72.6% to 90.6%. The classifier was tested on external TLM video data (n = 100) where the area under the curve was 0.92. Limitations, reasons for caution Further research is needed to involve a more extensive and diverse dataset, e.g. video generation, and more detailed annotations including polar body appearance, pronuclei appearance and disappearance, and blastocyst expansion and hatching. Wider implications of the findings Generative AI offers potential in generating high-fidelity human embryo images. This approach ensures sufficient training datasets while addressing class imbalances for robust AI-based methods, ultimately transforming embryo selection and enhancing IVF outcomes. A set of 5,000 generated images (1024x1024-pixel) at each embryonic developmental stage was open-source for future AI studies. Trial registration number not applicable

SCGAN: Semi-Centralized Generative Adversarial Network for image generation in distributed scenes

Distributed Generative Adversarial Networks (GANs) are pivotal for generative tasks in distributed data environments. Conventional architectures, whether centralized or decentralized, inadequately balance global and local data distribution integration. We propose the Semi-Centralized Distributed GAN (SCGAN), a hybrid framework combining the merits of both approaches. SCGAN employs dual generators — one decentralized, one centralized — and a feature extractor that optimizes their outputs by dynamically calculating feature distances, thereby achieving a balanced semi-centralized generator. Extensive experiments on five benchmark image datasets reveal that SCGAN surpasses existing models, delivering superior Fréchet Inception Distance (FID) scores and generating highly realistic images. Moreover, SCGAN’s robustness and versatility are demonstrated through its efficacy in downstream applications, including image augmentation and distributed privacy protection. Notably, SCGAN’s generated images were evaluated using a distributed classifier, a quality metric (IS), and human observation, consistently outperforming existing methods. This study highlights SCGAN’s significant advancements in distributed generative modeling and its nuanced approach to managing complex data distributions, particularly excelling in privacy-preserving scenarios.

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Synthetic Genitourinary Image Synthesis via Generative Adversarial Networks: Enhancing Artificial Intelligence Diagnostic Precision.

In the realm of computational pathology, the scarcity and restricted diversity of genitourinary (GU) tissue datasets pose significant challenges for training robust diagnostic models. This study explores the potential of Generative Adversarial Networks (GANs) to mitigate these limitations by generating high-quality synthetic images of rare or underrepresented GU tissues. We hypothesized that augmenting the training data of computational pathology models with these GAN-generated images, validated through pathologist evaluation and quantitative similarity measures, would significantly enhance model performance in tasks such as tissue classification, segmentation, and disease detection. To test this hypothesis, we employed a GAN model to produce synthetic images of eight different GU tissues. The quality of these images was rigorously assessed using a Relative Inception Score (RIS) of 1.27 ± 0.15 and a Fréchet Inception Distance (FID) that stabilized at 120, metrics that reflect the visual and statistical fidelity of the generated images to real histopathological images. Additionally, the synthetic images received an 80% approval rating from board-certified pathologists, further validating their realism and diagnostic utility. We used an alternative Spatial Heterogeneous Recurrence Quantification Analysis (SHRQA) to assess the quality of prostate tissue. This allowed us to make a comparison between original and synthetic data in the context of features, which were further validated by the pathologist's evaluation. Future work will focus on implementing a deep learning model to evaluate the performance of the augmented datasets in tasks such as tissue classification, segmentation, and disease detection. This will provide a more comprehensive understanding of the utility of GAN-generated synthetic images in enhancing computational pathology workflows. This study not only confirms the feasibility of using GANs for data augmentation in medical image analysis but also highlights the critical role of synthetic data in addressing the challenges of dataset scarcity and imbalance. Future work will focus on refining the generative models to produce even more diverse and complex tissue representations, potentially transforming the landscape of medical diagnostics with AI-driven solutions.

HDMA-CGAN: Advancing Image Style Transfer with Deep Learning

The integration of artificial intelligence (AI) and deep learning heralds a transformative era in pattern recognition and computer vision, notably in image style transfer. We introduce the hierarchical dynamic multi-attention cycle generative adversarial network (HDMA-CGAN), an innovative deep learning architecture poised to redefine image style transfer capabilities. HDMA-CGAN employs a novel multi-attention mechanism and color optimization strategies, enabling precise style replication with improved fidelity and vibrancy. Our model surpasses existing benchmarks in image quality, validated by leading metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Fréchet inception distance (FID). Although HDMA-CGAN advances the state of the art, it necessitates high computational resources and faces challenges with very high-resolution images. Future work could explore optimizing the model’s efficiency for real-time applications and extending its application to video content. This work enhances the tools available for visual content creation and digital media enhancement, leveraging advanced pattern recognition and AI techniques to significantly impact computer vision and image processing.

IMG-25. PREDICTING ANATOMICAL TUMOR GROWTH IN PEDIATRIC HIGH-GRADE GLIOMAS VIA DENOISING DIFFUSION MODELS

Abstract BACKGROUND Pediatric diffuse midline glioma (DMG) has a poor prognosis with radiotherapy as the standard of palliative care. Radiation strategies using individual geometric doseshaping have the potential to maximize patient benefit. This study aimed to establish a proof-of-principle to generate clinically relevant anatomical representations of the progression of pediatric DMG magnetic resonance image (MRI) slices through denoising diffusion implicit models (DDIM). METHODS Based on the BraTS23 data, we analyzed the generation of MRIs with enlarged tumor sizes starting from a baseline scan. Models were pre-trained on adult glioblastoma (GBM) prior to transfer learning on pediatric high-grade glioma. Test set performance was evaluated using the Frechet Inception Distance (FID), and Structural Similarity Index (SSIM). Qualitative evaluation involved visual classification of real vs generated images by two independent radiology trainees. The optimized network was tested to forecast anatomical tumor growth in five DMG patients from an independent dataset comprising longitudinal T2-FLAIR images. RESULTS We obtain high-fidelity generated images, supported by an FID of 12.4 and 0.8 SSIM on the adult test set. Despite the smaller pediatric BraTS dataset yielding a 34.8 FID and 0.84 SSIM, human observers failed to distinguish generated from real MRIs (mean recall and precision of 0.51). The generated anatomical predictions also closely resemble the observed longitudinal tumor progression in the pediatric DMG subset. Notably, the direction and extent of tumor growth align while the surrounding brain anatomy is preserved. Quantitatively, the mean DICE score between segmentations of generated and real tumors was 0.8. CONCLUSIONS Our model produces high-quality images that conform to growth patterns of pediatric DMGs. Further quantitative validation on larger longitudinal datasets is required, to ensure the method’s robustness. The proposed method enables a personalized approach for defining target regions in radiation therapy which could translate to improved clinical outcomes.

Data-driven crop growth simulation on time-varying generated images using multi-conditional generative adversarial networks

BackgroundImage-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap.MethodsWe present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation.ResultsExperiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model.ConclusionThe realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions.

Synthesis and evaluation of seamless, large-scale, multispectral satellite images using Generative Adversarial Networks on land use and land cover and Sentinel-2 data

ABSTRACT Artificial intelligence (AI) began to make its way into geoinformation science several decades ago and since then has constantly brought forth new cutting-edge approaches for diverse geographic use cases. AI and deep learning methods have become essential approaches for land use and land cover (LULC) classifications, which are important in urban planning and regional management. While the extraction of LULC information from multispectral satellite images has been a well-studied part of past and present research, only a few studies emerged about the recovery of spectral properties from LULC information. Estimates of the spectral characteristics of LULC categories could enrich LULC forecasting models by providing necessary information to delineate vegetation indices or microclimatic parameters. We train two identical Conditional Generative Adversarial Networks (CGAN) to synthesize a multispectral Sentinel-2 image based on different combinations of open-source LULC data sets. Large-scale synthetic multispectral satellite images of the administrative region of Bonn and Rhein-Sieg in Germany are generated with a Euclidean distance-based patch-fusion method. The approach generated a realistic-looking satellite image without noticeable seams between patch borders. Based on several metrics, such as difference calculations, the spectral information divergence (SID), and the Fréchet inception distance (FID), we evaluate the resulting images. The models reach mean SIDs as low as 0.026 for urban fabrics and forests and FIDs below 90 for bands B2 and B5 showing that the CGAN is capable of synthesizing distinct synthetic features matching with features typical for respective LULC categories and manages to mimic multispectral signatures. The method used in this paper to generate large-scale synthetic multispectral satellite images can be used as an approach to support scenario-oriented sustainable urban planning.

Precision sketching with de-aging networks in forensics

Addressing the intricacies of facial aging in forensic facial recognition, traditional sketch portraits often fall short in precision. This study introduces a pioneering system that seamlessly integrates a de-aging module and a sketch generator module to overcome the limitations inherent in existing methodologies. The de-aging module utilizes a deepfake-based neural network to rejuvenate facial features, while the sketch generator module leverages a pix2pix-based Generative Adversarial Network (GAN) for the generation of lifelike sketches. Comprehensive evaluations on the CUHK and AR datasets underscore the system’s superior efficiency. Significantly, comprehensive testing reveals marked enhancements in realism during the training process, demonstrated by notable reductions in Frechet Inception Distance (FID) scores (41.7 for CUHK, 60.2 for AR), augmented Structural Similarity Index (SSIM) values (0.789 for CUHK, 0.692 for AR), and improved Peak Signal-to-Noise Ratio (PSNR) metrics (20.26 for CUHK, 19.42 for AR). These findings underscore substantial advancements in the accuracy and reliability of facial recognition applications. Importantly, the system, proficient in handling diverse facial characteristics across gender, race, and culture, produces both composite and hand-drawn sketches, surpassing the capabilities of current state-of-the-art methods. This research emphasizes the transformative potential arising from the integration of de-aging networks with sketch generation, particularly for age-invariant forensic applications, and highlights the ongoing necessity for innovative developments in de-aging technology with broader societal and technological implications.

Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection

Background/aimsThe aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus...

Towards Generating Authentic Human-Removed Pictures in Crowded Places Using a Few-Second Video.

If we visit famous and iconic landmarks, we may want to take a photo of them. However, such sites are usually crowded, and taking photos with only landmarks without people could be challenging. This paper aims to automatically remove people in a picture and produce a natural image of the landmark alone. To this end, it presents Thanos, a system to generate authentic human-removed images in crowded places. It is designed to produce high-quality images with reasonable computation cost using short video clips of a few seconds. For this purpose, a multi-frame-based recovery region minimization method is proposed. The key idea is to aggregate information partially available from multiple image frames to minimize the area to be restored. The evaluation result presents that the proposed method outperforms alternatives; it shows lower Fréchet Inception Distance (FID) scores with comparable processing latency. It is also shown that the images by Thanos achieve a lower FID score than those of existing applications; Thanos's score is 242.8, while those by Retouch-photos and Samsung object eraser are 249.4 and 271.2, respectively.

Creating synthetic energy meter data using conditional diffusion and building metadata

Advances in machine learning and increased computational power have driven progress in energy-related research. However, limited access to private energy data from buildings hinders traditional regression models relying on historical data. While generative models offer a solution, previous studies have primarily focused on short-term generation periods (e.g., daily profiles) and a limited number of meters. Thus, the study proposes a conditional diffusion model for generating high-quality synthetic energy data using relevant metadata. Using a dataset comprising 1,828 power meters from various buildings and countries, this model is compared with traditional methods like Conditional Generative Adversarial Networks (CGAN) and Conditional Variational Auto-Encoders (CVAE). It explicitly handles long-term annual consumption profiles, harnessing metadata such as location, weather, building, and meter type to produce coherent synthetic data that closely resembles real-world energy consumption patterns. The results demonstrate the proposed diffusion model's superior performance, with a 36% reduction in Fréchet Inception Distance (FID) score and a 13% decrease in Kullback-Leibler divergence (KL divergence) compared to the following best method. The proposed method successfully generates high-quality energy data through metadata, and its code will be open-sourced, establishing a foundation for a broader array of energy data generation models in the future.

AG‐SDM: Aquascape generation based on stable diffusion model with low‐rank adaptation

AbstractAs an amalgamation of landscape design and ichthyology, aquascape endeavors to create visually captivating aquatic environments imbued with artistic allure. Traditional methodologies in aquascape, governed by rigid principles such as composition and color coordination, may inadvertently curtail the aesthetic potential of the landscapes. In this paper, we propose Aquascape Generation based on Stable Diffusion Models (AG‐SDM), prioritizing aesthetic principles and color coordination to offer guiding principles for real artists in Aquascape creation. We meticulously curated and annotated three aquascape datasets with varying aspect ratios to accommodate diverse landscape design requirements regarding dimensions and proportions. Leveraging the Fréchet Inception Distance (FID) metric, we trained AGFID for quality assessment. Extensive experiments validate that our AG‐SDM excels in generating hyper‐realistic underwater landscape images, closely resembling real flora, and achieves state‐of‐the‐art performance in aquascape image generation.

Fast SPECT/CT planar bone imaging enabled by deep learning enhancement.

The application of deep learning methods in rapid bone scintigraphy is increasingly promising for minimizing the duration of SPECT examinations. Recent works showed several deep learning models based on simulated data for the synthesis of high-count bone scintigraphy images from low-count counterparts. Few studies have been conducted and validated on real clinical pairs due to the misalignment inherent in multiple scan procedures. To generate high quality whole-body bone images from 2× and 3× fast scans using deep learning based enhancement method. Seventy-six cases who underwent whole-body bone scans were enrolled in this prospective study. All patients went through a standard scan at a speed of 20cm/min, which followed by fast scans consisting of 2× and 3× accelerations at speeds of 40 and 60cm/min. A content-attention image restoration approach based on Residual-in-Residual Dense Block (RRDB)is introduced to effectively recover high-quality images from fast scans with fine-details and less noise. Our approach is robust with misalignment introduced from patient's metabolism, and shows valid count-level consistency. Learned Perceptual Image Patch Similarity (LPIPS) and Fréchet Inception Distance (FID) are employed in evaluating the similarity to the standard bone images. To further prove our method practical in clinical settings, image quality of the anonymous images was evaluated by two experienced nuclear physicians on a 5-point Likert scale (5= excellent). The proposed method reaches the state-of-the-art performance on FID and LPIPS with 0.583 and 0.176 for 2× fast scans and 0.583 and 0.185 for 3× fast scans. Clinic evaluation further demonstrated the restored images had a significant improvement compared to fast scan in image quality, technetium 99m-methyl diphosphonate (Tc-99m MDP) distribution, artifacts, and diagnostic confidence. Our method was validated for accelerating whole-body bone scans by introducing real clinical data. Confirmed by nuclear medicine physicians, the proposed method can effectively enhance image diagnostic value, demonstrating potential for efficient high-quality fast bone imaging in practical settings.