Image Generation Research Articles

Evaluation of Natural Image Generation and Reconstruction Capabilities Based on the β-VAE Model

Natural image generation models are crucial in computer vision. However, the Variational Autoencoder (VAE) has limitations in image quality and diversity, while β-VAE achieves a balance between the decoupling of latent space and generative quality by adjusting the coefficient β. This article evaluates the performance of β-VAE in natural image generation and reconstruction tasks, comparing it with Conditional VAE and Information VAE. Train the model on the CelebA dataset, using three metrics: Mean Squared Error (MSE), Structural Similarity Index (SSIM), and Fréchet Inception Distance (FID) for matrix evaluation, to analyze the generation quality and reconstruction capability of each model. The results indicate that β-VAE performs well in reconstruction tasks, but due to the strong constraints of the latent space, the realism of the images is lower in generation tasks. Conditional VAE and Info VAE perform more balanced in terms of generation quality and task balance. This article suggests optimizing β-VAE by introducing adversarial training and flexible latent space modeling to enhance its generative capabilities. Future research can further validate the potential of β-VAE in different application scenarios.

Improving the Generalization and Robustness of Computer‐Generated Image Detection Based on Contrastive Learning

With the rapid development of image generation techniques, it becomes much more difficult to distinguish high‐quality computer‐generated (CG) images from photographic (PG) images, challenging the authenticity and credibility of digital images. Therefore, distinguishing CG images from PG images has become an important research problem in image forensics, and it is crucial to develop reliable methods to detect CG images in practical scenarios. In this paper, we proposed a forensics contrastive learning (FCL) framework to adaptively learn intrinsic forensics features for the general and robust detection of CG images. The data augmentation module is specially designed for CG image forensics, which reduces the interference of forensic‐irrelevant information and enhances discrimination features between CG and PG images in both the spatial and frequency domains. Instance‐wise contrastive loss and patch‐wise contrastive loss are simultaneously applied to capture critical discrepancies between CG and PG images from global and local views. Extensive experiments on different public datasets and common postprocessing operations demonstrate that our approach can achieve significantly better generalization and robustness than the state‐of‐the‐art approaches. This manuscript was submitted as a pre‐print in the following link https://papers.ssrn.com/-sol3/papers.cfm?abstract_id=4778441.

Markov spin models for image generation: explicit large deviations with respect to the number of pixels

Markov spin models for image generation: explicit large deviations with respect to the number of pixels

Feature Perception Based Graphic Advertising Image Generation Technology

Feature Perception Based Graphic Advertising Image Generation Technology

Enhanced Multi-Scale Cross-Attention for Person Image Generation

Enhanced Multi-Scale Cross-Attention for Person Image Generation

Research of orthodontic soft tissue profile prediction based on conditional generative adversarial networks.

This study constructed a new conditional generative adversarial network (CGAN) model to predict changes in lateral appearance following orthodontic treatment. Lateral cephalometric radiographs of adult patients were obtained before (T1) and after (T2) orthodontic treatment. The expanded dataset was divided into training, validation, and test sets by random sampling in a ratio of 8:1:1. The existing networks-Pix2Pix, Cycle-GAN, and CGAN-were trained to generate post-treatment outcomes. A new deep learning model, soft-P-CGAN, was proposed and incorporated a conditional vector input module, U-Net-based generator module, and PatchGAN-based discriminator module. Soft loss was designed to enhance generating soft tissue contours; a multiscale feature pyramid refined image quality. Predicted and actual post-treatment radiographs were superimposed and compared based on soft-tissue landmarks using mean radial error (MRE) and successful detection rate (SDR) within 2.0, 2.5, 3.0, and 4.0 mm. Any parameters of PT2 and T2 outcomes were compared using paired t-tests or Wilcoxon tests. Soft-P-CGAN showed superior performance using quantitative image quality assessment. The average MRE was 1.08 ± 0.75 mm with SDRs of 2.0, 2.5, 3.0, and 4.0 mm at 88.8 %, 95.1 %, 97.8 %, and 100 %, respectively. Soft tissue point A was the most accurate landmark, whereas predictions in the mandibular region were relatively inaccurate. None of the cephalometric predictions differed significantly from the actual results (P > 0.05). Soft-P-CGAN could predict post-treatment lateral appearance changes by learning the relationship between soft and hard tissue changes in lateral cephalograms. Most predictions were clinically acceptable, aiding clinicians in setting orthodontic treatment goals. This study explored and validated the use of image generation networks for predicting orthodontic lateral profiles and proposed new methods for enhancing their accuracy and interpretability.

TomoSAR 3D reconstruction: Cascading adversarial strategy with sparse observation trajectory

AbstractSynthetic aperture radar tomography (TomoSAR) has shown significant potential for the 3D Reconstruction of buildings, especially in critical areas such as topographic mapping, urban planning, and disaster monitoring. In practical applications, the constraints of observation trajectories frequently lead to the acquisition of a limited dataset of sparse SAR images, presenting challenges for TomoSAR 3D Reconstruction and affecting its signal‐to‐noise ratio and elevation resolution performance. The study introduces a cascade adversarial strategy based on the Conditional Generative Adversarial Network (CGAN), optimised explicitly for sparse observation trajectories. In the preliminary phase of the CGAN, the U‐Net architecture was employed to capture more global information and enhance image detail recovery capability, which is subsequently utilised in the cascade refinement network. The ResNet34 residual network in the advanced network stage was adopted to bolster feature extraction and image generation capabilities further. Based on experimental validation performed on the curated TomoSAR 3D super‐resolution dataset tailored for buildings, the findings reveal that the methodology yields a notable enhancement in image quality and accuracy compared to other techniques.

Analog Spiking U-Net integrating CBAM&ViT for medical image segmentation

SNNs are gaining popularity in AI research as a low-power alternative in deep learning due to their sparse properties and biological interpretability. Using SNNs for dense prediction tasks is becoming an important research area. In this paper, we firstly proposed a novel modification on the conventional Spiking U-Net architecture by adjusting the firing positions of neurons. The modified network model, named Analog Spiking U-Net (AS U-Net), is capable of incorporating the Convolutional Block Attention Module (CBAM) into the domain of SNNs. This is the first successful implementation of CBAM in SNNs, which has the potential to improve SNN model's segmentation performance while decreasing information loss. Then, the proposed AS U-Net (with CBAM&ViT) is trained by direct encoding on a comprehensive dataset obtained by merging several diabetic retinal vessel segmentation datasets. Based on the experimental results, the provided SNN model achieves the highest segmentation accuracy in retinal vessel segmentation for diabetes mellitus, surpassing other SNN-based models and most ANN-based related models. In addition, under the same structure, our model demonstrates comparable performance to the ANN model. And then, the novel model achieves state-of-the-art(SOTA) results in comparative experiments when both accuracy and energy consumption are considered (Fig. 1). At the same time, the ablative analysis of CBAM further confirms its feasibility and effectiveness in SNNs, which means that a novel approach could be provided for subsequent deployment and hardware chip application. In the end, we conduct extensive generalization experiments on the same type of segmentation task (ISBI and ISIC), the more complex multi-segmentation task (Synapse), and a series of image generation tasks (MNIST, Day2night, Maps, Facades) in order to visually demonstrate the generality of the proposed method.

Unbiased-Diff: Analyzing and Mitigating Biases in Diffusion Model-Based Face Image Generation

Unbiased-Diff: Analyzing and Mitigating Biases in Diffusion Model-Based Face Image Generation

Novel View Synthesis Based on Similar Perspective

ABSTRACTNeural radiance fields (NeRF) technology has garnered significant attention due to its exceptional performance in generating high‐quality novel view images. In this study, we propose an innovative method that leverages the similarity between views to enhance the quality of novel view image generation. Initially, a pre‐trained NeRF model generates an initial novel view image, which is subsequently compared and subjected to feature transfer with the most similar reference view from the training dataset. Following this, the reference view that is most similar to the initial novel view is selected from the training dataset. We designed a texture transfer module that employs a strategy progressing from coarse‐to‐fine, effectively integrating salient features from the reference view into the initial image, thus producing more realistic novel view images. By using similar views, this approach not only improves the quality of novel perspective images but also incorporates the training dataset as a dynamic information pool into the novel view integration process. This allows for the continuous acquisition and utilization of useful information from the training data throughout the synthesis process. Extensive experimental validation shows that using similar views to provide scene information significantly outperforms existing neural rendering techniques in enhancing the realism and accuracy of novel view images.

Convergence properties of Markov models for image generation with applications to spin-flip dynamics and to diffusion processes

Abstract In the field of Markov models for image generation, the main idea is to learn how non-trivial images are gradually destroyed by a trivial forward Markov dynamics over the large time window [ 0 , t ] converging towards pure noise for t → + ∞ , and to implement the non-trivial backward time-dependent Markov dynamics over the same time window [ 0 , t ] starting from pure noise at t in order to generate new images at time 0. The goal of the present paper is to analyze the convergence properties of this reconstructive backward dynamics as a function of the time t using the spectral properties of the trivial continuous-time forward dynamics for the N pixels n = 1 , . . , N . The general framework is applied to two cases : (i) when each pixel n has only two states S n = ± 1 with Markov jumps between them; (ii) when each pixel n is characterized by a continuous variable xn that diffuses on an interval ] x − , x + [ that can be either finite or infinite.

The Butterfly Effect: Color-Guided Image Generation from Unconditional Diffusion Models

The Butterfly Effect: Color-Guided Image Generation from Unconditional Diffusion Models

Application of Generative Artificial Intelligence in Dyslipidemia Care.

Dyslipidemia dramatically increases the risk of cardiovascular diseases, necessitating appropriate treatment techniques. Generative AI (GenAI), an advanced AI technology that can generate diverse content by learning from vast datasets, provides promising new opportunities to address this challenge. GenAI-powered frequently asked questions systems and chatbots offer continuous, personalized support by addressing lifestyle modifications and medication adherence, which is crucial for patients with dyslipidemia. These tools also help to promote health literacy by making information more accessible and reliable. GenAI helps healthcare providers construct clinical case scenarios, training materials, and evaluation tools, which supports professional development and evidence-based practice. Multimodal GenAI technology analyzes food images and nutritional content to deliver personalized dietary recommendations tailored to each patient's condition, improving long-term nutritional management for those with dyslipidemia. Moreover, using GenAI for image generation enhances the visual quality of educational materials for both patients and professionals, allowing healthcare providers to create real-time, customized visual aids. To apply successfully, healthcare providers must develop GenAI-related abilities, such as prompt engineering and critical evaluation of GenAI-generated data.

Synthetic Image Generation Model for Intelligent Vehicle Camera Function Testing in Rain and Fog

Synthetic Image Generation Model for Intelligent Vehicle Camera Function Testing in Rain and Fog

Improving Image Quality and Controllability in Speed + Angular Velocity to Image Generation Through Synthetic Data for Driving Simulator Generation

Improving Image Quality and Controllability in Speed + Angular Velocity to Image Generation Through Synthetic Data for Driving Simulator Generation

DiffSeqMol: A Non-Autoregressive Diffusion-Based Approach for Molecular Sequence Generation and Optimization

Background: The application of deep generative models for molecular discovery has witnessed a significant surge in recent years. Currently, the field of molecular generation and molecular optimization is predominantly governed by autoregressive models regardless of how molecular data is represented. However, an emerging paradigm in the generation domain is diffusion models, which treat data non-autoregressively and have achieved significant breakthroughs in areas such as image generation. Methods: The potential and capability of diffusion models in molecular generation and optimization tasks remain largely unexplored. In order to investigate the potential applicability of diffusion models in the domain of molecular exploration, we proposed DiffSeqMol, a molecular sequence generation model, underpinned by diffusion process. Results & Discussion: DiffSeqMol distinguishes itself from traditional autoregressive methods by its capacity to draw samples from random noise and direct generating the entire molecule. Through experiment evaluations, we demonstrated that DiffSeqMol can achieve, even surpass, the performance of established state-of-the-art models on unconditional generation tasks and molecular optimization tasks. Conclusion: Taken together, our results show that DiffSeqMol can be considered a promising molecular generation method. It opens new pathways to traverse the expansive chemical space and to discover novel molecules.

Research on image generation technology based on deep learning

Research on image generation technology based on deep learning

A EUROPEAN VIEW ON THE ETHICAL USE OF ARTIFICIAL INTELLIGENCE (AI)

Artificial intelligence (AI) is one of the most exciting technology topics of recent years. It has the potential to fundamentally change the way we work, study, and use modern technologies in many areas, such as text and image generators. However, as the use of AI becomes more widespread, it is also worth being aware of the potential problems it brings. Given the increasing dependence of our systems on AI, how we address these dilemmas could have a key impact on the future of society. Among AI industry specialists, China (Taiwan) is said to produce the components for AI development, the United States is said to research and develop AI technologies, and Europe is said to implement the regulations surrounding it. In this article, we will present the ethical dilemmas that AI research and use bring, with a focus on the relevant regulations introduced by the European Union.

제품디자인을 위한 이미지 생성형 AI 프롬프트 구성요소 사례 연구 - 미드저니와 의자를 중심으로

Image-Generative AI is recognized as a useful tool for product design that provides creative inspiration based on various ideas and improves the design process by shortening the working time through rapid visualization. In order to generate the desired image through Image- Generative AI, the importance of prompts is emphasized, and it is necessary to clearly grasp the elements constituting them. However, in the current prior research on product design, the prompt components are limited to some ranges, so it is difficult to comprehensively grasp various factors that can be considered in the product design process. Therefore, this study analyzed images and prompts centering on chairs among Image-Generative AI-based product design cases for the purpose of expanding and systematizing prompt components. Through the study, seven additional elements that were not previously covered, such as functions, objects, and characteristics, were identified, and a prompt component system for product design image generation was presented by integrating them with existing prompt components.

DIGITAL AGEISM PERSISTS IN ARTIFICIAL INTELLIGENCE-GENERATED IMAGES

Abstract Artificial intelligence systems (AI), including text-to-image models, often reflect and amplify societal age-related biases, a phenomenon known as digital ageism. This study examines if instances of digital ageism in AI-generated representations of older adults lessen over time. Examining how digital ageism evolves in AI-image generation can provide insight into the interplay between technology advancements, societal attitudes toward aging, and the well-being of older adults interacting with the dynamic digital landscape. This mixed methods project compared 164 images generated by Open AI’s DALL-E 2 image generator at two points, one year apart. Text prompts, identical at both points of inquiry, included terms from the geriatric lexicon (e.g., frail older adult, dementia, successful aging). Images were qualitatively evaluated to identify perceived gender, race, socioeconomic status, and emotional expression. Quantitative analysis of the frequency with which each category was represented found significant differences in gender and socioeconomic representation at both points of inquiry but no difference between the two points. The number of white racialized representations of older adults was 5-fold higher than other racialized representations at both points, with an increase of Asian racialized representations at the second point of inquiry. There was no significant difference between emotional expressions between the inquiry points. Despite the increased social emphasis on positive views on aging, AI text-to-image generators persistently generated predominantly white racialized middle-class male figures. A limitation of this study is that it focuses only on AI text-to-image generation and no other AI-generated content.