Image Generation Method Research Articles

Evaluation of Digital Image Generation Based on Virtual Interaction Design under Blockchain Technology

With the rapid development of current technology, computer vision technology has also achieved better development, providing more possibilities for digital image generation. However, traditional digital image generation methods have high operational requirements for designers due to difficulties in collecting data sets and simulating environmental scenes, which results in poor quality, lack of diversity, and long generation speed of generated images, making it difficult to meet the current needs of image generation. In order to better solve these problems and promote the better development of digital image generation, this paper introduced a virtual interaction design under current popular blockchain technology in image generation, built a generative model through the virtual interaction design under the blockchain technology, optimized the image generation, and verified it in practice. The research results show that the digital image generation number of the method in this paper is 641. This indicates that method generates more images, has better diversity in digital image generation, and better image quality. Compared to other digital image generation methods, this method can better meet the needs of high-quality and diverse images, better adapt to the needs of different image generation, and is of great significance for promoting the better development of image generation.

Imaging Techniques of Tomatoes (Solanum Lycopersicum L.) Grown with Different Organic and Conventional Fertilizer Applications

In this study, which was carried out in field conditions in Van in 2019 and 2020, different image generation methods (biocrystallization method, and circular and rising picture chromatography methods) were used to determine the difference between various organic and conventional fertilizer applications. 3% frozen tomatoes and 16% CuCl2.2H2O were applied in the biocrystallization method, 80% frozen tomatoes and 1% silver nitrate solution were used in the circular chromatography method, and 100% frozen tomatoes and 0.5% silver nitrate and 0.5% iron sulfate solution were employed in the rising picture method. In light of the visual findings obtained at the end of the study, it has been determined that there are some differences between organic and conventional fertilizer applications. As the alteration between organic and conventional products; the center number difference in the copper crystallization method; the smoothness of the rings formed in the circular chromatogram and the vividness of the colors; in the rising painting method, it clearly reveals the transitions between colors and the difference in light and dark tones that occur in colors.

DG-GAN: A High Quality Defect Image Generation Method for Defect Detection.

The surface defect detection of industrial products has become a crucial link in industrial manufacturing. It has a series of chain effects on the control of product quality, the safety of the subsequent use of products, the reputation of products, and production efficiency. However, in actual production, it is often difficult to collect defect image samples. Without a sufficient number of defect image samples, training defect detection models is difficult to achieve. In this paper, a defect image generation method DG-GAN is proposed for defect detection. Based on the idea of the progressive generative adversarial, D2 adversarial loss function, cyclic consistency loss function, a data augmentation module, and a self-attention mechanism are introduced to improve the training stability and generative ability of the network. The DG-GAN method can generate high-quality and high-diversity surface defect images. The surface defect image generated by the model can be used to train the defect detection model and improve the convergence stability and detection accuracy of the defect detection model. Validation was performed on two data sets. Compared to the previous methods, the FID score of the generated defect images was significantly reduced (mean reductions of 16.17 and 20.06, respectively). The YOLOX detection accuracy was significantly improved with the increase in generated defect images (the highest increases were 6.1% and 20.4%, respectively). Experimental results showed that the DG-GAN model is effective in surface defect detection tasks.

Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation

The performances of defect inspection have been severely hindered by insufficient defect images in industries, which can be alleviated by generating more samples as data augmentation. We propose the first defect image generation method in the challenging few-shot cases. Given just a handful of defect images and relatively more defect-free ones, our goal is to augment the dataset with new defect images. Our method consists of two training stages. First, we train a data-efficient StyleGAN2 on defect-free images as the backbone. Second, we attach defect-aware residual blocks to the backbone, which learn to produce reasonable defect masks and accordingly manipulate the features within the masked regions by training the added modules on limited defect images. Extensive experiments on MVTec AD dataset not only validate the effectiveness of our method in generating realistic and diverse defect images, but also manifest the benefits it brings to downstream defect inspection tasks. Codes are available at https://github.com/Ldhlwh/DFMGAN.

Attention guided domain alignment for conditional face image generation

Despite the great success of Generative Adversarial Network (GAN) in face image generation, it is still a challenge to generate a faithful yet high-fidelity face image given an exemplar image and a conditional input from a distinct domain (e.g., a semantic segmentation mask or an edge map). Existing methods learn image-level features to align distinct domains in an intermediate domain, which ignore the spatial relationship of the facial semantic parts and therefore lead to semantic mismatching problems. In addition, it is computationally expensive to establish dense correspondences in the intermediate domain, especially for high-resolution face images. To address these problems, we propose a novel attention guided domain alignment method for conditional face image generation, which aligns two domains directly under the guidance of the local attention learned from semantically similar face parts. In particular, we assign a dedicated index for each feature block and adopt a top-k ranking operation to correspond block-wise features in two distinct domains, which exploits the spatial relationship of facial parts and preserves the texture structure during alignment. The local attention is then learned from the retrieved blocks, which reduces computation complexity substantially and therefore enables to build alignment in a high resolution. The aligned features are finally fused with adaptive weights learned from their long-range correlation coefficients, which capture the semantic coherence of the style features between the two domains. Extensive experimental results on the CelebAMask-HQ dataset demonstrate that the proposed method is superior to the state-of-the-art methods.

Transformer-based image generation from scene graphs

Graph-structured scene descriptions can be efficiently used in generative models to control the composition of the generated image. Previous approaches are based on the combination of graph convolutional networks and adversarial methods for layout prediction and image generation, respectively. In this work, we show how employing multi-head attention to encode the graph information, as well as using a transformer-based model in the latent space for image generation can improve the quality of the sampled data, without the need to employ adversarial models with the subsequent advantage in terms of training stability.The proposed approach, specifically, is entirely based on transformer architectures both for encoding scene graphs into intermediate object layouts and for decoding these layouts into images, passing through a lower dimensional space learned by a vector-quantized variational autoencoder. Our approach shows an improved image quality with respect to state-of-the-art methods as well as a higher degree of diversity among multiple generations from the same scene graph. We evaluate our approach on three public datasets: Visual Genome, COCO, and CLEVR. We achieve an Inception Score of 13.7 and 12.8, and an FID of 52.3 and 60.3, on COCO and Visual Genome, respectively. We perform ablation studies on our contributions to assess the impact of each component. Code is available at https://github.com/perceivelab/trf-sg2im.

Anomaly-GAN: A data augmentation method for train surface anomaly detection

Train surface anomaly detection is an essential task in vision-based railway safety inspection. Although existing deep learning methods show great potential, their anomaly detection accuracy is affected by the lack of abnormal images. The number of abnormal images on the train surface is far less than that of normal images. One effective way to solve this problem is to expand the abnormal sample. However, most of train surface anomaly image generation methods faces difficulties in producing images with high quality and rich diversity at the same time. In addition, generating small-area anomalies is not ideal. An Anomaly-GAN based on mask pool, abnormal aware loss, and local versus global discriminators is thus proposed in this paper to solve these problems. The mask pool consists of prior-knowledge-based masks and expert-experience-based masks that guide model in generating anomalies with different shapes, rotation angles, spatial locations, and part numbers. The anomaly aware loss focuses on small-area anomalies, thereby promoting the generated anomalies with more detailed textures and richer semantics. The local versus global consistency discriminators combine local and global feature expressions that lead to the generation of more realistic and natural abnormal samples. The experiments show that, compared with other advanced data augmentation algorithms, the images generated by Anomaly-GAN achieve the best FID and LPIPS scores in all anomaly categories. In addition, compared with the case without data augmentaion, the proposed data enhancement method improves the performance of CNN on mAP and mIOU by 25.6% and 24.2%, respectively. Test code is available in https://github.com/AI-Dream-Chaser/Anomaly-GAN.

SeTGAN: Semantic‐text guided face image generation

AbstractIn this article, we propose a method to jointly control face image generation through semantic segmentation maps and text. Existing semantic segmentation maps lack detailed face attributes such as beards, and it is difficult to explicitly represent the gender of the target person by virtue of the semantic maps. State‐of‐the‐art face image generation methods guided by semantic segmentation maps mostly solved this by introducing the original image for supervision, which cannot accurately control the detailed attributes of the target face. At the same time, the text‐guided image generation method perform poorly in controlling the front and side of the face pose. Therefore, we propose an idea that the semantic segmentation map controls the coarse content of the target image and the text controls the fine details of the target image. Through the well‐designed mapping network and content mixing mechanism, the model in this article flexibly draws on the advantages of the two modes, and can generate high‐resolution images that are diverse, high‐quality, and more faithful to the target in detail attributes than most of previous methods. Extensive experiments demonstrate the superior performance of the proposed method in terms of accuracy and fidelity.

Researches Advanced in Image Generation based on Deep Learning

Image generation has always been a study hotspot in machine learning, which aims to build models to learn specific semantic distributions from massive image data to generate realistic simulated images. Thanks to the deep learning technology’s quick development, generative models are constantly being developed and huge success has been achieved in image generation tasks. According to difference between generative models, the existing image generation methods based on deep learning can mainly be separated into three models: image generation based on Variational Autoencoder (VAE), image generation based on Generative Adversarial Network (GAN) and image generation combined the VAE and GAN. Focusing on the three frameworks, in this paper, the development process and related principles of each type of generation model are described respectively. After that, the different generation results of different generation models for the agreed training set are compared intuitively, the advantages and problems of various models are proposed, and reasonable improvement measures are proposed for some problems. Finally, the development prospects of various models are prospected.

P‐2.9: A review of image generation methods based on deep learning

Image as a medium of visual information transmission have the advantages of vividness, intuition and easy understanding. They play an important role in information transmission and utilization. In recent years, due to the rapid development of deep learning technology in the field of image processing, image generative model based on neural network has become one of the current research hotspots. In the field of deep learning, unsupervised learning model has received more and more attentions, especially in the field of deep generative models, which has made breakthrough progress [1] . Among them, Variational Auto-Encoder (VAE), Generative Adversarial Network (GAN), and Diffusion Model are the three most representative research methods in the field of unsupervised learning. They have been applied more and more in the field of deep generative models. Especially, the high-quality image generative models based on the generative adversarial network continue to be hot. The diffusion model is a rising star, which is favored by more and more researchers. This paper first summarizes the main research work, improvement mechanism and features of image generation methods based on VAE and GAN, then introduces the principle of the rising diffusion model and its representative models. Finally, the advantages and limitations of the above methods are compared and analyzed, and prospects for future research are put forward.

Novel Creation Method of Feature Graphics for Image Generation Based on Deep Learning Algorithms

In this paper, we propose a novel creation method of feature graphics by deep learning algorithms based on a channel attention module consisting of a separable deep convolutional neural network and an SENet network. The main innovation of this method is that the image feature of sample images is extracted by convolution operation and the key point matrix is obtained by channel weighting calculation to create feature graphics within the channel attention module. The main problem of existing image generation methods is that the complex network training and calculation process affects the accuracy and efficiency of image generation. It greatly reduced the complexity of image generation and improved the efficiency when we trained the image generation network with the feature graphic maps. To verify the superiority of this method, we conducted a comparative experiment with the existing method. Additionally, we explored the influence on the accuracy and efficiency of image generation of the channel number of the weighting matrix based on the test experiment. The experimental results demonstrate that this method highlights the image features of geometric lines, simplifies the complexity of image generation and improves the efficiency. Based on this method, images with more prominent line features are generated from the description text and dynamic graphics are created for the display of the images generated, which can be applied in the construction of smart museums.

Research on Image Generation Methods Based on Adversarial Neural Networks

Image generation has become a heated research topic in recent years owing to its wide landing scenes and great potential in all walks of life. Especially after the emergence of adversarial neural networks, both the training process and results have been greatly improved compared with previous model methods. This paper focuses on the advantages of directly using Generative Adversarial Nets (GAN) to generate images, as well as its main problems: training instability, pattern collapse, and global correlation, and introduces the strategies and skills of subsequent improved GAN for these problems. Through experiments, we compare the improved network with the original GAN and try to combine the core strategies of these networks. In the experiment, the image quality generated by the combined network is higher.

Small training dataset convolutional neural networks for application-specific super-resolution microscopy.

Machine learning (ML) models based on deep convolutional neural networks have been used to significantly increase microscopy resolution, speed [signal-to-noise ratio (SNR)], and data interpretation. The bottleneck in developing effective ML systems is often the need to acquire large datasets to train the neural network. We demonstrate how adding a "dense encoder-decoder" (DenseED) block can be used to effectively train a neural network that produces super-resolution (SR) images from conventional microscopy diffraction-limited (DL) images trained using a small dataset [15 fields of view (FOVs)]. The ML helps to retrieve SR information from a DL image when trained with a massive training dataset. The aim of this work is to demonstrate a neural network that estimates SR images from DL images using modifications that enable training with a small dataset. We employ "DenseED" blocks in existing SR ML network architectures. DenseED blocks use a dense layer that concatenates features from the previous convolutional layer to the next convolutional layer. DenseED blocks in fully convolutional networks (FCNs) estimate the SR images when trained with a small training dataset (15 FOVs) of human cells from the Widefield2SIM dataset and in fluorescent-labeled fixed bovine pulmonary artery endothelial cells samples. Conventional ML models without DenseED blocks trained on small datasets fail to accurately estimate SR images while models including the DenseED blocks can. The average peak SNR (PSNR) and resolution improvements achieved by networks containing DenseED blocks are and , respectively. We evaluated various configurations of target image generation methods (e.g., experimentally captured a target and computationally generated target) that are used to train FCNs with and without DenseED blocks and showed that including DenseED blocks in simple FCNs outperforms compared to simple FCNs without DenseED blocks. DenseED blocks in neural networks show accurate extraction of SR images even if the ML model is trained with a small training dataset of 15 FOVs. This approach shows that microscopy applications can use DenseED blocks to train on smaller datasets that are application-specific imaging platforms and there is promise for applying this to other imaging modalities, such as MRI/x-ray, etc.

An Image Generation Method of Unbalanced Ship Coating Defects Based on IGASEN-EMWGAN

During the process of ship coating, various defects will occur due to the improper operation by the workers, environmental changes, etc. The special characteristics of ship coating limit the amount of data and result in the problem of class imbalance, which is not conducive to ensuring the effectiveness of deep learning-based models. Therefore, a novel hybrid intelligent image generation algorithm called the IGASEN-EMWGAN model for ship painting defect images is proposed to tackle the aforementioned limitations in this paper. First, based on a subset of imbalanced ship painting defect image samples obtained by a bootstrap sampling algorithm, a batch of different base discriminators was trained independently with the algorithm parameter and sample perturbation method. Then, an improved genetic algorithm based on the simulated annealing algorithm is used to search for the optimal subset of base discriminators. Further, the IGASEN-EMWGAN model was constructed by fusing the base discriminators in this subset through a weighted integration strategy. Finally, the trained IGASEN-EMWGAN model is used to generate new defect images of the minority classes to obtain a balanced dataset of ship painting defects. The extensive experimental results are conducted on a real unbalanced ship coating defect database and show that, compared with the baselines, the values of the ID and FID scores are significantly improved by 4.92% and decreased by 7.29%, respectively, which prove the superior effectiveness of the proposed model in this paper.

A Context-Aware Image Generation Method for Assisted Design of Movie Posters Using Generative Adversarial Network

Considering the continuous development of the film industry and the improvement of the living standard among people, movies have gradually come to the civilians. A good movie poster can effectively reflect the content of the movie, attract the audience, stimulate the demand and achieve a good publicity effect. The current movie poster design work is mainly carried out by professional designers, which requires a lot of time and labor cost. In this paper, we propose a context-aware image generation method for assisted design of movie posters using generative adversarial network (named as MPAD-CIP for short). First, the basic information and visual contents of the movie are perceived, and the representative images are extracted, with the use of convolution operations. Then, a backbone network of deep convolutional generative neural network is formulated to generate images for summary of movies. The backbone network is composed of two components: a generator and a discriminator. Their combination realizes the computer-assisted movie poster design by sensing visual context. In the experimental part, the proposed MPAD-CIP method is compared with several benchmark models to demonstrate that the posters generated by this paper are more realistic and versatile, and some of the generated posters are exhibited.

Linearized Analysis of Noise and Resolution for DL-Based Image Generation.

Deep-learning (DL) based CT image generation methods are often evaluated using RMSE and SSIM. By contrast, conventional model-based image reconstruction (MBIR) methods are often evaluated using image properties such as resolution, noise, bias. Calculating such image properties requires time consuming Monte Carlo (MC) simulations. For MBIR, linearized analysis using first order Taylor expansion has been developed to characterize noise and resolution without MC simulations. This inspired us to investigate if linearization can be applied to DL networks to enable efficient characterization of resolution and noise. We used FBPConvNet as an example DL network and performed extensive numerical evaluations, including both computer simulations and real CT data. Our results showed that network linearization works well under normal exposure settings. For such applications, linearization can characterize image noise and resolutions without running MC simulations. We provide with this work the computational tools to implement network linearization. The efficiency and ease of implementation of network linearization can hopefully popularize the physics-related image quality measures for DL applications. Our methodology is general; it allows flexible compositions of DL nonlinear modules and linear operators such as filtered-backprojection (FBP). For the latter, we develop a generic method for computing the covariance images that is needed for network linearization.

Conditional synthetic food image generation

Generative Adversarial Networks (GAN) have been widely investigated for image synthesis based on their powerful representation learning ability. In this work, we explore the StyleGAN and its application of synthetic food image generation. Despite the impressive performance of GAN for natural image generation, food images suffer from high intra-class diversity and inter-class similarity, resulting in overfitting and visual artifacts for synthetic images. Therefore, we aim to explore the capability and improve the performance of GAN methods for food image generation. Specifically, we first choose StyleGAN3 as the baseline method to generate synthetic food images and analyze the performance. Then, we identify two issues that can cause performance degradation on food images during the training phase: (1) inter-class feature entanglement during multi-food classes training and (2) loss of high-resolution detail during image downsampling. To address both issues, we propose to train one food category at a time to avoid feature entanglement and leverage image patches cropped from high-resolution datasets to retain fine details. We evaluate our method on the Food-101 dataset and show improved quality of generated synthetic food images compared with the baseline. Finally, we demonstrate the great potential of improving the performance of downstream tasks, such as food image classification by including high-quality synthetic training samples in the data augmentation.

A deeper generative adversarial network for grooved cement concrete pavement crack detection

Periodic grooved cement concrete pavement crack detection is of great importance for pavement condition monitoring and maintenance. The current state-of-the-art (SOTA) detection solutions highly depend on datasets. However, due to the limited access to crack images, more efficient methods are urgently needed to advance the detection of cracking on grooved cement concrete pavement. This study proposes an improved deeper Wasserstein generative adversarial network with gradient penalty (WGAN-GP) to generate datasets of pavement images with a size of 512 × 512 pixels 2. Poisson bleeding is adopted to create the synthesized grooved cement concrete pavement crack images based on the generated crack images and groove images. The robustness of the proposed improved deeper WGAN-GP model is validated by Faster R-CNN, YOLOv3, and YOLOv4 models trained on original crack images and generated crack images for region-level detection. U-Net and W-segnet are used to achieve pixel-level crack detection to evaluate the effectiveness of proposed model. Results show that the improved deeper WGAN-GP could generate more realistic transverse, longitudinal and oblique crack images. In addition, the Poisson bleeding algorithm contributes to synthesizing grooved cement concrete pavement crack images. Moreover, it is observed that YOLOv3 trained by the augmented dataset could achieve a mean average precision (MAP) of 81.98%, 6% MAP higher than the non-augmented dataset. U-Net and W-segnet benefit from augmented dataset with a better pixel-level segmentation result. Based on the results, it can be concluded that the improved deeper WAGN-GP image generation method can provide a straightforward way to fill the data shortage gap of grooved cement concrete pavement cracks, thus increasing the problem-solving capability of the SOTA crack detection models.

From Perception to Creation: Exploring Frontier of Image and Video Generation Methods

From Perception to Creation: Exploring Frontier of Image and Video Generation Methods

Malware Image Generation and Detection Method Using DCGANs and Transfer Learning

Malware Image Generation and Detection Method Using DCGANs and Transfer Learning