GAN Model Research Articles

Self-supervised spatial–temporal transformer fusion based federated framework for 4D cardiovascular image segmentation

Availability of high-quality large annotated data is indeed a significant challenge in healthcare. In addition, privacy concerns and data-sharing restrictions often hinder access to large and diverse medical image datasets. To reduce the requirement for annotated training data, self-supervised pre-training strategies on nonannotated data have been extensively used, whereas collaborative algorithm training without the need to exchange the underlying data. In this paper, we introduce a novel federated learning-based self-supervised spatial–temporal transformer’s fusion (SSFL) for cardiovascular image segmentation. The integration of spatial–temporal swin transformer is used to extract the features from 3D SAX multiple phases (full cycle of cardiac heart). An efficient self-supervised contrastive framework consisting of a spatial–temporal transformer network with 25 encoders is used to model the temporal features. The spatial and temporal features are fused and forwarded to the decoder for cardiac heart segmentation using cine MRI images. To further improve segmentation, we use an attention-based unpaired GAN model to map or transfer the style from ACDC to M&Ms and use synthetically generated volumes in the proposed self-supervised approach. Experiments with three different cardiovascular image segmentation tasks, such as segmentation of the right ventricle, left ventricle, and myocardium, showed significant improvement compared to the state-of-the-art segmentation framework.

Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images

Staining is a crucial step in histopathology that prepares tissue sections for microscopic examination. Hematoxylin and eosin (H&E) staining, also known as basic or routine staining, is used in 80% of histopathology slides worldwide. To enhance the histopathology workflow, recent research has focused on integrating generative artificial intelligence and deep learning models. These models have the potential to improve staining accuracy, reduce staining time, and minimize the use of hazardous chemicals, making histopathology a safer and more efficient field. In this study, we introduce a novel three-stage, dual contrastive learning-based, image-to-image generative (DCLGAN) model for virtually applying an "H&E stain" to unstained skin tissue images. The proposed model utilizes a unique learning setting comprising two pairs of generators and discriminators. By employing contrastive learning, our model maximizes the mutual information between traditional H&E-stained and virtually stained H&E patches. Our dataset consists of pairs of unstained and H&E-stained images, scanned with a brightfield microscope at 20 × magnification, providing a comprehensive set of training and testing images for evaluating the efficacy of our proposed model. Two metrics, Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), were used to quantitatively evaluate virtual stained slides. Our analysis revealed that the average FID score between virtually stained and H&E-stained images (80.47) was considerably lower than that between unstained and virtually stained slides (342.01), and unstained and H&E stained (320.4) indicating a similarity virtual and H&E stains. Similarly, the mean KID score between H&E stained and virtually stained images (0.022) was significantly lower than the mean KID score between unstained and H&E stained (0.28) or unstained and virtually stained (0.31) images. In addition, a group of experienced dermatopathologists evaluated traditional and virtually stained images and demonstrated an average agreement of 78.8% and 90.2% for paired and single virtual stained image evaluations, respectively. Our study demonstrates that the proposed three-stage dual contrastive learning-based image-to-image generative model is effective in generating virtual stained images, as indicated by quantified parameters and grader evaluations. In addition, our findings suggest that GAN models have the potential to replace traditional H&E staining, which can reduce both time and environmental impact. This study highlights the promise of virtual staining as a viable alternative to traditional staining techniques in histopathology.

RAMPART: Reinforcing Autonomous Multi-agent Protection through Adversarial Resistance in Transportation

In the field of multi-agent autonomous transportation, such as automated payload delivery or highway on-ramp merging, agents routinely exchange knowledge to optimize their shared objective and adapt to environmental novelties through Cooperative Multi-Agent Reinforcement Learning (CMARL) algorithms. This knowledge exchange between agents allows these systems to operate efficiently and adapt to dynamic environments. However, this cooperative learning process is susceptible to adversarial poisoning attacks, as highlighted by contemporary research. Particularly, the poisoning attacks where malicious agents inject deceptive information camouflaged within the differential noise, a pivotal element for differential privacy (DP)-based CMARL algorithms, pose formidable challenges to identify and overcome. The consequences of not addressing this issue are far-reaching, potentially jeopardizing safety-critical operations and the integrity of data privacy in these applications. Existing research has strived to develop anomaly detection-based defense models to counteract conventional poisoning methods. Nonetheless, the recurring necessity for model offloading and retraining with labeled anomalous data undermines their practicality, considering the inherently dynamic nature of the safety-critical autonomous transportation applications. Further, it is imperative to maintain data privacy, ensure high performance, and adapt to environmental changes. Motivated by these challenges, this paper introduces a novel defense mechanism against stealthy adversarial poisoning attacks in the autonomous transportation domain, termed Reinforcing Autonomous Multi-agent Protection through Adversarial Resistance in Transportation (RAMPART). Leveraging a GAN model at each local node, RAMPART effectively filters out malicious advice in an unsupervised manner, whilst generating synthetic samples for each state-action pair to accommodate environmental uncertainties and eliminate the need for labeled training data. Our extensive experimental analysis, conducted in a Private Payload Delivery Network (PPDN) —a common application in the autonomous multi-agent transportation domain—demonstrates that RAMPART successfully defends against a DP-exploited poisoning attack with a \(30\% \) attack ratio, achieving an F1 score of 0.852 and accuracy of \(96.3\% \) in heavy-traffic environments .

A Research on the Dynamization Effect of Brand Visual Identity Design: Mediated by Digital Information Smart Media

The article utilizes the literature research method, case study method, and practical verification method. The article discusses brand visual identity and motion graphics design principles. The article outlines dynamic brand visual identity design trends that digital information and AI enable. It explains AI generative models like GAN and diffusion models that generate graphics and effects. Examples like Stable Diffusion and Midjourney show AI's potential for diverse, abstract visuals in motion graphics. AI could also enable interactive effects by combining with AR/VR. Overall, AI can empower dynamic, personalized graphic design and branding. Key points are that dynamic design brings interactivity and better conveys brand meaning. Brand visual design is diversifying, with core brand image and dynamic performance reinforcing each other. AI can boost efficiency, innovation, and meaning in dynamic design. Though mainstream, 2D branding remains relevant. The article highlights the future potential of AI in motion graphics and visual storytelling, as it can generate new interpretations and experiences.

ML-Driven Facial Synthesis from Spoken Words Using Conditional GANs

A Human Brain may translate a person's voice to its corresponding face image even if never seen before. Training adeep learning network to do the same can be used in detecting human faces based on their voice, which may be used in findinga criminal that we only have a voice recording for. The goal in this paper is to build a Conditional Generative Adversarial Network that produces face images from human speeches which can then be recognized by a face recognition model to identifythe owner of the speech. The model was trained, and the face recognition model gave an accuracy of 80.08% in training and 56.2% in testing. Compared to the basic GAN model, this model has improved the results by about 30%. Key Word: Face image synthesis, Generative adversarial network, Face Recognition

High‐performance diffusion model for inverse design of high Tc superconductors with effective doping and accurate stoichiometry

AbstractThe pursuit of designing superconductors with high Tc has been a long‐standing endeavor. However, the widespread incorporation of doping in high Tc superconductors significantly impacts electronic structure, intricately influencing Tc. The complex interplay between the structural composition and material performance presents a formidable challenge in superconductor design. Based on a novel generative model, diffusion model, and doping adaptive representation: three‐channel matrix, we have designed a high Tc superconductors inverse design model called Supercon‐Diffusion. It has achieved remarkable success in accurately generating chemical formulas for doped high Tc superconductors. Supercon‐Diffusion is capable of generating superconductors that exhibit high Tc and excels at identifying the optimal doping ratios that yield the peak Tc. The doping effectiveness (55%) and electrical neutrality (55%) of the generated doped superconductors exceed those of traditional GAN models by more than tenfold. Density of state calculations on the structures further confirm the validity of the generated superconductors. Additionally, we have proposed 200 potential high Tc superconductors that have not been documented yet. This groundbreaking contribution effectively reduces the search space for high Tc superconductors. Moreover, it successfully establishes a bridge between the interrelated aspects of composition, structure, and property in superconductors, providing a novel solution for designing other doped materials.image

Empowering robust biometric authentication: The fusion of deep learning and security image analysis

Many societal entities now have more excellent standards for the efficacy and dependability of identification systems due to the ongoing advancement of computer technology. Traditional identification methods, such as keys and smart cards, have been supplanted by biometric systems in highly secure environments. This research presents a smart computational method for automatically authenticating fingerprints for identity (ID) verification and personal identification. Compared to more traditional machine learning algorithms, the results from applying Deep learning (DL) in areas like computer vision, image identification, robotics, and voice processing have generally been positive. Due to their capacity to analyse big data size and deal with fluctuations in biometric data (such as ageing or expression problems), DL has been heavily used by the artificial intelligence research community. Several biometric systems have succeeded with automatic feature extraction employing deep learning approaches like Convolutional Neural Networks (CNNs). In this research, we provide a biometric process that uses convolutional neural networks. This work introduces a deep learning-based biometric identification system that uses Monte Carlo Dropout (MC Dropout). Combining these two systems makes the authentication process more secure and dependable. Fingerprint image enhancement techniques involve the application of Gabor filters and structure-adaptive anisotropic filters, which have proven to be effective in enhancing the clarity and distinctiveness of fingerprint patterns. To improve the efficiency of deep learning models, this work proposes the Inception-Augmentation GAN (IAGAN) model for data augmentation. The study adds to security development by integrating novel biometric identification and authentication approaches with cutting-edge neural network technology. In this research, we provide a new activation function to speed up the convergence of deep neural networks. The results of 99.6% on Gabor filters and 99.8% on the structure-adaptive anisotropic filter with GACNN with MCD show that deep neural networks can excel over competing approaches with enough training data.

Retracted: Cyclic GAN Model to Classify Breast Cancer Data for Pathological Healthcare Task.

[This retracts the article DOI: 10.1155/2022/6336700.].

Intrusion Detection System for In-Vehicle CAN-FD Bus ID Based on GAN Model

Intrusion Detection System for In-Vehicle CAN-FD Bus ID Based on GAN Model

GAN-CNN Ensemble: A Robust Deepfake Detection Model of Social Media Images Using Minimized Catastrophic Forgetting and Generative Replay Technique

Deep-fake photographs are difficult to discern from real ones, especially when utilized in social media platforms. Anyone can willfully create disinformation about public personalities, politicians, and celebrities using these deep fake photographs. So, it is an important need of society to work for an effective model for its detection. The models for deep fake detection commonly use CNN-based detectors. These detectors experience a drop in performance when used for transfer learning or continual learning techniques. A significant limitation in this process is CNN's catastrophe forgetting defect. For the solution of this problem, a Generative replay technique in the form of a GAN-CNN model is implemented that works to minimize this catastrophe forgetting issue that further helps for better detection. It involves generating and storing samples from previous tasks and then replaying them during the training of new tasks which makes CNN more robust to identify deep fakes. The GAN model used in this work is traditional DCGAN improved with necessary adjustments to achieve training stability. It is observed that the model attained a good accuracy of 98.67%(training),70.08% (testing) and minimum loss with a value of 0. 0337 for 100 epochs. Also, it acquired good precision values of 68% and 72%, Recall values are 74% and 66%, and F1 scores of 71% and 69% for classes 0 and 1 respectively. The model outcome is found stable and reliable in deep fake detection under dynamic training conditions. Optimum values of evaluation parameters ensure the model's capacity to learn new tasks preserving the existing task-learning knowledge.

Exploring the Decorative Innovation of Interactive Digital Art in Interior Design under the Guidance of Ecological Concepts

Abstract In this article, we study how interactive digital art is applied in interior design under the ecological concept, focusing on color innovation and image technology to enhance the aesthetics and functionality of interior decoration. The article studies the three elements of color design and their application in product design. The K-Means clustering method is used to classify color imagery words in interior design, and the GAN model is combined to generate interior design colors. The results show that the error qualification rate for the color design scheme coding combination serial number is 88.2%. The composition of interior image decoration systems, digitization processing, and image fusion technology are also a part of this. A method for analyzing the impact of indoor digital decoration based on the ecological concept was proposed after studying the application range of interior decorative colors using cluster analysis. A questionnaire survey was conducted to analyze the perceptual imagery of interior decoration design. Interactive digital art can effectively enhance the aesthetics and functionality of interior design while reaffirming the ecological concept.

Channel Attention GAN-Based Synthetic Weed Generation for Precise Weed Identification.

Weed is a major biological factor causing declines in crop yield. However, widespread herbicide application and indiscriminate weeding with soil disturbance are of great concern because of their environmental impacts. Site-specific weed management (SSWM) refers to a weed management strategy for digital agriculture that results in low energy loss. Deep learning is crucial for developing SSWM, as it distinguishes crops from weeds and identifies weed species. However, this technique requires substantial annotated data, which necessitates expertise in weed science and agronomy. In this study, we present a channel attention mechanism-driven generative adversarial network (CA-GAN) that can generate realistic synthetic weed data. The performance of the model was evaluated using two datasets: the public segmented Plant Seedling Dataset (sPSD), featuring nine common broadleaf weeds from arable land, and the Institute for Sustainable Agro-ecosystem Services (ISAS) dataset, which includes five common summer weeds in Japan. Consequently, the synthetic dataset generated by the proposed CA-GAN obtained an 82.63% recognition accuracy on the sPSD and 93.46% on the ISAS dataset. The Fréchet inception distance (FID) score test measures the similarity between a synthetic and real dataset, and it has been shown to correlate well with human judgments of the quality of synthetic samples. The synthetic dataset achieved a low FID score (20.95 on the sPSD and 24.31 on the ISAS dataset). Overall, the experimental results demonstrated that the proposed method outperformed previous state-of-the-art GAN models in terms of image quality, diversity, and discriminability, making it a promising approach for synthetic agricultural data generation.

Principles of AIGC technology and its application in new media micro-video creation

Abstract AIGC, as a new generation of intelligent generation technology, provides new ideas for the progress and development of the new media field. This study constructs a communication model for new media micro-video creation based on the information communication model, analyzing the communication and development of new media micro-video creation. Combining the application characteristics of AIGC technology in new media micro-video creation and the needs of new media micro-video creation, it proposes the information hiding improvement method based on the GAN model and diffusion model. It constructs the AIGC optimization system based on new media micro-video creation. The model’s application effect in new media micro-video creation is explored after it passes the performance test. The results show that the average scores of all the indicators of the sense of using experience are above 3.5, the overall average score of micro-video creation is 4.12, and the peak traffic and cumulative traffic of the works created by utilizing this method have been improved by 17,000 and 1,910,000, respectively. The efficiency and quality of micro-video creation can be enhanced by using this study as a feasible reference.

MaGAT: Mask-Guided Adversarial Training for Defending Face Editing GAN Models From Proactive Defense

MaGAT: Mask-Guided Adversarial Training for Defending Face Editing GAN Models From Proactive Defense

MissII: Missing Information Imputation for Traffic Data

Cyber-Physical-Social Systems (CPSS) offer a new perspective for applying advanced information technology to improve urban transportation. However, real-world traffic datasets collected from sensing devices like loop sensors often contain corrupted or missing values. The incompleteness of traffic data poses great challenges to downstream data analysis tasks and applications. Most existing data-driven methods only impute missing values based on observed data or hypothetical models, thus ignoring the incorporation of social world information into traffic data imputation. The connection between real-world social activities and CPSS is crucial. In this paper, a novel theory-guided traffic data imputation framework, namely MissII, is proposed. In MissII, we first estimate the traffic flow between two PoIs (Points of Interest) according to spatial interaction theory by considering the physical environment information (e.g., population distributions) and human social interactions (e.g., destination choice game). Moreover, we further refine the estimated traffic flow by considering the effects of road interactions and PoIs. Then, the estimated traffic flow is input into the non-parametric GAN model as real samples to guide the training process. Extensive experiments are conducted on real-world traffic dataset to demonstrate the effectiveness of the proposed framework.

Spatial Steerability of GANs via Self-Supervision from Discriminator.

Generative models make huge progress to the photorealistic image synthesis in recent years. To enable human to steer the image generation process and customize the output, many works explore the interpretable dimensions of the latent space in GANs. Existing methods edit the attributes of the output image such as orientation or color scheme by varying the latent code along certain directions. However, these methods usually require additional human annotations for each pretrained model, and they mostly focus on editing global attributes. In this work, we propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space or requiring extra annotations. Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias. Along with training the GAN model from scratch, these heatmaps are being aligned with the emerging attention of the GAN's discriminator in a self-supervised learning manner. During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects. Moreover, we incorporate DragGAN into our framework, which facilitates fine-grained manipulation within a reasonable time and supports a coarse-to-fine editing process. Extensive experiments show that the proposed method not only enables spatial editing over human faces, animal faces, outdoor scenes, and complicated multi-object indoor scenes but also brings improvement in synthesis quality. Code, models, and demo video are available at https://genforce.github.io/SpatialGAN/.

Few-Shot Image Generation via Style Adaptation and Content Preservation.

Training a generative model with limited data (e.g., 10) is a very challenging task. Many works propose to fine-tune a pretrained GAN model. However, this can easily result in overfitting. In other words, they manage to adapt the style but fail to preserve the content, where style denotes the specific properties that define a domain while content denotes the domain-irrelevant information that represents diversity. Recent works try to maintain a predefined correspondence to preserve the content, however, the diversity is still not enough and it may affect style adaptation. In this work, we propose a paired image reconstruction approach for content preservation. We propose to introduce an image translation module to GAN transferring, where the module teaches the generator to separate style and content, and the generator provides training data to the translation module in return. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in a few-shot setting.

Research on Pricing Methods of Convertible Bonds Based on Deep Learning GAN Models

This paper proposes two data-driven models (including LSTM pricing model, WGAN pricing model) and an improved model of LSM based on GAN to analyze the pricing of convertible bonds. In addition, the LSM model with higher precision in traditional pricing model is selected for comparative study with other pricing models. It is found that the traditional LSM pricing model has a large error in the first-day pricing, and the pricing function needs to be further improved. Among the four pricing models, LSTM pricing model and WGAN pricing model have the best pricing effect. The WGAN pricing model is better than the LSTM pricing model (0.21%), and the LSM improved model (1.17%) is better than the traditional LSM model (2.26%). Applying the generative deep learning model GAN to the pricing of convertible bonds can circumvent the harsh preconditions of assumptions, and significantly improve the pricing effect of the traditional model. The scope of application of each model is different. Therefore, this paper proves the feasibility of the GAN model applied to the pricing of convertible bonds, and enriches the pricing function of derivatives in the financial field.

A two-stage model extraction attack on GANs with a small collected dataset

Due to their capacity for image generation, GAN models may be considered as a solution for the use of private data, which enhances their commercial value. However, unlike discriminative models such as CNNs, extraction attacks on GANs have not received significant attention, with only a few relevant works available. This paper proposes a novel two-stage extraction attack on GANs that does not require access to the victim GAN model. In the first stage, data augmentation is performed using GAN inversion techniques and vector arithmetic, tailored to scenarios with small query budgets. The second stage comprises two types of training: the innocent transfer training on unprocessed collected samples and the additional training on the augmented set. We provide three options for additional training: training of the discriminator alone (D), training of the generator alone (G), and training of both (D+G). We then conduct a quantity and quality evaluation of our extraction attacks compared to the baseline attack. Experiment results demonstrate that our proposed approach can successfully extract a GAN model, where the distribution of the extracted GAN closely resembles the distribution of the target GAN, even with a hundred generated images of the target GAN. Specifically, we achieve stable performance with better extraction, a more disentangled latent space for the extracted GAN, and significant semantic attribute editing performance using the two-stage extraction with additional training of the discriminator (D). By demonstrating that even a hundred leaked images may result in severe extraction, this study raises serious concerns regarding GAN model leakage.

A hybrid Cycle GAN-based lightweight road perception pipeline for road dataset generation for Urban mobility.

One of the major problems that cause continual trouble in deep learning networks is that training a large network requires massive labelled datasets. The preparation of a massive labelled dataset is a cumbersome task and requires lot of human interventions. This paper proposes a novel generator network 'Sim2Real' transfer is a recent and fast-developing field in machine learning used to bridge the gap between simulated and real data. Training with simulated datasets often converges due to its size but fails to generalize real-world applications. Simulated datasets can be used to train and test deep learning models, enables the development and evaluation of new algorithms and architectures. By simulating road dataset, researchers can generate large amounts of realistic road-traffic dataset that can be used to study and understand several problems such as vehicular object tracking and classification, traffic situation analysis etc. The main advantage of such a transfer algorithm is to use the abundance of a simulated dataset to generate huge realistic-looking datasets to solve data-intense tasks. This work presents a novel, robust sim2real algorithm that converts the labels of a semantic segmentation map to a realistic-looking street view using the Cityscapes dataset and aims to achieve robust urban mobility for smart cities. Further, the generalizability of the Cycle Generative Adversarial Network (CycleGAN) architecture was tested by using an origami robot dataset for sim2real transfer. We show that the results were found to be qualitatively satisfactory for different traffic analysis applications. In addition, road perception was done using a lightweight SVM pipeline and evaluated on the KITTI dataset. We have incorporated Cycle Consistency Loss and Identity Loss as the metrics to evaluate the performance of the proposed Cycle GAN model. We inferred that the proposed Cycle GAN model provides an Identity loss of less than 0.2 in both the Cityscapes dataset and KITTI datasets. Also, we understand that the super-pixel resolution has a good impact on the quantitative results of the proposed Cycle GAN models.