Self-attention Generative Adversarial Network Research Articles

Enhanced conditional self-attention generative adversarial network for detecting cotton plant disease in IoT-enabled crop management

Enhanced conditional self-attention generative adversarial network for detecting cotton plant disease in IoT-enabled crop management

Application of Self-Attention Generative Adversarial Network for Electromagnetic Imaging in Half-Space.

In this paper, we introduce a novel artificial intelligence technique with an attention mechanism for half-space electromagnetic imaging. A dielectric object in half-space is illuminated by TM (transverse magnetic) waves. Since measurements can only be made in the upper space, the measurement angle will be limited. As a result, we apply a back-propagation scheme (BPS) to generate an initial guessed image from the measured scattered fields for scatterer buried in the lower half-space. This process can effectively reduce the high nonlinearity of the inverse scattering problem. We further input the guessed images into the generative adversarial network (GAN) and the self-attention generative adversarial network (SAGAN), respectively, to compare the reconstruction performance. Numerical results prove that both SAGAN and GAN can reconstruct dielectric objects and the MNIST dataset under same measurement conditions. Our analysis also reveals that SAGAN is able to reconstruct electromagnetic images more accurately and efficiently than GAN.

Social Self-Attention Generative Adversarial Networks for Human Trajectory Prediction

Social Self-Attention Generative Adversarial Networks for Human Trajectory Prediction

Securing synthetic faces: A GAN-blockchain approach to privacy-enhanced facial recognition

In recent years, facial recognition technology has become increasingly integrated into society, making privacy protection crucial. Previous techniques offered minimal secrecy safeguards through simple obscuration methods. This paper addresses the strict privacy requirements of face image data by developing a novel framework that synergistically integrates Generative Adversarial Networks (GANs), clustering algorithms, and Blockchain technology. The methodology proposes a cutting-edge Privacy-Preserving Self-Attention GAN (PPSA-GAN) to generate realistic synthetic facial imagery. An integrated mini-batch K-means clustering algorithm anonymizes these images into distinct groupings, maximizing privacy preservation. Blockchain integration complements the system by fortifying trust through decentralized ledgers for transparent yet secure data storage and auditing. Rigorous benchmarking on the CelebA dataset confirms the PPSA-GAN architecture’s state-of-the-art performance, attaining an impressive Inception Score of 13.99 and a Fréchet Inception Distance of 35.50. The mini-batch clustering forms 125 distinct clusters, effectively anonymizing facial attributes within the synthetic images. Blockchain integration further bolsters privacy assurances via tamper-proof historical records, showcasing precision, recall, F1-score, and accuracy values of 0.948, 0.938, 0.943, and 0.947, respectively. This multifunctional framework represents a novel contribution, fostering an ethical technological ecosystem that balances progress and privacy. Prospective deployment horizons encompass identity verification, surveillance infrastructure, and augmentation of medical image repositories, seeding an enlightening future for facial recognition domains.

Unsupervised unpaired multiple fusion adaptation aided with self-attention generative adversarial network for scar tissues segmentation framework

Late gadolinium enhancement (LGE) is a specialized imaging technique used in cardiovascular magnetic resonance (CMR) imaging to detect and characterize areas of scar tissue or fibrosis within the heart muscle. After a heart attack, the affected region of the heart muscle becomes scarred due to insufficient blood supply. LGE CMR enhances the infarcted myocardium to appear with distinctive brightness compared to healthy tissues. It is primarily used to assess various heart conditions and can identify the extent and location of scar tissue, helping in risk stratification and guiding treatment decisions. Automated myocardium segmentation in LGE CMR poses challenges due to intensity heterogeneity caused by the accumulation of contrast agents in infarcted areas. Furthermore, LGE CMR images with gold standard labels are particularly scarce compared to other sequences, making the task even more demanding. Besides, the heart’s motion involves complex non-rigid deformations, including regional wall thickening, circumferential, and longitudinal ventricular shortening, making it challenging to model. This work presents an unsupervised unpaired domain adaptive scar tissue segmentation framework aided with the self-attention generative adversarial network. Besides, we proposed a novel segmentation architecture based on a parallel transformer for scar segmentation and compared the results with and without style transfer GAN and unpaired style self-supervised attention GAN. Besides, to model longitudinal changes in scare, we used eight days’ time points as a source and 1 month and 12 as a target and applied our proposed attention-generative network to transfer style from one-time point to another time. Experiments on four different LGE MICCAI datasets from 2019 to 2023 achieved significantly better performance than state-of-the-art segmentation methods.

Self-Attention Generative Adversarial Network Interpolating and Denoising Seismic Signals Simultaneously

In light of the challenging conditions of exploration environments coupled with escalating exploration expenses, seismic data acquisition frequently entails the capturing of signals entangled amidst diverse noise interferences and instances of data loss. The unprocessed state of these seismic signals significantly jeopardizes the interpretative phase. Evidently, the integration of attention mechanisms and the utilization of generative adversarial networks (GANs) have emerged as prominent techniques within signal processing owing to their adeptness in discerning intricate global dependencies. Our research introduces a pioneering approach for reconstructing and denoising seismic signals, amalgamating the principles of self-attention and generative adversarial networks—hereafter referred to as SAGAN. Notably, the incorporation of the self-attention mechanism into the GAN framework facilitates an enhanced capacity for both the generator and discriminator to emulate meaningful spatial interactions. Subsequently, leveraging the feature map generated by the self-attention mechanism within the GAN structure enables the interpolation and denoising of seismic signals. Rigorous experimentation substantiates the efficacy of SAGAN in simultaneous signal interpolation and denoising. Initially, we benchmarked SAGAN against prominent methods such as UNet, CNN, and Wavelet for the concurrent interpolation and denoising of two-dimensional seismic signals manifesting varying levels of damage. Subsequently, this methodology was extended to encompass three-dimensional seismic data. Notably, performance metrics reveal SAGAN’s superiority over comparative methods. Specifically, the quantitative tables exhibit SAGAN’s pronounced advantage, with a 3.46% increase in PSNR value over UNet and an impressive 11.90% surge compared to Wavelet. Moreover, the RMSE values affirm SAGAN’s robust performance, showcasing an 11.54% reduction in comparison to UNet and an impressive 29.27% decrement relative to Wavelet, hence unequivocally establishing the SAGAN method as a preeminent choice for seismic signal recovery.

Enhancing MIMO-OFDM channel estimation in 5G and beyond with conditional self-attention generative adversarial networks

Enhancing MIMO-OFDM channel estimation in 5G and beyond with conditional self-attention generative adversarial networks

SASEGAN-TCN: Speech enhancement algorithm based on self-attention generative adversarial network and temporal convolutional network.

Traditional unsupervised speech enhancement models often have problems such as non-aggregation of input feature information, which will introduce additional noise during training, thereby reducing the quality of the speech signal. In order to solve the above problems, this paper analyzed the impact of problems such as non-aggregation of input speech feature information on its performance. Moreover, this article introduced a temporal convolutional neural network and proposed a SASEGAN-TCN speech enhancement model, which captured local features information and aggregated global feature information to improve model effect and training stability. The simulation experiment results showed that the model can achieve 2.1636 and 92.78% in perceptual evaluation of speech quality (PESQ) score and short-time objective intelligibility (STOI) on the Valentini dataset, and can accordingly reach 1.8077 and 83.54% on the THCHS30 dataset. In addition, this article used the enhanced speech data for the acoustic model to verify the recognition accuracy. The speech recognition error rate was reduced by 17.4%, which was a significant improvement compared to the baseline model experimental results.

Advancing financial fraud detection: Self-attention generative adversarial networks for precise and effective identification

The burgeoning prominence of electronic payment methods has significantly intensified the importance of financial fraud detection, especially concerning credit card fraud. The challenges encompass data imbalance, overfitting, and a limited capacity to realistically emulate the complexity and dynamics of fraudulent activities. This paper introduces self-attention generative adversarial networks (SAGANs) for the detection of credit card fraud. Leveraging self-attention mechanisms, SAGANs can distinguish salient features and patterns within extensive transaction datasets, thereby fostering a more profound understanding and refined identification of credit card fraud. By incorporating generative adversarial networks (GANs), our model is capable of generating data that mirrors actual fraudulent behavior, which consequently enhances and optimizes fraud detection algorithms. Empirical findings from numerous credit card transaction datasets underscore the considerable superiority of our approach in terms of accuracy and recall. Our proposed SAGANs more competently emulate the complexity and diversity inherent in fraudulent behavior, thereby augmenting both the precision and efficacy of fraud detection.

Wavelet-Based Self-Attention GAN With Collaborative Feature Fusion for Image Inpainting

Image inpainting is a significant task in the applications of computer vision, that aims to fill in damaged regions with visually realistic contents. With the development of deep learning, generative adversarial network (GAN)-based image inpainting approaches have achieved remarkable progress. However, these methods only utilize one-sided structure information to assist inpainting, which can not achieve satisfactory results, especially when synthesizing large-area missing complex images. In order to tackle this problem, a wavelet-based self-attention GAN (WSA-GAN) with collaborative feature fusion is proposed, which is embedded with a wavelet-based self-attention (WSA) and a collaborative feature fusion (CFF). The WSA is designed to conduct long-range dependence among multi-scale frequency information to highlight significant structure details for better generating texture boundaries. The CFF is presented to couple channel-guided space and space-affected channel streams to facilitate the interaction of spatial and channel features, which can effectively avoid potential domain conflicts. Besides, a novel wavelet consistency loss and a hierarchical pyramid feature matching (PFM) discriminator are introduced to stabilize the model training. Extensive experiments on three public datasets, including Paris StreetView, CelebA-HQ and Places, demonstrate that the proposed method outperforms the state-of-the-art methods both quantitatively and qualitatively.

BaMSGAN: Self-Attention Generative Adversarial Network with Blur and Memory for Anime Face Generation

In this paper, we propose a novel network, self-attention generative adversarial network with blur and memory (BaMSGAN), for generating anime faces with improved clarity and faster convergence while retaining the capacity for continuous learning. Traditional self-attention generative adversarial networks (SAGANs) produce anime faces of higher quality compared to deep convolutional generative adversarial networks (DCGANs); however, some edges remain blurry and distorted, and the generation speed is sluggish. Additionally, common issues hinder the model’s ability to learn continuously. To address these challenges, we introduce a blurring preprocessing step on a portion of the training dataset, which is then fed to the discriminator as fake data to encourage the model to avoid blurry edges. Furthermore, we incorporate regulation into the optimizer to mitigate mode collapse. Additionally, memory data stored in the memory repository is presented to the model every epoch to alleviate catastrophic forgetting, thereby enhancing performance throughout the training process. Experimental results demonstrate that BaMSGAN outperforms prior work in anime face generation, significantly reducing distortion rates and accelerating shape convergence.

Self-Attention Generative Adversarial Network Enhanced Learning Method for Resilient Defense of Networked Microgrids Against Sequential Events

The resilient responses of networked microgrids (MGs) can greatly improve the survival of critical loads during extreme events. In order to efficiently handle the scarce data issue as well as improve the adaptability of deep reinforcement learning (DRL) methods for complex sequential extreme events (SEEs) such as hurricanes and tornadoes, a new learning-based method is proposed for the survival of critical loads in MGs during SEEs. A generative adversarial network (GAN) is applied to generate a sufficient extreme event-related database in a model-free way. Specifically, a self-attention GAN (SA-GAN) is developed to capture sequential features of the SEE process. Then, the SA-GAN is integrated into a DRL framework, and the corresponding Markov decision process (MDP) and the environment are designed to realize adaptive networked MG reconfiguration for the survival of critical loads. Faced with uncertain distributed generator (DG) output and sequential line damage, the SA-GAN-DRL method provides an adaptive model-free solution to continuously supply critical loads during SEEs. The effectiveness of the proposed method is validated using a 7-bus test system and the IEEE 123-bus system, and the results demonstrate both a strong learning ability with limited practice data, and robustness and adaptability for highly changeable SEE processes.

CervixFormer: A Multi-scale swin transformer-Based cervical pap-Smear WSI classification framework

Background and ObjectivesCervical cancer affects around 0.5 million women per year, resulting in over 0.3 million fatalities. Therefore, repetitive screening for cervical cancer is of utmost importance. Computer-assisted diagnosis is key for scaling up cervical cancer screening. Current recognition algorithms, however, perform poorly on the whole-slide image (WSI) analysis, fail to generalize for different staining methods and on uneven distribution for subtype imaging, and provide sub-optimal clinical-level interpretations. Herein, we developed CervixFormer—an end-to-end, multi-scale swin transformer-based adversarial ensemble learning framework to assess pre-cancerous and cancer-specific cervical malignant lesions on WSIs. MethodsThe proposed framework consists of (1) a self-attention generative adversarial network (SAGAN) for generating synthetic images during patch-level training to address the class imbalanced problems; (2) a multi-scale transformer-based ensemble learning method for cell identification at various stages, including atypical squamous cells (ASC) and atypical squamous cells of undetermined significance (ASCUS), which have not been demonstrated in previous studies; and (3) a fusion model for concatenating ensemble-based results and producing final outcomes. ResultsIn the evaluation, the proposed method is first evaluated on a private dataset of 717 annotated samples from six classes, obtaining a high recall and precision of 0.940 and 0.934, respectively, in roughly 1.2 minutes. To further examine the generalizability of CervixFormer, we evaluated it on four independent, publicly available datasets, namely, the CRIC cervix, Mendeley LBC, SIPaKMeD Pap Smear, and Cervix93 Extended Depth of Field image datasets. CervixFormer obtained a fairly better performance on two-, three-, four-, and six-class classification of smear- and cell-level datasets. For clinical interpretation, we used GradCAM to visualize a coarse localization map, highlighting important regions in the WSI. Notably, CervixFormer extracts feature mostly from the cell nucleus and partially from the cytoplasm. ConclusionsIn comparison with the existing state-of-the-art benchmark methods, the CervixFormer outperforms them in terms of recall, accuracy, and computing time.

Reduction of rain effect on wave height estimation from marine X-band radar images using unsupervised generative adversarial networks

ABSTRACT An intelligent single radar image de-raining method based on unsupervised self-attention generative adversarial networks is proposed to improve the accuracy of wave height parameter inversion results. The method builds a trainable end-to-end de-raining model with an unsupervised cycle-consistent adversarial network as an AI framework, which does not require pairs of rain-contaminated and corresponding ground-truth rain-free images for training. The model is trained by feeding rain-contaminated and clean radar images in an unpaired manner, and the atmospheric scattering model parameters are not required as a prior condition. Additionally, a self-attention mechanism is introduced into the model, allowing it to focus on rain clutter when processing radar images. This combines global and local rain clutter context information to output more accurate and clear de-raining radar images. The proposed method is validated by applying it to actual field test data, which shows that compared with the wave height derived from the original rain-contaminated data, the root-mean-square error is reduced by 0.11 m and the correlation coefficient of the wave height is increased by 14% using the de-raining method. These results demonstrate that the method effectively reduces the impact of rain on the accuracy of wave height parameter estimation from marine X-band radar images.

Self-attention generative adversarial capsule network optimized with atomic orbital search algorithm based sentiment analysis for online product recommendation

Self-Attention based Generative Adversarial Capsule Network optimized with Atomic orbital search algorithm based Sentiment Analysis is proposed in this manuscript for Online Product Recommendation (SFA-AGCN-AOSA-SA-OPR). Here, Collaborative filtering (CF) and product-product (P-P) similarity method is utilized for designing the new recommendation system. CF is employed for predicting the best shops and P-P similarity method is employed to predict the better product. Initially, the datas are gathered via Amazon Product recommendation dataset. After that, the datas are given to pre-processing. During pre-processing, Markov chain random field (MCRF) co-simulation is used to remove the unwanted content and filtering relevant text. The preprocessing output is fed to feature extraction. The features, like manufacturing date, Manufacturing price, discounts, offers, quality ratings, and suggestions or reviews are extracted using Gray level co-occurrence matrix (GLCM) window adaptive algorithm based feature extraction method. Finally, Self-Attention based Generative Adversarial Capsule Network (SFA-AGCN) categorizes the product recommendation as excellent, good, very good, bad, very bad. Atomic orbital search algorithm optimizes the SFA-AGCN weight parameters. The performance metrics, like accuracy, precision, sensitivity, recall, F-measure, mean absolute error (MAE), mean squared error (MSE), mean absolute percentage error (MAPE) is examined. The efficiency of the proposed method provides higher mean absolute percentage error 98.23%, 88.34%, 90.35% and 78.96% and lower Mean squared error 92.15%, 90.25%, 89.64% and 92.48% compared to the existing methods, such as sentiment analysis of online product reviews using DLMNN and future prediction of online product using IANFIS (DLMNN-IANFIS-SA-OPR), intelligent sentiment analysis approach using edge computing based deep learning technique (DCNN-SA-OPR), sentiment analysis for online product reviews in Chinese depending on sentiment lexicon and deep learning (CNN-BiGRU-SA-OPR) and sentiment analysis on product reviews depending on weighted word embedding and deep neural networks (CNN-LSTM-SA-OPR) respectively.

Digital core image reconstruction based on residual self-attention generative adversarial networks

Digital core image reconstruction based on residual self-attention generative adversarial networks

Small-Sample Seabed Sediment Classification Based on Deep Learning

Seabed sediment classification is of great significance in acoustic remote sensing. To accurately classify seabed sediments, big data are needed to train the classifier. However, acquiring seabed sediment information is expensive and time-consuming, which makes it crucial to design a well-performing classifier using small-sample seabed sediment data. To avoid data shortage, a self-attention generative adversarial network (SAGAN) was trained for data augmentation in this study. SAGAN consists of a generator, which generates data similar to the real image, and a discriminator, which distinguishes whether the image is real or generated. Furthermore, a new classifier for seabed sediment based on self-attention densely connected convolutional network (SADenseNet) is proposed to improve the classification accuracy of seabed sediment. The SADenseNet was trained using augmented images to improve the classification performance. The self-attention mechanism can scan the global image to obtain global features of the sediment image and is able to highlight key regions, improving the efficiency and accuracy of visual information processing. The proposed SADenseNet trained with the augmented dataset had the best performance, with classification accuracies of 92.31%, 95.72%, 97.85%, and 95.28% for rock, sand, mud, and overall, respectively, with a kappa coefficient of 0.934. The twelve classifiers trained with the augmented dataset improved the classification accuracy by 2.25%, 5.12%, 0.97%, and 2.64% for rock, sand, mud, and overall, respectively, and the kappa coefficient by 0.041 compared to the original dataset. In this study, SAGAN can enrich the features of the data, which makes the trained classification networks have better generalization. Compared with the state-of-the-art classifiers, the proposed SADenseNet has better classification performance.

Unsupervised self-attention lightweight photo-to-sketch synthesis with feature maps

Face-sketch synthesis is important for gaining a clear portrait photo of suspects when solving crimes. Recent research has made a great process in self-attention generative adversarial networks. We propose a method of unsupervised learning in the synthesis of face sketch-to-photo using a new attention module. The method of processing on a small reference set of photo-sketch pairs adds to the attention module, a focus on the regions distinguishing photos from sketches on the basis of the feature maps obtained by the auxiliary classifier. Unlike previous attention-based methods, which cannot handle the geometric changes between domains, our model can translate images requiring holistic changes. At the same time, we reduce the layers of the discriminator according to different residual layers to optimize our network. With the proposed approach, we can train our networks using a small reference set of photo-sketch pairs together with a large number of face-photo datasets and more distinguishing facial-feature regions in the self-attention model. Experiments have shown the superiority of the proposed method to existing face sketch-to-photo synthesis models using fixed network architectures and hyper-parameters.

A pavement crack synthesis method based on conditional generative adversarial networks.

A Generative Adversarial Network (GAN) based asphalt pavement crack image generation method was proposed to improve the dataset size of the road images. Five open-source road crack datasets were leveraged to construct an image dataset, which contained two labels - transverse cracks and longitudinal cracks. The constructed dataset was used to facilitate crack detection and classification research by providing a diverse collection of labeled crack images derived from multiple public sources. The network structure of fully connected, convolutional and attention mechanisms based on the Conditional Generative Adversarial Network (CGAN) was used in this project. The purpose of this study was to train a generative model on selected categories of input pavement crack images and generate realistic crack images of those categories. We aim to tune the parameters of the GAN and optimize hyperparameters to improve the realism possibility of generated images. It also explored the generated images with different sizes and evaluated the performance of networks with different architectures. In particular, we analyzed the structural characteristics of conditional GAN. Results demonstrated that the Self-Attention Generative Adversarial Networks (SAGAN) model, which combines self-attention mechanisms with CGAN, can effectively address challenges related to limited crack image data and the inability to selectively generate images from specific categories. By conditioning the generator on category information, the SAGAN model was able to generate high-quality images while focusing on the target categories. Overall, the self-attention and conditional aspects of the SAGAN framework helped improve the generation of realistic pavement crack images.

Multi-Head Self-Attention Generative Adversarial Networks for Multiphysics Topology Optimization

Machine learning surrogates for topology optimization must generalize well to a large variety of boundary conditions and volume fractions to serve as a stand-alone model. However, when analyzing design performance using physics-based analysis, many of the recently published methods suffer from low reliability, with a high percentage of the generated structures performing poorly. Disconnected regions of solid material between boundary conditions lead to unstable designs with significant outliers skewing the performance on test data. In this work, multi-head self-attention generative adversarial networks are introduced as a novel architecture for multiphysics topology optimization. This network contains multi-head attention mechanisms in high-dimensional feature spaces to learn the global dependencies of data (i.e., connectivity between boundary conditions). The model is demonstrated on design of coupled thermoelastic structures and its performance is evaluated with respect to the physics-based objective function used to generate training data. The proposed network achieves over a 36 times reduction in mean objective function error and an eight times reduction in volume fraction error compared to a baseline approach without attention mechanisms.