Attentive Generative Adversarial Network Research Articles

Specular highlight removal by federated generative adversarial network with attention mechanism

Specular highlight removal ensures the acquisition of high-quality images, which finds its important applications in stereo matching, text recognition and image segmentation. In order to prevent the leakage of images containing personal information, such as identification card (ID) photos, clients often train specular highlight removal models using local data resulting in a lack of precision and generalization of the trained model. To address this challenge, this paper introduces a new method to remove highlight in images using federated learning (FL) and attention generative adversarial network (AttGAN). Specifically, the former builds a global model in the central server and updates the global model by aggregating model parameters of clients. This process does not involve the transmission of image data, which enhances the privacy of clients; the later combining attention mechanisms and generative adversarial network aims to improve the quality of highlight removal by focusing on key image regions, resulting in more realistic and visually pleasing results. The proposed FL-AttGAN method is numerically evaluated, using SD1, SD2 amd RD datasets. The results show that the proposed FL-AttGAN outperforms existent methods.

SRARDA: A lightweight adaptive residual dense attention generative adversarial network for image super-resolution

Image super-resolution (SR) is the task of inferring a high resolution (HR) image from one/multiple single low resolution (LR) input(s). Traditional networks are evaluated by pixel-level metrics such as Peak-Signal-to-Noise Ratio (PSNR) etc., which do not always align with human perception of image quality. They often produce excessively smooth images that lack high-frequency texture and appear unnatural. Therefore, in this paper, we propose a lightweight adaptive residual dense attention generative adversarial network (SRARDA) for image SR. Firstly, our generator adopts the residual in residual (RIR) structure but redesigns the basic module. By using dynamic residual connection (ARC) to dynamically adjust the importance of residual and main paths, we design a novel adaptive residual dense attention block (ARDAB) that enhances the feature extraction capability of the generator. In addition, we build a high-frequency filtering unit (HFU) to extract more high-frequency features from the LR space. Finally, to fully utilize the discriminator, we use WGAN to compute the difference between the HR image and the reconstructed image. Experiments demonstrate that SRARDA effectively addresses the issue of excessive smoothing in reconstructed images, while also enhancing visual quality.

Multi Attention Generative Adversarial Network for Pedestrian Trajectory Prediction Based on Spatial Gridding

Multi Attention Generative Adversarial Network for Pedestrian Trajectory Prediction Based on Spatial Gridding

AU3-GAN: A Method for Extracting Roads from Historical Maps Based on an Attention Generative Adversarial Network

In recent years, the integration of deep learning technology based on convolutional neural networks with historical maps has made it possible to automatically extract roads from these maps, which is highly important for studying the evolution of transportation networks. However, the similarity between roads and other features (such as contours, water systems, and administrative boundaries) poses a significant challenge to the feature extraction capabilities of convolutional neural networks (CNN). Additionally, CNN require a large quantity of labelled data for training, which can be a complex issue for historical maps. To address these limitations, we propose a method for extracting roads from historical maps based on an attention generative adversarial network. This approach leverages the unique architecture and training methodology of the generative adversarial network to augment datasets by generating data that closely resembles real samples. Meanwhile, we introduce an attention mechanism to enhance UNet3 + and achieve accurate historical map road segmentation images. We validate our method using the Third Military Mapping Survey of Austria-Hungary and compare it with a typical U-shaped network. The experimental results show that our proposed method outperforms the direct use of the U-shaped network, achieving at least an 18.26% increase in F1 and a 7.62% increase in the MIoU, demonstrating its strong ability to extract roads from historical maps and provide a valuable reference for road extraction from other types of historical maps.

An Intelligent Deep Learning Framework for Traffic Flow Imputation and Short-term Prediction Based on Dynamic Features

The accurate prediction of traffic flow has emerged as a focal point in the cutting-edge sphere of intelligent transportation. Extant methodologies rely on deep learning for short-term forecasts but are hampered by two critical limitations: the omission of vital traffic flow data and an overemphasis on static rather than dynamic attributes. This paper presents a solution to these challenges through the creation of a dynamic attention generative adversarial network (DATGAN) that is specifically engineered to impute missing traffic data accurately. DATGAN introduces two groundbreaking features: first, an innovative attention module that is adept at identifying local correlations, and second, novel spatial-correction graph convolutional layers that are designed to capture the ever-changing spatial dynamic dependencies. Building on this foundation, the parallel spatial–temporal Transformer (PSTTransformer) is proposed, which integrates both spatial and temporal attention blocks to manage dynamic spatial-temporal dependencies for future short-term traffic predictions. This dual-block approach not only embeds periodic information via a fusion self-attention mechanism in the temporal attention blocks, but also synergizes graph convolutional networks with self-attention mechanisms in the spatial attention blocks through a gate mechanism. A comprehensive comparative analysis conducted on traffic datasets revealed that our methodology outperforms current leading models by reducing the mean squared error by 5% in data imputation tasks and shows robust performance in traffic prediction scenarios.

Reconstructing images with attention generative adversarial network against adversarial attacks

Reconstructing images with attention generative adversarial network against adversarial attacks

Di-GraphGAN: An enhanced adversarial learning framework for accurate spatial-temporal traffic forecasting under data missing scenarios

Nowadays, various disturbances in urban transportation data acquisition/processing/storage lead to the inevitable data missing problem, which undermines the valuable traffic information and greatly threats the reliability of existing benchmark traffic prediction models. Inspired from the powerful generative learning ability of GANs, we propose an integrated spatiotemporal Data imputation Graph Attention Generative Adversarial Networks (Di-GraphGAN) for accurate and efficient spatial-temporal traffic forecasting under data missing scenarios. Specifically, we first propose a traffic data imputation module named DI-LSTM, which adopts the architecture of LSTM Network with an extra Time Damping unit to accurately estimating the missing values. Then, we facilitate Di-GraphGAN with an original developed Task-Efficient Graph Attention Networks (TE-GAT) for better graph representation learning and a Temporal Contextual Attention (TCA) mechanism to capture the dynamic spatiotemporal traffic patterns. Finally, extensive evaluations are conducted on two real-world traffic speed datasets from China, demonstrating that Di-GraphGAN achieves state-of-the-art performance in both traffic forecasting and spatiotemporal data imputation tasks.

3D EAGAN: 3D edge-aware attention generative adversarial network for prostate segmentation in transrectal ultrasound images

3D EAGAN: 3D edge-aware attention generative adversarial network for prostate segmentation in transrectal ultrasound images

A Residual Dense Attention Generative Adversarial Network for Microscopic Image Super-Resolution.

With the development of deep learning, the Super-Resolution (SR) reconstruction of microscopic images has improved significantly. However, the scarcity of microscopic images for training, the underutilization of hierarchical features in original Low-Resolution (LR) images, and the high-frequency noise unrelated with the image structure generated during the reconstruction process are still challenges in the Single Image Super-Resolution (SISR) field. Faced with these issues, we first collected sufficient microscopic images through Motic, a company engaged in the design and production of optical and digital microscopes, to establish a dataset. Secondly, we proposed a Residual Dense Attention Generative Adversarial Network (RDAGAN). The network comprises a generator, an image discriminator, and a feature discriminator. The generator includes a Residual Dense Block (RDB) and a Convolutional Block Attention Module (CBAM), focusing on extracting the hierarchical features of the original LR image. Simultaneously, the added feature discriminator enables the network to generate high-frequency features pertinent to the image's structure. Finally, we conducted experimental analysis and compared our model with six classic models. Compared with the best model, our model improved PSNR and SSIM by about 1.5 dB and 0.2, respectively.

Unsupervised image-to-image translation with multiscale attention generative adversarial network

Unsupervised image-to-image translation with multiscale attention generative adversarial network

A Stage-Wise Residual Attention Generation Adversarial Network for Mandibular Defect Repairing and Reconstruction.

Surgical reconstruction of mandibular defects is a clinical routine manner for the rehabilitation of patients with deformities. The mandible plays a crucial role in maintaining the facial contour and ensuring the speech and mastication functions. The repairing and reconstruction of mandible defects is a significant yet challenging task in oral-maxillofacial surgery. Currently, the mainly available methods are traditional digitalized design methods that suffer from substantial artificial operations, limited applicability and high reconstruction error rates. An automated, precise, and individualized method is imperative for maxillofacial surgeons. In this paper, we propose a Stage-wise Residual Attention Generative Adversarial Network (SRA-GAN) for mandibular defect reconstruction. Specifically, we design a stage-wise residual attention mechanism for generator to enhance the extraction capability of mandibular remote spatial information, making it adaptable to various defects. For the discriminator, we propose a multi-field perceptual network, consisting of two parallel discriminators with different perceptual fields, to reduce the cumulative reconstruction errors. Furthermore, we design a self-encoder perceptual loss function to ensure the correctness of mandibular anatomical structures. The experimental results on a novel custom-built mandibular defect dataset demonstrate that our method has a promising prospect in clinical application, achieving the best Dice Similarity Coefficient (DSC) of 94.238% and 95% Hausdorff Distance (HD95) of 4.787.

Fused Attentive Generative Adversarial Network based Personalized English Learning Material Recommendation System utilizing Knowledge Graph

The information era has arrived swiftly in the world. Because it is the universal language, English influences how information is transmitted and exchanged throughout the world. Learning English is like having a valuable instrument for learning priceless knowledge. For this reason, it is imperative that English instruction be improved. The Fused Attentive Generative Adversarial Network based Personalized English Learning Material Recommendation System utilizing Knowledge Graph. Initially, input data are collected, given to preprocessing method. In preprocessing remove data redundancy using Generalized Correntropy Sparse Gauss–Hermite Quadrature Filter (GCSG-HQF). In pre-processed output fed to Recommendation System. Here, FAGAN is used to recommend English Learning Material. In general, FAGAN classifier does not express any optimization adaption approaches for determining optimum parameters to assure the Knowledge Graph. Here Clouded Leopard Optimization Algorithm utilized to optimize FAGAN classifier, for Personalized English Learning Material Recommendation System. In Clouded Leopard Optimization Algorithm utilized to optimise weight parameter of the FAGAN. With the use of performance metrics likes accuracy, precision, sensitivity, specificity, F1-score, ROC, computational time are analyzed. The proposed FAGAN-ELMRS-CLOA method attains 30.58%, 28.73% and 35.62%, higher accuracy, 20.48%, 24.73%, 29.32% higher computational time and 30.98%, 26.66% and 21.32% higher ROC analysed with existing models such as personalized English learning material recommendation system depend on knowledge graph (CNN-ELMRS-KG), semantic method for document classification utilizing deep neural networks with multimedia knowledge graph (DNN- ELMRS-MKG), and personalized recommendation of English learning depend on knowledge graph with graph convolutional network (PR- ELMRS-GCN) respectively.

Retraction Note: Memory-guided visual attention generative adversarial network for colorization of nighttime thermal infrared images

Retraction Note: Memory-guided visual attention generative adversarial network for colorization of nighttime thermal infrared images

SAGAN: Skip attention generative adversarial networks for few-shot image generation

The task of producing high-quality, realistic, and diverse images based on a few instances of newly emerging or long-tail categories is known as few-shot image generation. Despite prior works showing outstanding results, the quality and diversity of the outputs are still limited. In this paper, we tackle this problem by presenting a range of innovative fusion techniques based on attention mechanisms with generative adversarial networks. Our proposed framework introduces global skip attention that links matching residual blocks of symmetric encoder-decoder pairs to generate new instance objects. Additionally, we incorporate an alignment algorithm based on spatial transformer networks into our pipeline encoder to address feature misalignment. In the decoding phase of our attention-based decoder, we propose a novel attention mechanism within each fusion residual block, which leads to capturing long-range dependencies in feature maps. An attention reconstruction loss function has been proposed to balance adversarial training between the generator and discriminator, mitigate mode collapse, and guide the generator to focus on specific regions of interest within images. Finally, we apply a back summation to the decoding outputs, resulting in unified features through a weighted combination of similar characteristics. Extensive experiments conducted on five few-shot image datasets demonstrate the effectiveness of our proposed model. The source code of the proposed model can be found on GitHub https://github.com/Aldhubri/SAGAN.

HyA-GAN: remote sensing image cloud removal based on hybrid attention generation adversarial network

ABSTRACT In recent years, with the continuous advancement of remote sensing technology, remote sensing images have found widespread application in various fields. However, remote sensing images are often hindered by the presence of clouds, which can significantly reduce their usability. Removing clouds from remote sensing images has thus become an essential preprocessing step in remote sensing image analysis. While numerous models based on deep learning have produced impressive results in the domain of image denoising, there has been limited research on applying deep neural networks to tackle the issue of cloud removal in remote sensing images. This paper proposes a novel model, the Hybrid Attention Generative Adversarial Network (HyA-GAN), that blends the channel attention mechanism and the spatial attention mechanism and integrates it into a generative adversarial network. By incorporating both the channel attention mechanism and the spatial attention mechanism, HyA-GAN permits the network to prioritize regions that are vital for image processing. This feature leads to an improvement in the network’s capacity to recover data and generate remote sensing images with superior cloud removal effects. When comparing HyA-GAN with other existing cloud removal models on the RICE dataset, it outperformed them in both peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The experiments demonstrate the high potential of HyA-GAN in addressing the challenge of removing clouds from remote sensing images.

Autoencoder-Based Collaborative Attention GAN for Multi-Modal Image Synthesis

Multi-modal images are required in a wide range of practical scenarios, from clinical diagnosis to public security. However, certain modalities may be incomplete or unavailable because of the restricted imaging conditions, which commonly leads to decision bias in many real-world applications. Despite the significant advancement of existing image synthesis techniques, learning complementary information from multi-modal inputs remains challenging. To address this problem, we propose an autoencoder-based collaborative attention generative adversarial network (ACA-GAN) that uses available multi-modal images to generate the missing ones. The collaborative attention mechanism deploys a single-modal attention module and a multi-modal attention module to effectively extract complementary information from multiple available modalities. Considering the significant modal gap, we further developed an autoencoder network to extract the self-representation of target modality, guiding the generative model to fuse target-specific information from multiple modalities. This considerably improves cross-modal consistency with the desired modality, thereby greatly enhancing the image synthesis performance. Quantitative and qualitative comparisons for various multi-modal image synthesis tasks highlight the superiority of our approach over several prior methods by demonstrating more precise and realistic results. The source code will be available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/caohaifang123/ACA-GAN</uri> .

Secure internet of medical things based electronic health records scheme in trust decentralized loop federated learning consensus blockchain

Electronic Health Records (EHRs) have become an increasingly significant source of information for healthcare professionals and researchers. Two technical challenges are addressed: motivating federated learning members to contribute their time and effort, and ensuring accurate aggregation of the global model by the centralized federated learning server. To overcome these issues and establish a decentralized solution, the integration of blockchain and federated learning proves effective, offering enhanced security and privacy for smart healthcare. The proposed approach includes a gamified element to incentivize and recognize contributions from federated learning members. This research work offers a solution involving resource management within the Internet of Medical Things (IoMT) using a newly proposed trust decentralized loop federated learning consensus blockchain. The obtained raw data is pre-processed by using handling missing values and adaptive min-max normalization. The appropriate features are selected with the aid of hybrid weighted-leader exponential distribution optimization algorithm. Because, data with multiple features exhibits varying levels of variation across each feature. The selected features are then forwarded to the training phase through the proposed pyramid squeeze attention generative adversarial networks to classify the EHR as positive and negative. The proposed classification model demonstrates high flexibility and scalability, making it applicable to a wide range of network architectures for various computer vision tasks. The introduced model provides better outcomes in terms of 98.5% in the training accuracy and 99% in the validation accuracy over Medical Information Mart for Intensive Care III (MIMIC-III) dataset, which is more efficient than the other traditional methods.

Predicting Pedestrian Trajectories with Deep Adversarial Networks Considering Motion and Spatial Information

This paper proposes a novel prediction model termed the social and spatial attentive generative adversarial network (SSA-GAN). The SSA-GAN framework utilizes a generative approach, where the generator employs social attention mechanisms to accurately model social interactions among pedestrians. Unlike previous methodologies, our model utilizes comprehensive motion features as query vectors, significantly enhancing predictive performance. Additionally, spatial attention is integrated to encapsulate the interactions between pedestrians and their spatial context through semantic spatial features. Moreover, we present a novel approach for generating simulated multi-trajectory datasets using the CARLA simulator. This method circumvents the limitations inherent in existing public datasets such as UCY and ETH, particularly when evaluating multi-trajectory metrics. Our experimental findings substantiate the efficacy of the proposed SSA-GAN model in capturing the nuances of pedestrian interactions and providing accurate multimodal trajectory predictions.

Multi-scale fusion and adaptively attentive generative adversarial network for image de-raining

Multi-scale fusion and adaptively attentive generative adversarial network for image de-raining

Self-Supervised Attentive Generative Adversarial Networks for Video Anomaly Detection.

Video anomaly detection (VAD) refers to the discrimination of unexpected events in videos. The deep generative model (DGM)-based method learns the regular patterns on normal videos and expects the learned model to yield larger generative errors for abnormal frames. However, DGM cannot always do so, since it usually captures the shared patterns between normal and abnormal events, which results in similar generative errors for them. In this article, we propose a novel self-supervised framework for unsupervised VAD to tackle the above-mentioned problem. To this end, we design a novel self-supervised attentive generative adversarial network (SSAGAN), which is composed of the self-attentive predictor, the vanilla discriminator, and the self-supervised discriminator. On the one hand, the self-attentive predictor can capture the long-term dependences for improving the prediction qualities of normal frames. On the other hand, the predicted frames are fed to the vanilla discriminator and self-supervised discriminator for performing true-false discrimination and self-supervised rotation detection, respectively. Essentially, the role of the self-supervised task is to enable the predictor to encode semantic information into the predicted normal frames via adversarial training, in order for the angles of rotated normal frames can be detected. As a result, our self-supervised framework lessens the generalization ability of the model to abnormal frames, resulting in larger detection errors for abnormal frames. Extensive experimental results indicate that SSAGAN outperforms other state-of-the-art methods, which demonstrates the validity and advancement of SSAGAN.